Hi all,
I’ve noticed that there are some users who are rising in rankings by uploading thousends of (almost useless) low-resolution photos. In my opinion it would be better that the rankingsystem counts bytes instead of quantity of photos.
Look at this quality for example: https://www.mapillary.com/app/stats?user=skippern&pKey=46xNchQsmbMaWhXUCbjh0g&lat=-20.675711297071132&lng=-40.502843660999986&z=17&focus=photo
In my opinion are those pictures almost useless with this resolution.
Does the photo sequence you linked show useful information? Yes.
Is everything in the photo easily identified and reasonably clear? Yes.
Are the photos blurry, dark, or extremely low resolution? No. These are 1080p. Shady conditions and high speeds might contribute to a low bitrate, though.
Do the leaderboards matter. NO.
There are plenty of photos out there that I would agree are not useful (e.g. much of this sequence: https://www.mapillary.com/app/?lat=33.81304&lng=-117.919177&z=17&focus=photo&menu=false&pKey=4tJ7OoEgJm0ewp2K5eoMtQ). But unless people are actually trying to vandalize the database with fake photos, I don’t think it’s a problem. This has been discussed on the forum before, and the general consensus is that something is better than nothing, and as long as you can extract information from the photo, e.g. in an OpenStreetMap context, then it’s fine.
The referred images are not high resolution but they contain a lot of usefull information and are much better than nothing.
I agree that there are a lot of very bad images in Mapillary and I have uploaded a lot. Again: They are better than nothing.
I completely agree that the scoring system is not fair. A month ago I spend a lot of time and money producing about 1200 images: 1000 sphrical and 200 with my pro DSLR camera. It will count the same as putting my ultra low res action cam on my bike to work and home again but has a very different value.
I would prefer a system where humans would score images and sequences on both technical qualities, how interesting or beautiful the image is and other parameters. It should be used to select the best images for people who casually browse, but also help scoring. Scoring should also be done depending on how much your area is already covered. But we can which.
So don’t look at the score board. I think it is broken and I am currently #13.
This is great feedback, thanks! We’re thinking a lot about how we can provide feedback and encouragement not only on number of photos but also as you say on the quality. Though, we need to spend some more time on it so all input is great.
Also consider that those people using cameras with a 0.5s delay will naturally have more photos than someone with a 3s delay. Some cameras, like a GoPro or maybe a Garmin Virb, have great resolution and really fast delays, which could have one user uploading six times as many photos.
Likewise, a spherical photo is essentially 24 normal photos (if using something like the Google Camera App), or one quite high-res photo. Either way, it provides a lot more information than a straight photo, but many cameras have a much longer delay, like the Ricoh Theta S with 8s.
I agree there is lots to think about here. It may be too much to want to work out “one true measure” of activity and maybe the way to go is to create alternative leaderboards. Number of photos, meters covered, mapping minutes, 360 photos, quality/beauty of photos… etc. As Sandra mentioned, we’re working on it so don’t hold back any ideas! =)
The list of captured images by user shouldn’t be labelled as ‘Leaderboard’. The number of captured images doesn’t really account the value of the contributed images. The (technically) quality of an image is only one aspect of a huge list of indicators.
The following would may account the quality:
To represent all theses values in only one integer isn’t a great idea. I would recommend a badge system instead (like available in discourse (software used by this forum)).
You could ‘earn’ badges for activities like
Also capture the flag badges would be possible:
Afaik Mapillary has enough information about the images to realize all these badges easily. Finally you could assign points to all badges and create a leaderboard on top that really accounts the quality of the images.
PS: There could also be a platform (maybe git repo) where the community could commit new badges.
First of all, this sequence referred to was processed with outdated versions of FFMPEG, that unfortunately detorriated the image quality, and was not detected by me until a new version of FFMPEG had solved this.
My process is to record movies while driving, using action cameras set to “record while moving”, and post-process to image sequences using FFMPEG and a series of python scripts. Both my cameras are set to record in 1080p (sequences before September 1 was recorded with one camera), and camera set up are to cover as large area as possible, hopefully to cover 360º sometimes in the future. Quality on images might vary as the cameras are mounted outside on a moving vehicle, mist, rain, dust and speed will all have a negative factor on the images.
Leadershipboard is not important for me, I seem to be the most productive in Brazil, but if somebody was to dethrone me, than I’ll rest in peace that I am not the nerdiest nerd in Brazil any more.
My goal is to have a coverage that competes with Google Streetview, and that this coverage can benefit OpenStreetMap.
If people are bothered that I get a high position on the Leadershipboard, than I invite them to cover the areas I currently are covering, so that I don’t need to.
I think you are doing the right thing, @Skippern. Mapillary needs both quantity and quality and your quality was pretty ok for Mapillary. One of Mapillarys strenghts is that it covers areas where Google does not come, and have much more frequent updates. A weakness is that the images may be in bad weather or just of various kinds of inferior quality. Even bad images are valueable in many applications so they should be included.
The leader boards tells part of the story, and I hope Mapillary will make it possible to make some quality measurements or markings, both for leader boards but also to present good images first.
PS. @Skippern, if you consider 360, then look at http://eu.360.tv/en/ if you can mount it 30-40 cm above your car. Perhaps with some heavy duty magnets like Magnet Mount with RigWheels RigMount High Power Magnetic Camera Mount . You will still get the same image quality as today, but it will be in 360 
The Giroptic has some problems though, but hopefully it’ll be fixed in an upcoming update. For instance the GPS is not enabled yet (but you can use a GPX tracker). I’m adding support for it in the iOS app currently and it supports the OSC standard, it just doesn’t really follow it and has some issues. Nothing that can’t be solved in a software update though.
@tryl Absolutely interesting option, currently working with 2x VIRB, and looking into multiple VIRB options, but a Giroptic would give me continuous 360 coverage.
Problems with GPX in Giroptic can easily be handled with multiple sources for GPX, currently both my VIRBs + one Nüvi is recording GPX, and the tracks combined for processing. Due to Snap-to-Road in the Nüvi I am thinking of getting a dedicated GPX logger, and not use the Nüvi in the process at all.
Currently aiming at coverage, and usability for OpenStreetMap, but when re-entering areas already covered quality will be more important.
@Anders will future updates of the iOS app also be able to control and monitor VIRB XE? And will it be possible for the future to also gather GPX from the phone while recording with external cameras?
@Skippern Virb XE is already supported, and also multi-cam, see Support for Multiple Action Cameras - The Mapillary Blog. For the Virbs we use the built-in GPS though, figured it would be better since it’s most likely to be on the outside of the car (clear sky) and can also be placed ~1-2m from the phone (giving an error),
Regarding Giroptic, support for it in the iOS app will be out soon. Just now actually managed to get timelapse mode working and that is capable of 1 capture/s, which is quite nice.
@Anders I would like to record GPX with additional GPS receivers as I have had problems with sequencing and gaps in the FIT files recorded by the XE. Since I work with tropical conditions where interference on GPS signals might interrupt or offset GPS, it is often a good idea to have recording with multiple GPS units since different units handle the outages differently.
Most outages from scintillation is very short, i.e., 30 seconds - 2 minutes, but because the way units mark satellites as unhealthy can take up to 15 minutes before the unit records correctly again. I have seen the Elite and XE have different ‘back in operation’ delay, so other external sources of GPX could reduce the lost time.
I use the VIRB app for monitoring at the moment, but will have a look at the support in the iOS app shortly, looking forward to test Giroptic, I might have an opportunity in November.
Very important, yes?
Absolutely! We’ve been considering this wider topic - how to induce people referring people and see those links of connection. All of your thoughts are very welcome!
How about ranking per unique distance, i.e., running the same road multiple times will not count? I know that kind of counting will partly penalise me because several tracks follow the same, or almost the same route for some part.
Seeing your friends activities, can ignite the competitive spirit in some people… “My friend xx managed yy photos last week, I better do more than him this week”
Also would be cool to be able to see your friends tracks on the map in a different colour, or see the points where your tracks crosses those of your friends.
I agree that the more interesting areas should be worth more, be it on the leaderboard or however contributors are rewarded.
This last week I spent two days in the area of Mammoth Caves National Park in Kentucky.
My numbers may be down this week from last week (last week I was the leaderboard top scorer with one point six million pictures)
I think this week’s pictures should be worth more.
In general I headed west until noon then came back east in order to keep the sun behind me.
As twisty as the Kentucky back roads are there are a few places where I was going into the sun, however, the vast majority of this week’s pictures were of good quality on small back roads where no one had mapped. The only green down there when I started were the still pictures I took for Google Earth on my last trip down there.
I did cover the interstate going down and back. These pictures have very little of interest in them. I was able to cover one leg of interstate that had yet to be turned green. The big roads just don’t compare to the back roads.
I agree that interstate pictures should not count for as much as back roads.
Covering already covered areas should not be worth as much as new areas.
Pictures going into the sun or with glare or mounts showing should also be penalized.
I think we as contributors should always try for the “best” possible pictures. We need to pay attention to lighting, ideally going west until noon then coning back east after the sun hits it’s zenith. Going north always seems to give possible results.
We can’t always do this.
Some highways are inevitable in our travels, hopefully when you get where you are going you will have some time to explore the local area and the back roads.
The explore feature is great for finding new unmapped places.
If it is for the leader board, the badges or just your own good feeling, we need to always be looking for new interesting places to map with nice clear informative pictures.