@pbb I’m sorry I made a mistake about UKM calculation here Global #CompletetheMap III which is probably why the leaderboard shows that you have 200UKM for August.
Actually, CtM doesn’t consider previously captured UKM. As I see many roads you mapped on Saturday were captured before August, therefore they don’t count for August’s CtM UKM, otherwise, you would have earned 315 UKM so far, instead of 200UKM.
I will discuss with @eneerhut that if we should keep as it is or switch to using refresh UKM for this CtM session.
Hi @tao, there must be something wrong here. I captured 60km and 20k photos on Saturday, which I uploaded during the night. All 20k photos have been added to the score, but only around 25-30 UKM.
Here you can see the route that has been registered for that Saturday. I am pretty sure that far more than half of this was unique:
@tao, maybe you can help. I’m not sure who’s in charge or who does what. @eneerhut mentioned that the scoring system was wrong regarding participant scoring (Reply 79 and above), but hasn’t changed anything or responded on the forum or to private messages in 6 days. Anything you can do?
That previous statement was misinformation… @eneerhut never confirmed that statement. That statement also contradicts what the opening post implies when suggesting where to choose your grid.
Regarding the UKM and whether it counts if you map an area that was mapped before the challenge:
Tao said it counts:
Pbb noticed some ambiguity:
Tao responded to state that it does indeed count as UKM, even it was covered before August.
But oops, no it doesn’t:
Maybe there was some miscommunication due to non-native languages? Or perhaps Tao did not realize that despite the sense in allowing all roads to count fresh in a challenge, it wasn’t coded that way.
Fact is, it doesn’t seem like mapping a road previously covered pre-August counts towards your UKM. That means that if you live somewhere totally covered (like I did pre-this challenge), there’s hardly any point in nominating, as you can’t get any points for UKM at all. I suppose that does go along with idea of COMPLETE the Map, but still. It’s not close to an even playing field (geographic anomalies aside).
On the other note I’ve been working to get answers to, the current scoring system:
It’s quite clear to me that the current system is so very flawed that people in large cities with willing people and the right social tactics can game the system very easily to get their points aggressively boosted. If you look at Budapest (no offense to you guys, but you have the most people), a lot (7) of the (currently) 19 participants have fewer than 30 photos. That’s not really “participating” in my book. The point of the participant multiplier was to
which is not what is happening with some of these people. I think it’s not exactly fair to boost one’s score 10% even if your raw score was 500 and theirs is 0.01.
Then there is the issue of the broken multiplier, where if a mapper contributes to multiple grids, the highest participant multiplier is applied to ALL of their photos and UKM!!! This happens even if the majority of their points is earned in a different grid. Edoardo admitted that this is not correct:
That was a week ago, and I have heard nothing about whether it will be fixed this challenge, which I definitely think it should.
Thanks @pkoby, you are absolutely right. I had misunderstood @tao’s reply, since I had also uploaded some very old photos that same week. But reading it again I have to agree with your interpretation, it now seems that old coverage does block for gaining new UKMs.
To your list of comments to the scoring system, I want to add one more point; there is no limit to how many photos you can take per second. Some take frames from video, and are scoring big time just based on the number of photos they upload.
I think we can make it so that the next challenge looks at coverage regardless of whether or not it is unique.
But for this challenge, the instructions did outline the need to nominate a location without much existing coverage to maximise your score.
Remember that the grid will be centered around the latitude and longitude that you provide, so choose a location that you can easily travel around if possible. If you want to maximise your score, look for locations with a large amount of unmapped roads, paths and trails.
Thanks for making a summary of the issues and discussions thus far. Definitely agree with a lot of what you have said.
UKM
The instructions were correct on the calculation of UKM, but there was contradictory advice in subsequent answers. This was my fault because Tao was responding with reference to our API and what is possible to calculate from there. The ability to calculate UKM within a given month relative to all-time UKM is only a recent addition that was not reflected in the way the challenge was set up.
Bottom line: is that only contributions on un-mapped roads count for UKM. We will make use of the new API call to factor in coverage unique to that challenge period.
Image scoring
There is definitely a bias towards a more frequent image capture interval. Our computer vision team prefers overlap between images as it allows for better reconstructions. That being said, a balance in the scoring methodology needs to be struck so that lower quality video at a high frame rate is not being over rewarded relative to higher quality imagery taken at a less frequent interval.
Bottom line: Open to feedback for future challenges. Looking at how we can find a balance using our existing API.
Participant multiplier
I think everyone is in agreement here that the multiplier is not serving its intended function. My priority now though is to ensure the challenge remains consistent with the outlined instructions. We will then have a period of feedback and review during this month to make sure that future challenges are operated more fairly.
Bottom line: Participant multiple will remain for this challenge and apply to all contributions for the affected user. I am going through participants and will do my best to determine whether this has been abused in any manner.
Overall scoring
In light of these discussions, I am going through the scoring and will be on the look at for any anomalies and/or unfair manipulation of the scoring. This is in accordance with the instructions though and not subsequent feedback.
The feedback thus far has been fantastic and we’re really glad people have been so active and thoughtful in their contributions of imagery. The best way forward is to wrap up this challenge, consolidate feedback and work towards a much fairer challenge that the community can agree on.
Stay tuned on the challenge wrap up, and let me know in the coming hours if there is anything else to review before results are released.
I’m Javier from Tenerife. First of all sorry for not taking part of this forum before. Too much concentrated in the challenge, maybe.
I would like to give congratulations to all the winner and recognize the effort of every participant. I would like also to share some thoughts about the scoring system hopping it could improve the next round if it’s not too late.
Regarding the UKM counting previously mapped roads or not, I think it should take into account not mapped roads and also roads which have not been mapped since a given time, let’s say one year, for example. This way encourages coverage but also update of older images keeping record of changes in the area. It would be great to take into account separately both directions of the roads and also that a 360 camera captures both at the same time.
Measure the image quality and reward it would be a great improvement.
The absolute number of images should exclude duplicates. It have been discussed if taking many images per second is fair. I think it’s as long as the images are not duplicated in the terms flagged by the mapillary tools. I could upload more much images taking video at 30 fps and disabling the flag duplicates option, but most of this images would be the same in terms of position and direction. Does the platform take this into account? Also if I take two images separated by one kilometre, does they count as 1 UKM? I think it shouldn’t.
Finally, about promoting the collaboration, instead of multiply by a factor that depends on the total number of participants, I would take into account their global contribution. I propose this method:
personal score = your total unique KM contributed + your images / 1000
grid score = group total unique KM contributed + group images / 1000
your final score = (personal score + grid score) / 2
This way, your contribution is augmented with the collaboration of others, but not if other contribute too few images and never more than an exceptionally great indiviual contribution.
Some examples:
Lonely participant with 100 UKM and 100000 images = 200 points
Group of two each with 50 UKM and 50000 images = 150 points (increment from 100 personal score to150)
Group of two,
user1 with 75 UKM and 75000 images = 175 points (from 150 to 175),
user2 with 25 UKM and 25000 images = 125 points (from 50 to 125)
Group of two,
user1 with 100 UKM and 99990 images = 199,995 points (increment only from 199,99),
user2 with 0 UKM and 10 images = 100,005 (increment from 0,01)
Group of 11,
user1 with 90 UKM and 90000 images = 190 points (incremented from 180)
user2-11 with 1 UKM and 1000 images = 101 points each (from 2)
I’d welcome criticism or improvements from all of you.
. Now fixed 
in the form of a prize 