Quality score: unexpected results

Sometimes the quality score assigned is not what I’d expect - will list examples in this thread.

Mapillary - while not a bad image, the right hand side is a bit blurry. I would not have expected a score of 5.

Thanks for making a note of this @Richlv! If you can list many examples it should be helpful for our review.

Thanks, couldn’t figure out the best place to list those before and did not make notes - but will add any I spot here from now on.

Is it possible to pay someone for that work ?
Examples are very easy to find.

Mapillary - would score higher than 3
Mapillary - would score higher than 2
Mapillary - would score higher than 2

Same with many (most) other pictures in the sequences in this area/time.

Contrast the images above with Mapillary - I’d even mark this one as 4 instead of 5.

Mapillary seems similar, but with a lot of hood in the pic - but even that one got 4.

Mapillary - would score lower than 5.

There’s some useful building detail, but in general I’d probably score this one at 3, given how much the bus is obscuring :slight_smile:

Would appreciate if the quality slider were two-sided, so that one can exclude good quality pics, see where the lower quality sequences are, then include those in a journey.

1 Like

That would include areas where there are both good and bad quality pics, though.
Wouldn’t just excluding low quality pics identify blank areas?

Would rate Mapillary higher than 2.

A little bit of dashboard is too severely punished. The tilt of my camera is very sensitive. I aimed at having a little bit of dashboard to be sure that I don’t picture too much sky.
Too much sky on the other hand is not punished at all.

Pictures with a smartphone from a car regularly obtain a 5/5.
Nevertheless it is often not possible to read small not blurred text in those pictures.

Snow cover or falling snow?

I guess I’m not that much concerned about the quality rating, except to get better results from filters :slight_smile:
Falling rain/snow make some detail in the image harder to see, so it makes sense to drop quality score a bit - if rain/snow can be detected, it already is lowering the clarity enough.

Snow cover is a bit more complicated. On one hand, it does hide detail/features. On the other, you already mentioned regions where snow cover is there for most/all of the year. And in other regions, it can be very useful to see what a place looks like with snow. Unless/until Mapillary decides to publish the detail and reasoning, it’s all just guesses.

Would rate Mapillary a bit lower than 5.

Would definitely rate higher than 2. I’d put 4 on this one.

So they might indeed punish snow cover. That would be sad.

Really, Mapillary algorithm, 1? :slight_smile:
I’d go with 4.

Mapillary - definitely higher than 2, at least 3, maybe 4.

Same day. Same time (lighting). Same camera. Same speed.
Just the opposite direction.
Mapillary - 5
Mapillary - 3

OK, if this gets quality score 1, I think that’s enough feedback in this thread - closing the topic :slight_smile:

1 Like

This is helpful. Thanks for collating these examples!