Australian sign type not in database

Hi. Just letting you know about a traffic sign that Mapillary doesn’t seem to detect.

In Australia, we have speed derestriction signs. Their meaning differs between states. But basically they signify that the signposted speed limit ends there, and the limit changes to the state’s default speed limit. ( Speed limits in Australia - Wikipedia )

Would you be able to add these to the signs Mapillary detects, please? :folded_hands:

Looks graphically and semantically close to regulatory--end-of-prohibition--g1 or regulatory--end-of-speed-limit-zone--g1/2/3. :grin:
According to Wikipedia, this sign in Australia originates from the Vienna Convention on Road Signs and Signals but its semantics apparently have shifted or drifted recently in some Australian states. Imho even if the sign’s semantics cover lifting speed limits only then regulatory--end-of-prohibition--g1 basically continues to apply because a speed limit prohibits driving faster than the posted speed. But, this is just an academic discussion. :person_shrugging: AI probably just needs more training or a little bit of threshold parameter tuning to resolve detections to regulatory--end-of-prohibition--g1 more often in this case. In the end, it is up to the data consumer how they interpret a piece of data. :wink:

1 Like

Hi, @GITNE. I think I get what you’re saying. regulatory--end-of-prohibition--g1 sounds correct, from your comments, but I can’t really take it in right now after the week I’ve had, sorry. :sweat_smile:

For clarification, in case you misunderstood me, Mapillary isn’t misidentifying this sign in my images. It’s simply not showing a detected traffic sign of any kind.

What is the threshold parameter that you mentioned? I’m curious what exactly is involved in training the model to identify a new type of sign. I assume it has “learned” that Australian speed signs are usually a rectangle of a particular size and aspect ratio, with a red circle around a number. Does that make it relatively easy to introduce a new variety of Australian speed sign, same shape and size but different circular graphic? Or does every new sign require the same amount of human effort and training imagery?

I hope you recover and get well soon. :wink:

I think Mapillary engineers understand what you meant. :+1: Thanks for pointing it out.

What I was trying to say was that the traffic sign you have pointed out is already known to the detection AI (input side patterns) and it is also on the list of detectable traffic signs (output side labels). The reason why detection rates are so poor or perhaps just poorer for the Australian traffic sign variant compared to other variants is probably just due to the threshold setting for this particular regulatory--end-of-prohibition--g1 detection. But, this is just my guess based on observation. I have spotted many regulatory--end-of-prohibition--g1 false negatives in my images too. So, thank you for bringing this up.

Of course, I cannot speak for Mapillary but in general these pattern recognition AIs do not solely work based on a training dataset but also have a set of threshold parameters for detections, that is detected patterns. It is usually like a [0, 1] value that indicates at which level a pattern becomes a true positive (a detection). Training an AI model is computationally extremely costly (and data demanding). Furthermore, pattern recognition is inherently non-binary. Hence, these are just two reasons why AI model designers put in threshold parameters for detections. Threshold parameters make detections binary at a certain level and it is much cheaper to tune a threshold (output) parameter than having to retrain an AI model. There are also other reasons for threshold parameters in AI models.

AFAIK the Mapillary traffic sign AI does not distinguish between national or jurisdiction traffic sign variants per se, except for the g1/2/3 classes. Mapillary traffic sign detections are also the result of a multi-tier scene labeling process.

Also worth pointing out that “GRID” (black text of yellow background diamond shape) warning signs (cattle/stock barriers made from a metal grid across a road) are detected/displayed as “PARE”. Same problem with “FLOODWAY” as well. There might be some others.

Oh yeah, various advisory signs are displayed as mph and 40km/h school zones (with the words above the usual red circle) are sometimes interpreted as the old imperial black text on white background “Speed limit 40 mph”.

“Trucks Turning” black on yellow diamond shape is interpreted as something I cant read.

I generally don’t show anyone the sign recognition output for Australia as there are enough errors to make it all quite embarrassing.

“DIP” also displays as “PARE”