Mapillary (V4) API - Map feature search - OAuthException

Imagery, data, and integrations
1

Hey!

I’m wondering what possible reasons could be behind the following exception?

{
   "error": {
      "message": "An unexpected error has occurred. Please retry your request later.",
      "type": "OAuthException",
      "is_transient": true,
      "code": 2,
      "fbtrace_id": "AS26d006E6BBsmkJJnBUXnZ"
   }
}

Here is the URL I am trying to request:

https://graph.mapillary.com/map_features?fields=id%2Cgeometry%2Caligned_direction%2Cobject_value%2Cobject_type&start_first_seen_at=2021-01-01T00%3A00%3A00%2B0000&end_first_seen_at=2022-01-01T00%3A00%3A00%2B0000&start_last_seen_at=2021-01-01T00%3A00%3A00%2B0000&end_last_seen_at=2022-01-01T00%3A00%3A00%2B0000&object_values=regulatory--maximum-speed-limit-%2A%2Cregulatory--end-of-maximum-speed-limit-%2A%2Cregulatory--speed-limit-zone-%2A%2Cregulatory--end-of-speed-limit-zone-%2A%2Cregulatory--end-of-prohibition-%2A&access_token=<TOKEN>&limit=2000&bbox=-101.25%2C36.59788913307021%2C-95.625%2C40.97989806962013

Sometimes the response contains a valid result that looks as follows:

{
   "data": [
      {
         "id": "1282906358808750",
         "geometry": {
            "type": "Point",
            "coordinates": [
               -96.937176085639,
               40.821923767334
            ]
         },
         "aligned_direction": 104.43353291653,
         "object_value": "regulatory--maximum-speed-limit-75--g2",
         "object_type": "trafficsign"
      },
      ...
      {
         "id": "276937214262912",
         "geometry": {
            "type": "Point",
            "coordinates": [
               -96.416872969551,
               39.711468384362
            ]
         },
         "aligned_direction": 188.0721253321,
         "object_value": "regulatory--maximum-speed-limit-55--g2",
         "object_type": "trafficsign"
      }
   ]
}

But in most cases, the exception mentioned above is returned.

1 Like
2

Hi @chrisbeddow !

As a Mapillary API expert, could you please help us figure out what the possible reasons could be behind the exception mentioned above?

3

The exception says: “Please retry your request later”. Sometimes it helps. But, for example, similar request for bbox=2.8125%2C50.73645513701065%2C5.625%2C52.48278022207821 fails all the time.

4

v4 API calls fail for me since 2022-01-27. Maybe something’s broken on Mapillary side.

5

… and while writing this it mysteriously works again…

7

Unlikely that this will help. I get this kind of error responses more or less regular when accessing the /images endpoint, and the advice “An unexpected error has occurred. Please retry your request later.” helps — exactly the same request will get a response sooner or later. However, the v4 API got very unreliable the last days (starting last Saturday) and one must be currently very patient to get successful responses.

9

+1 This also happens to me.
My script is doing nowhere near 50k requests per minute.
Even when just test hitting the endpoint using Postman, I’ll get this error.

1 Like
10

+1 for me as well.
Along with this the V3 API allowed us to get data on a country wide basis but now using tiles in V4, this request is pretty infeasible. Have tried using the map_features from the graph endpoint and the tiles enpoint. Any suggestions on how to get data on a country wide basis?

@chrisbeddow

11

It’s often the very first request after 24h which fails in this way. I think we can rule out request limits, unless there is some kind of global rate limit for all of mapillary.

12

@eserte @reichg24 @bjor @aromanov

I see you are including the access token in the URL. It is best to send it in the header of your request.

For everyone, I recommend using this bbox approach, but it returns only up to 2000 items of data.

If you are doing a country level and it is a big country, then you should first figure out the zoom 14 tiles (14/x/y) you need for the whole country, then you want to download them all.

Try the Mapillary Python SDK for this if you’re using Python: mapillary.interface | Mapillary Python SDK

13

Hi @chrisbeddow !

Thank you for your recommendations!

I’m using a map feature search because I need filtered results for a large area. I mean, I’m trying to reduce the number of requests needed to get the data I’m interested in.

Here is a code similar to what I’m using:

import json

import mercantile
import requests
from requests.exceptions import HTTPError

TOKEN = "MLY|7...d"

QUADKEY = "023110"

LIMIT = 2000


def main():
    params = {
        "fields": "id,geometry,aligned_direction,object_value,object_type",
        "start_first_seen_at": "2021-10-01T00:00:00+0000",
        "end_first_seen_at": "2022-01-01T00:00:00+0000",
        "start_last_seen_at": "2021-10-01T00:00:00+0000",
        "end_last_seen_at": "2022-01-01T00:00:00+0000",
        "object_values": "regulatory--no-left-turn--g*",
        "limit": LIMIT,
        "bbox": quadkey_to_bbox(QUADKEY),
    }
    headers = {
        "Content-Type": "application/json",
        "Authorization": f"OAuth {TOKEN}",
    }
    print(json.dumps(params, indent=2))
    response_content = search_map_features(params, headers)
    print(json.dumps(response_content, indent=2))
    print(len(response_content["data"]))


def search_map_features(params, headers):
    with requests.Session() as session:
        response = session.get("https://graph.mapillary.com/map_features", params=params, headers=headers)
        if response.status_code != requests.codes.ok:
            raise_http_error(response)
        return response.json()


def raise_http_error(response):
    if isinstance(response.reason, bytes):
        try:
            reason = response.reason.decode("utf-8")
        except UnicodeDecodeError:
            reason = response.reason.decode("iso-8859-1")
    else:
        reason = response.reason
    http_error_message = f"{response.status_code} HTTP Status: {reason} for url: {response.url}"
    raise HTTPError(http_error_message, response=response)


def quadkey_to_bbox(quadkey):
    tile = mercantile.quadkey_to_tile(quadkey)
    bbox = mercantile.bounds(tile)
    return f"{bbox.west},{bbox.south},{bbox.east},{bbox.north}"


if __name__ == "__main__":
    main()

Sometimes this code returns the following result:

{
  "fields": "id,geometry,aligned_direction,object_value,object_type",
  "start_first_seen_at": "2021-10-01T00:00:00+0000",
  "end_first_seen_at": "2022-01-01T00:00:00+0000",
  "start_last_seen_at": "2021-10-01T00:00:00+0000",
  "end_last_seen_at": "2022-01-01T00:00:00+0000",
  "object_values": "regulatory--no-left-turn--g*",
  "limit": 2000,
  "bbox": "-101.25,36.59788913307021,-95.625,40.97989806962013"
}
{
  "data": [
    {
      "id": "1059714538214858",
      "geometry": {
        "type": "Point",
        "coordinates": [
          -96.548783633697,
          39.189423358123
        ]
      },
      "aligned_direction": 273.93481148483,
      "object_value": "regulatory--no-left-turn--g1",
      "object_type": "trafficsign"
    },
    {
      "id": "1059714571548188",
      "geometry": {
        "type": "Point",
        "coordinates": [
          -96.548449367512,
          39.189418245144
        ]
      },
      "aligned_direction": 274.80228197329,
      "object_value": "regulatory--no-left-turn--g1",
      "object_type": "trafficsign"
    },
    {
      "id": "623827028704848",
      "geometry": {
        "type": "Point",
        "coordinates": [
          -99.096355950596,
          40.699431403659
        ]
      },
      "aligned_direction": 87.11265738214,
      "object_value": "regulatory--no-left-turn--g1",
      "object_type": "trafficsign"
    }
  ]
}
3

But in most cases, it fails with a 500 HTTP Status code.

Traceback (most recent call last):
  ...
    raise HTTPError(http_error_message, response=response)
requests.exceptions.HTTPError: 500 HTTP Status: Internal Server Error for url: https://graph.mapillary.com/map_features?fields=id%2Cgeometry%2Caligned_direction%2Cobject_value%2Cobject_type&start_first_seen_at=2021-10-01T00%3A00%3A00%2B0000&end_first_seen_at=2022-01-01T00%3A00%3A00%2B0000&start_last_seen_at=2021-10-01T00%3A00%3A00%2B0000&end_last_seen_at=2022-01-01T00%3A00%3A00%2B0000&object_values=regulatory--no-left-turn--g%2A&limit=2000&bbox=-101.25%2C36.59788913307021%2C-95.625%2C40.97989806962013

I agree with @eserte that exceptions have become more frequent after 2022-01-27.

14

It will have a max number of 2000 items per response I believe. Maybe the response is too large? I wonder if the response size pre-filter has an effect… but anyway, what zoom for the quadkeys are you using? Best to try smaller to be safe, with more calls which is okay.

15

The same thing happens even for quadkeys of the 14th zoom level. But less frequently.

16

The error may happen even if the result set is zero.

17

I’ve been using the auth in the header.
I do a bbox approach where I split the bbox up if the returned size is == limit.

BUT,
I have changed nothing in my code, and it has been working for the past few hours today.

18

Thanks for releasing the handbrake — API calls work fine for me again.

19

@bjor @eserte looks indeed like a bug, and should be fixed now. Please drop a note here if it’s still not working as expected!

2 Likes
20

It was working nice for a while, but now it’s back to giving me 500 errors for every request. I have a rate limiter set to 5000/1min (though I’m never reaching those speeds).
No body with an error is returned but this is in the response headers:

www-authenticate: OAuth “Facebook Platform” “unknown_error” “An unexpected error has occurred. Please retry your request later.”

Here is an example request url:

https://graph.mapillary.com/map_features?fields=id%2Cgeometry%2Cobject_value&object_values=object--bench&limit=2000&bbox=[-124.848974%2C24.396308%2C-66.885444%2C49.384358]