Error when harvesting images via API: lexical error: invalid char in json text

Mobile apps
1

Hello all,

I am having a most pesky issue as of recently - I am harvesting images off Mapillary (via RStudio 1.1.442) based on selection criteria. My curl request basically looks like this:

 for (i in 1:nrow(spdf@data)){             #start loop to go through the rows in my dataset, which represent the plots, over which I am basing the search criteria

    # get the lon and lat of centroid
    lon<-spdf@data$centroidLon[i]
    lat<-spdf@data$centroidLat[i]
    
    #specify the growing season  
    start_season<-"2017-03-01"
    end_season<-"2017-10-01"
    
    # do the harvest through the mapillary api
    res <-GET(paste0( "https://a.mapillary.com/v3/images/?",
                      "closeto=",lon,",",lat,
                      "&radius=",500,
                      "&start_time=",start_season,
                      "&end_time=",end_season,
                      "&per_page=",1,
                      "&client_id=",clientID))
    
    # read the results as text
    json <- content(res, as="text")
    
    # read the result as JSON and store the results in the m_parse variable
    m_parse<-fromJSON(json, simplifyVector=TRUE)
    
    if (!is.null(m_parse$features$properties$key)){
      # store the useful information from the m_parse variable into the spdf data frame
      spdf@data$key[i]<-m_parse$features$properties$key
      spdf@data$ca[i]<-m_parse$features$properties$ca
      spdf@data$captured_at[i]<-m_parse$features$properties$captured_at
      spdf@data$lon[i]<-m_parse$features$geometry$coordinates[[1]][1]
      spdf@data$lat[i]<-m_parse$features$geometry$coordinates[[1]][2]

All is good, but every now and again I get the error:

No encoding supplied: defaulting to UTF-8.
 Show Traceback
 
 Rerun with Debug
 Error: lexical error: invalid char in json text.
                                       <html>  <head><title>502 Bad Ga
                     (right here) ------^

This is problematic as the script is quite lengthy and is put into a function that runs it many times. So when it crashes somewhere in the middle due to this error, I have to run it all from the beginning.

I get what the error is telling me, but no matter which rjson library I use, it still rears its ugly head (I usually use jsonlite). This leads me to the conclusion that it might be an issue of deep nesting or some wrappers put around the data by the service providers.

Can anyone comment on this issue and how to work around it?

M

3

Thanks for reporting @momut1. The best channel for this sort of thing as support@mapillary.com.

4

Hi enerhut, thanks for the info, I shall contact them and post the solution whenst it arriveth.

Bless!

5

Hi momut, as I remember, these 5xx errors were frequently happening due to the high database load a couple of weeks ago. Now it should be back to normal. Sorry for the inconvenience.

These errors are still possible to occur by a little chance. To make the application robust against these errors, it’s recommended to check the response’s HTTP status code (regardless of the response content): if it’s 502, 503, or 504, then resend the query after a few seconds.

Tao

1 Like
6

Ciao Tao,

Thanks for the reply. Indeed the error doesn’t persist and after a few refreshments and running the request again it goes through. It is indeed 502/3/4 errors.

I was simply interested why this happens and if there is something on either side (mind or yours) that can be done to ameliorate this.

M