Web uploads not arrived: cut at 999 images

Past week I uploaded several large sequences using the web interface. Now I see that only the first 999 images are published?

What happended to the hundreds of other images I uploaded?? There are no images waiting to process anymore in my account.

(My sequences contain mostly more than 2500 images each)

2 Likes

Have experienced the same problem with the web uploader in New Edge recently.

Had in previous weeks found that of be it 2500 or 4500 pics in a batch, only some 999 were shown on the world map, even if I waited for like four days.

In the early hours of 16 June uploaded four smaller batches, of 995, 990, 986 and 978 photos respectively; when returning to the web uploader in the morning found that the Publish button referred to only 957, 934, 911 and 887 pics to be uploaded.

Had a similar experience a few days earlier - at which time I believed it might be my mistake, didn’t make note of date/time - where of 991, 956 and 774 pics only 692, 711 and 596 were to be published.

Similar for the Desktop uploaded, where several sequences haven’t appeared yet, in spite of having been uploaded ten or more days ago and Status in the app states ‘Complete’ - will go over that list later, and specify which uploads have so far failed to appear, which only appeared in part.

@Brenna, @asturksever : would you be able to bring this to the attention of whoever does clever things with the upload server, please, as there are ten of thousands of pics waiting on my end, and as my internet account only supports 295kB/sec with a 100GB/mo cap (volume between midnight and 10am counting as half), any opportunity missed is “wasted forever” 


Met vriendelijke groet (Dutch for ‘with friendly greeting’),

2 Likes

You are right. Now I tried to upload partial sequences below 999 files, after the upload the system even ‘forgets’ a couple of files at the beginning or end of that sequence. Very strange things are happening. New upload procedures with bugs?

1 Like

@micmin1972 @koninklijke

Recently, Mapillary has started to check duplicated files in order to prevent duplicated files. All uploading tools check duplications and uploading session fails if at least one image is a duplicate. You will be getting notification email regarding this file so you can remove these files for uploading your files without duplication. As you might know, Web and Desktop Uploader uses mapillary_tools as "engine” there this duplication rule applies for uploading methods.

I strongly recommend you to use the latest version of Desktop Uploader (1.2.7) which filters duplicated images and does not upload these images to Mapillary if you have doubt that you might have duplication in your uploading folder.

If you are keen on to use Web Uploader. Web Uploader works simply like below;

1- Browse images
2- Review images
3- Upload images
4- Publish images

If there will be any duplication among your files, this session will be failed, your images will be uploaded but not published on Mapillary. You will be notified by email if there is any duplication with the file name and image key.

There are two ways to handle this duplication issue.

1-Removing your duplicated file from Mapillary

You can delete duplicated files from Mapillary. I’m aware this is workaround solution at the moment.

  • Go to image using image key which you receive on the duplication email
    Mapillary
  • Delete duplicated image
  • Hit Try Again on Web Uploader

If you have " A server error occurred " Clicking Try Again can help you to publish your images unless there are no duplication in your uploaded files.

2-Removing your duplicated file from your upload.

This solution requires re-uploading all imagery one again without duplication.

  • Refresh your Web Uploader session on your browser
  • Delete all file which are uploaded
  • Upload all images without duplication
  • Publish your images

Please let me know if this helps. If you think that your successfully uploaded your images but they are not published yet, please contact support@mapillary.com for further clarification.

Best,
Said

1 Like

Hi
I am absolutely sure I removed my duplicates [consecutive images within 6 meters] before uploading. So it does not explain the 999 upload ‘limit’ and forgetting quite some images.

Maybe can you tell me what the ‘duplicate’ criteria settings of the upload engine are?

Btw. I never received any emails about specific files being a duplicate.

1 Like

I have a similar problem: only the first 999 or 1000 files are uploaded. Even more, after uploading other sequences, since several days ago I received the following error when I try to upload other images:

Processing of your upload failed

Some of the images in your upload failed to process. This means your upload will not be processed further and will not be published. Please remove the following images that failed to process, and start a new upload without them:

and a list of files, from the number 1001 to the latest file. But I cannot delete any files in the upload, the only action button is “Start new upload”, which raises tghe same error window once and again.

1 Like

Mapillary / Facebook tries to limit the upload of duplicates (which I can understand very well) but the implementation of it seems not very well right now.

E.g. I always remove my duplicates within 6 meters in driving direction using python before uploading. So I checked which files were skipped in one of my latest sequence uploads and they were images in the reverse direction on the same road. So did their algorithm labelled them ‘duplicate’? Strange.

1 Like

And now the web uploader does not work any more.
I dare not use the desktop uploader.
It is going down the drain.

1 Like

web uploader does still work for me today, but there seem to be a lot of changes that have been applied to their uploader. But It works for me when:

  • I prepare a batch less than 999 files
  • Prepare the batch manually before uploading so no duplicates are present

and even then several images are removed automatically after uploading, not sure why


1 Like

I am stuck in a vicious circle.
“Some of the images in your upload failed to process. This means your upload will not be processed further and will not be published. Please remove the following images that failed to process, and start a new upload without them:”
They really want me to search 30 files among 999 files.
Have they gone out of their mind ?

It is time to fire some people at Mapillary.
Releases are untested, and make things worse.

3 Likes

What are duplicates ?

1 Like

Sorry for the frustrations everyone. I know the current workflow to handle this is far from ideal. We’re working to resolve this.

Having a system to handle duplicate uploads was very important for us to improve the quality of data on Mapillary, but the way it has been implemented needs work.

In the meantime, I’m curious to get a sense of what are the reasons people prefer the Web Uploader over the Desktop Uploader. Apart from those on Linux who have to use the Web Uploader, are there other things about the Web Uploader that make it your uploader of choice?

2 Likes

I think you should make clear that the Desktop Uploaded is the preferred method for uploading. Last time I tried to use the web uploader, for no good reason, to be honest, and it was slow, until it failed. Then I used the Desktop Uploader and it worked fine. Maybe you could mention this in the web uploader interface.

1 Like

Do you have a sequence where images have been removed so I can check?

When I started contributing images for use in perfecting OpenStreetMap, Mapillary was suggested as the best place; at the time there was the choice between a legacy uploader, the web uploader, and some - -was it perl or python?- tools which were beyond my knowledge to use.

At some point a desktop uploader was mentioned in forum posts, tried (was it version 1.9.n?) which took very long (couple of hours) to prepare say 4400 images for upload, with no discernible gain in upload speed / reduction in time needed to upload, and in addition it created one folder per image with in each folder something like 23 files charting progress : deleting the 100.000 or so files from a 4400 pic upload takes yonks - irrespective whether the computer deletes 100 or 300 files per second.

= = =

Tried v 1.2.4, where I found that some sequences just didn’t appear on the map - not even after like 3 or 4 days.
Found similar for the web uploader : in both instances like 999 or 1499 pics out of be it 2500 or 4400 were on display after three days.

Now diverge from desktop vs web uploader to duplicate removal : took the time to find the last pic in a batch which appeared on the map, then re-uploaded the remainder - there may well have been a duplicate at the junction between shown, and not shown on map. As long as it concerns a couple of pics overlap : just delete those : they’re not malignant, are they?

And if I wanted to inflate the number of pics : upload the daily ride to the shops - different shop each day - would be far more effective.

Have couple of days ago sent a mail to support, c/o Saïd, with a number of incomplete sequences; some of these have now appeared, others seem lost forever.

Hope this helps? If not : ask!

Dear Peter,
Sent a mail, detailing a number of mislaid sequences to support@mapillary on Date: Sun, 21 Jun 2020 17:19:09 +0200 ; would be happy to copy+paste the list here?

= = =
Copy+paste from mail mentioned above :

= = =

Dear Support (frequently Dear Saïd),

Mentioned in a reply to a forum post on Saturday that parts of some series of uploaded photos were missing, and promised to look into this and compile a list at some point. Herewith a list of batches of photos uploaded over the past weeks; the date is the date the photos were taken.

This message was researched and written on the Saturday evening, held back to read again the next day, check for inconsistencies; have now seen Saïd’s message to Micmin and myself, not amended this message as I will respond to that separately.

Mapillary Desktop uploader

2020-04-15 - looks complete

2020-05-02 : more missing, but looks like uploaded via Web-browser uploader (as I can’t see the . mapillary folder) : Mapillary does show from the start of the batch in my folder at 16:01 till 16:21 , and 16:51 - 16:57 ; thus missing 16:21 till 16:51 - south of the river Rupel, extends till east of the river Zenne, where there is a new flood control ‘polder’.

2020-05-02 : the 4424 item return journey along the Schelde river from 17:00 till 18:54 was uploaded according to the log files in the .mapillary folder, but only the 17:18 till 17:27 pics are shown on the green dots/line : any news on the remainder, please? Desktop Uploader shows the batch as Complete 


2020-05-07 - incomplete : Desktop Uploader shows two batches, 3575 and 2522 pics = 6097 , while my folder contains 6843 ; Uploaded status = complete, gap in M’s green dots/line between 17:52 and 18:00 - missing some 450 pics, [hence more must be missing (correct, had the GoPro running while waiting at the level crossing, from 18:21 till 18:26 = approx. 300 removed as you could barely see the grass grow 
 )]

2020-05-09 - looks complete ; three batches : 606, 2350 and 5325 hero 2018 pics, can’t see the TG5 single shots ??

2020-05-14 - looks complete ; 830 + 6434 pics, can’t see the TG5 single shots ??

2020-05-16 - looks complete ; sequence cut has (fortunately!!) overlooked the TG5 single shots - still kilometres of connecting lines, but may be the only way for OSM mappers to find those 


2020-05-30 - looks complete ; two batches : 3847 + 3805 hero 2018 pics, can’t see the TG5 single shots ??

2020-06-01 - begin to see where things fall down : the single shots with the TG5 will have the same date/time and lat/lon values, as they’re geotagged from the same Garmin etrex10 track-log- which is the most accurate I can get; take 2020-06-01 10-37-49 : in the Hero 2018 batch it shows just a narrow tarmac strip with a wide sand verge ahead, while in the TG5 batch it shows a sideways shot of a forest track, fence and gate.
OK. seem to have put a finger on the ‘lost’ TG5 pics on stretches of roads where I’ve taken single shots of particular details along or to the side of the road while the Hero was also running and either looking straight ahead or also pointed towards the same subject? Or did I?

2020-06-04 - here we have both the single pics connected by kilometres of lines (@sequence cut? actually, those lines may be the only way for OSM-mappers to find the TG5 pics!!) - does that disprove 


That’s the end of the entries from the Desktop Uploader History page checked; now on to the various SD-cards and USB-sticks which I used to upload pics from a very energy efficient Pentium Silver 2-in-1 flip-book:

Mapillary Web-based Uploader

2020-04-18 : ‘bruyns gedaan’ : part visible, part not to be seen on mapillary’s green lines/dots yet ; have split that into ‘part done’ and ‘2do’. visible 15:27 - 15:44 then resumes at 16:24 - 16:41 ; there also are pics 15:44 till 15:51 and 16:02 - 16:07 which show ‘new’ streets , the uphill struggle 16:07 till 16:11 barely adds to the sequence taken a couple of days earlier - different weather, that’s all in retrospect - 16:15 - 16:19 is a different path, looks easy to cycle down to start with, then narrows and gets steeper, thus needs to be marked on OSM as unsuitable for general cycling , the rest 16:19 till 16:24 again largely the same as before apart from weather.

2020-04-18 bruyns paepe 2do : 16:41 - 16:44 is new , as is 16:54 - 16:57 which looks in more detail at the new development (the footpaths to the communal entrances are shown as service roads on OSM, but they’re not ; there is a local OSM mapper in that area, so these may be superfluous) ; from 17:21 you get a back street, a new cycle path, and a road which was public but where part has now become private / no access. that series continues till it meets up with the beginning at 17:47, and it isn’t clear why it isn’t shown?

2020-04-19 : Mapillary shows 16:03 till 16:20 , but full batch was till 17:37 : what happened to the rest, please?

2020-04-26 : looks complete

2020-04-30 - starts at 14:54 and continues to 15:49 ,

2020-06-06 Zoersel hero 2018 : visible from start of batch till 12:33 , and from 12:43 : bit in between is also now a new ‘fietsstraat’, despite having been photographed some weeks earlier, when it wasn’t yet. [NOTE : this concerns the eastern part of the Kasteellei in Wijnegem, south of the Albertcanal]

2020-06-06 Zoersel TG5 : visible on Mapillary till 14:28 , then from 18:27 onwards (final pic shows how people thought about safety over the centuries : initially, a Holy Mother statue at the corner, then a little light over it, and now a 24/7 dome camera); the pics in the nature reserve may not show all that many traffic signs, but do contain images (for example of the circular wall around the pond, and of the ‘shelter’ which can help OSM, while the other pics do at the least show ‘nothing to see here’?

2020-06-12 Zoersel Halle : both the Hero 2018 and TG5 batches look complete

2020-06-14 : so far only uploaded four batches, each just under 1000 pics : 200614 5p1, 5p2, 5p3 and h01 with 995, 990, 986 and 988 pics, of which 957, 934, 911 and 887 were published (as stated on the ‘publish nnn images’ screen) : what happened to the rest, please? Were the last ones omitted or , ? Yes, they were, missing from 12:17 till 12:22 , 14:22 till 14:36 and 17:32 - 18:04 in the TG5 series, and from 12:10 for the first Hero 2018 series - thus missing a new Fietsstraat (cycle street) behind the schools.

Could’ve thought of nicer ways to spend the Saturday evening, but these missing uploads are irksome, and forum posts by e.g. micmin and filip/philippe show that I’m not the only one experiencing this problem, thus hope that by specifying the series of missing pics, and the approximate times (mapillary only shows the minutes, not the seconds for each pic) ‘the support team’ can put a finger on what causes the missing pics in some of the batches.

With as ever ‘Vriendelijke groet’, (Dutch for '; with friendly greeting),

First, how do I know the version of Desktop Uploader?

–

Around June 21, I intended to use the Desktop Uploader to upload 44 photos. There are no duplicates.
All photos are visible from the links below.

However, even a few days later, when I look at Map on Mapillary (Mapillary), even if I look at Feed and Uploads, the corresponding data does not exist.

I always delete the “.mapillary” folder, which is a group of log files, immediately because the disk system load is high.
So even after checking the photo folder after the fact, I wasn’t sure if it was uploaded.

Therefore, I uploaded again those on 6/25.
I uploaded and “Complete” was indicated by Desktop Uploader, but again, the corresponding photo does not appear in the map, Feed, or Uploads.
After that, I received an email titled “Mapillary images or videos you tried to import failed”.
There was a description such as “duplicate image file(s) detected,” in the contents.

After that, I discovered the existence of this discussion on the bulletin board.

–

If I didn’t upload it on 6/21, I don’t know the reason for the failure indicated by “Mapillary images or videos you tried to import failed”.
Or if I was uploading on 6/21, I don’t know why there isn’t a corresponding photo on the map, Feed or Uploads.

In any case, I can be confident that this new feature is implemented incorrectly.

I’m not going to follow the detailed steps of deleting each one and doing the upload operation again. The “.mapillary” folder has been deleted, so I think it is impossible to execute it.
I give up these data.
The same photo uploaded to Google Street View is fine, so it’s ok. (Google マップ)
I want to go to the next fun place early and shoot.

–

The attached photo is a screenshot of the “mapillary_import_summary.json” file that I forgot to delete after the upload operation on June 25.
As you have attached, 44 photos have been uploaded.

If you look at “mapillary_import_summary.json”, you will see 44 successes except for the step “import_meta_process”.
The “.mapillary” folder has been deleted, so the contents cannot be examined.

Screen Shot

Using the desktop uploader files have failed to upload for two different reasons for me:

  1. images are in the same position as previous ones (happens at a stop or rarely it appears my camera app doesn’t acquire a new position and just replicates the previous one)
  2. there is a problem with the exif thumbnail. This error has been mentioned long time ago already on github. The uploader does not give any helpful hint on this. But if you use exiftool (exiftool -thumbnailimage= IMG_nnn.jpg) to remove the offending thumbs the pictures get uploaded without problem.

The latter happens to ~1-5% of my pictures varying by batch. Maybe this is also the cause of your lost pictures.

Mapillaryに新たな重耇チェックが導入されたした。
アップロヌド埌、Web公開されるたでのステップに導入された機胜であるようです。
新芏にアップロヌドする画像にのみ適甚されおいたす。
既にWeb公開されおいるものには圱響しないようです。

その、サヌバヌ偎での「厳しい重耇チェック」ずは、セッション内に画像の重耇があるず刀断した堎合には、そのセッション党おの写真をWeb非公開のたたで保留する、ずいう措眮です。

セッションずは、今たで私がシヌケンスず呌んでいたものずほが同矩のようです。
ただし、シヌケンスが1000枚を超える堎合には、1000枚皋床ごずに切断したものを1぀のセッションずしお扱っおいるようです。

この刀定は、アップロヌドの操䜜がすべお完了した埌に行われるものです。

最新バヌゞョンの Desktop Uploader (1.2.7) では、サヌバヌ偎での「厳しい重耇チェック」ず同等の重耇チェックが行われお、
重耇ず刀断された画像を陀いたセッションずしおアップロヌドされるようです。
ですから、最新バヌゞョンの Desktop Uploader を䜿甚すれば、セッションが䞞ごずWeb非衚瀺になっおしたう被害には遭わないのだず思いたす。

しかし、Desktop Uploader のバヌゞョンを知る方法が私にはありたせん。珟圚、質問䞭です。
6/24頃にPCで「Desktop Uploaderの最新バヌゞョンが䜿甚可胜です」ずいうポップアップを芋たこずがあるなぁ、ずいう蚘憶しかありたせん。

たた、Web Uploader には重耇排陀の機胜はただ備わっおいないようです。
私はそれは䜿甚しおいないため、asturkseverが投皿した以䞋のコメントを読んだずころによるず、
重耇した画像を排陀した䞊で再床セッションの党画像のアップロヌド凊理を行うこずで、アップロヌド出来るようです。

以䞋にasturkseverが曞いおいるこずが党く正しいのだず思いたす。
http://mapillary.trydiscourse.com/t/web-uploads-not-arrived-cut-at-999-images/4165/4?u=potaro67v
私は、asturkseverさんはMapillaryの䞭の人だずばかり思っおいたのですが、もしかしたら、トップレベルのナヌザヌなのでしょうか

しかし、Desktop Uploader のバヌゞョンが(1.2.7)であるこずの確認方法が私には芋぀からないので、その確認方法を質問したした。
http://mapillary.trydiscourse.com/t/web-uploads-not-arrived-cut-at-999-images/4165/17?u=potaro67v
しかし、18時間経った珟圚、ただ回答はありたせん。

このコメントで私が觊れおいるトラブル䟋は、2m間隔で撮圱した静止画44枚で構成されおいるセッションです。
Google Street Viewぞのアップロヌドが正垞にできおいるこずは、コメント䞭に瀺したずおりです。
䜍眮情報も44枚ずもすべお異なっおいたす。( = Exif情報もすべおの画像で異なっおいる、ずいうこずです)

そのようなセッション党䜓を゚ラヌずしおWeb非衚瀺にしおしたうずいうこずは、珟圚の
サヌバヌ偎での「厳しい重耇チェック」
が、明らかに間違った刀断をしおいるこずを瀺しおいたす。

このような「重耇が存圚しおいない」セッション(シヌケンス)が゚ラヌになる件は、asturkseverのコメント䞭では想定しおいたせん。
たた、䞊蚘の掲瀺板での他のナヌザヌの嘆きにも、重耇が存圚しないのにセッション䞞ごずWeb非衚瀺ずいう被害に遭っおいる方が倚いのだず思いたす。
残念なこずです。

◆◆

さお、Desktop Uploader (1.2.7)(たぶん)での重耇チェックの機胜をチェックしたす。
テストを行った結果から、私は珟圚の凊理内容に぀いお以䞋のような掚枬をしたした。

■重耇の刀定は、EXIF情報のみで行っおいる(画像の内容は芋おいない)

この掚枬結果は、私が珟圚独自に行っおいる動画凊理アップロヌドの䜜業には圱響がないずいうこずを瀺しおいたす。
同じく、倚くのDesktop Uploader 利甚者も安泰です。
私はこの結果に満足です。たた仕様倉曎されるこずがありたせんように。

その前提は、Desktop Uploader を通垞の方法で䜿甚した堎合には、そこを通過したセッションはサヌバヌでWeb非衚瀺の措眮がなされるこずはない、ずいうものです。
぀たり、サヌバヌよりもDesktop Uploaderのほうが刀定基準が厳しいか、たたは同䞀であるこずが前提です。

そうではない堎合には、䞍本意にセッション䞞ごずWeb非衚瀺になっおしたう被害者が頻発するこずになりたす。
私は察凊できるけど、䞀般の方々が被害を受けるず可哀想です。



テスト甚の画像はこのようにしお準備したした。

さお、テストを始めたす。

◆01.(5枚) ノヌマル
1f-5f
Publish Completed

時刻衚蚘は耇数枚で同䞀です(4.91fpsのため)。しかしすべおの写真がアップロヌド出来たした。
䜍眮情報は、隣接した写真ではほんの10cm皋床の差で䞊んでいたす。

◆02.(6枚) 違う画像でEXIF情報が同じもの
6f-11f
1枚しかアップロヌドされたせんでした。(6fのみ)

この結果より、重耇はEXIFで刀断しおいるものず掚枬できる。
Desktop Uploader操䜜のスクリヌンキャプチャ : bandicam 2020-06-27 01-16-59-885 - YouTube

◆03.(7枚) 既にアップロヌド枈みのものず䞀郚重耇するもの
1f-5f,12f,13f
Desktop Uploaderは、画像ファむルのフォルダ名を、そのアプリ内で蚘録しおいるず考えられたす。
そのため、同䞀フォルダ(パス)からの耇数回のアップロヌドは、凊理せずに゚ラヌを返したす。
しかし、フォルダ名(パス名)が異なれば別のものずしお扱っおくれたす。
たた、Desktop Uploader内での重耇チェックはPCのみでの刀断であり、Mapillaryサヌバヌにアップロヌド枈の画像ずのチェックは行われおいないようです。
そのため、アップロヌド前に重耇ファむルがカットされるこずはなく、7画像すべおがアップロヌドされたした。
Desktop Uploader操䜜のスクリヌンキャプチャ : bandicam 2020-06-27 05-54-57-468 - YouTube
そしお、サヌバヌでのチェックにより重耇が発芋されたずの゚ラヌメヌルが届きたした。
゚ラヌメヌル : Dropbox - File Deleted
このセッションは、7枚党おがWeb非衚瀺です。
Dropbox - File Deleted

◆04.(2枚) 重耇を陀いおから同䞀フォルダを䜿甚しお再床アップロヌド
12f,13f
(1f-5fを䞀旊退避しお、03のテストず同䞀フォルダからアップロヌドしたした。ログはそのたたにしおありたす)
Desktop Uploader操䜜のスクリヌンキャプチャ : bandicam 2020-06-27 06-15-59-832 - YouTube
「Folder already uploaded」の゚ラヌになりたす。
(その埌、䞀旊退避した 1f-5fを元に戻したした)
このような゚ラヌになった堎合には、「重耇したファむルを削陀しおから再床アップロヌドする」ずいう、Web Uploader のような再凊理方法は出来ないずいうこずです。

◆05.(2枚) Web非衚瀺の既にアップロヌド枈みのものず䞀郚重耇するもの
12f,13f
テスト03でアップロヌド枈ですが、Web非衚瀺になっおいる2画像を䜿甚したす。
Desktop Uploaderの操䜜は完了したした。
Desktop Uploader操䜜のスクリヌンキャプチャ : bandicam 2020-06-27 06-25-09-304 - YouTube
Dropbox - File Deleted
Mapillary

◆06.(8枚) 同䞀の画像で、EXIFのみ異なるもの(同䞀時刻の画像はある)
14f-21f
ただし、画像は14fのものを党おにコピヌし、14f-21fのEXIFはそのたた䜿甚する。
Desktop Uploaderの操䜜は完了したした。
Desktop Uploader操䜜のスクリヌンキャプチャ : bandicam 2020-06-27 06-52-24-077 - YouTube
Dropbox - File Deleted
Mapillary
(同じ画像で、䜍眮情報違いの8枚です)

今日のテストはここたで。
この仕様が倉化ないならば、私の珟圚の䜜業にはなんの障害にもなりたせん。
ここ䞀週間のトラブルが移行期の䞀時的なものであるこずを期埅したす。

この芳点ずテスト内容ずをここに公開しおしたったわけですから、特にテスト06に぀いおはMapillaryさんはいい顔をしないかもしれたせんね。
しかしこれが私にずっおは倧事なこずなのです。
動画を3分毎に分割しながら長時間出来る仕組みを、私は持っおいたす。
その分割時に僅かな時間の隙間が出来たす。その隙間をダミヌフレヌム(同䞀の画像を䜿甚するしか無いでしょう)で埋めるこずを認めおもらえるず、凊理効率が高くなるのです。

以䞊、レポヌト終わり。

1 Like

I would like to introduce my first mass upload work (using a desktop uploader) after the introduction of a new strict duplicate check in Mapillary by last week.

There is no change in the operation method with the desktop uploader so far.
In addition, the trouble that causes various troubles for every 500 sheets is still occurring.

先週、Mapillaryに新たな厳重な重耇チェックが導入された埌の、私の最初の倧量アップロヌド䜜業(デスクトップアップロヌダヌ䜿甚)をご玹介したす。

今たでのデスクトップアップロヌダヌず、操䜜方法は倉曎ありたせん。
たた、500枚刻みで様々なトラブルが発生するずいう障害の存圚にも倉化はありたせん。
そしお、同じEXIF情報を持぀画像がカットされおしたう、ずいう動䜜も、以前ずの違いはありたせん。この機胜は以前からむンプリメントされおいたした。

デスクトップアップロヌダヌを䜿甚しお、䜍眮情報を付加した2983枚の静止画をアップロヌドした䟋を瀺したす。
デスクトップアップロヌダヌのバヌゞョンを知りたいのですが、私にはそれを調べる方法がありたせん。
デスクトップアップロヌダヌを䜿甚しようずしたずきに「最新版が存圚したす」ずいうメッセヌゞを芋た蚘憶がありたすが、具䜓的にはどのようにアップデヌトされたのかが私にはわかりたせん。

では、アップロヌド時のスクリヌンキャプチャを瀺したす。退屈な動画ですので、時刻ごずの芁点を先に曞いおおきたす。

00’04" アップロヌド凊理開始。たずは事前の準備にかなり時間が掛かりたす。
05’08" サヌバヌぞの転送開始。アップロヌド速床が遅いです。
31’31" 500枚を目前にしお、アップロヌドが止たりたす。
50’20" 䜕回かアップロヌドキャンセルをしおから再床アップロヌドを開始するこずによっお、先に進むこずが出来たす。
1:05’30" やっず、本来のアップロヌド速床に達したした。しかし、私のネットワヌクの胜力いっぱいではありたせん。Mapillary偎の受け入れが遅いだけです。
1:06’10" 1000枚でも
1:12’20" 1500枚でも、なにかサヌバヌ偎でデヌタ受け入れを躊躇しおいるこずが想像できたす。
1:18’43" 2000枚でも。
1:25’00" しかし2500枚は䜕事もなく通過したした。
1:29’05" アップロヌド完了したした。

アップロヌドした結果の Street-level Imagery は以䞋のようになりたした。

以前のコメント( http://mapillary.trydiscourse.com/t/web-uploads-not-arrived-cut-at-999-images/4165/19?u=potaro67v ) の「テスト◆06」で確認したかったのは、以䞋のURLから始たる30フレヌムのような、動画の自動分割によるギャップを埋める私独自の凊理ぞの察凊でした。
https://www.mapillary.com/map/im/fnyhu94bjnE87ErqqOna6U
珟圚の「厳重な重耇チェック」でも、無事に、私が思った通りの Street-level Imagery が䜜成されおいたす。安心したした。

このように、Mapillaryシステムは、既存の異垞の存圚も含めお今たでず同じ動䜜をしおいるず私は考えおいたす。

しかし、新たな厳重な重耇チェックの導入䜜業が行われおいた期間である数日間には、Mapillaryのアップロヌドシステム党䜓(特にサヌバヌ偎)の動䜜には異垞が存圚しおいたず思いたす。
同じ方法でアップロヌドしたシヌケンスが重耇゚ラヌになりたした。
以前のコメント ( http://mapillary.trydiscourse.com/t/web-uploads-not-arrived-cut-at-999-images/4165/17?u=potaro67v )でも觊れおいる通り、以䞋のようなメヌルが届き、Webには未だに公開されおいたせん。
I received an email titled “Mapillary images or videos you tried to import failed”.
There was a description such as “duplicate image file(s) detected,” in the contents.

デスクトップアップロヌダヌでアップロヌドされたデヌタは、今回むンプリメントされた「厳重な重耇チェック」で゚ラヌになるこずはないはずであるず私は考えおいたす。
しかし、私は、重耇゚ラヌず刀断されおWeb公開を保留されおいるデヌタを持っおいたす。
他のナヌザヌにも圓然、同様の障害が発生しおいるず思われたす。

ログファむルは、私のポリシヌにより、「アップロヌドが完了したら削陀する」こずにしおいたすので、ログの確認は私の手元では䞍可胜です。
もし、Mapillary瀟員の方がこの異垞に興味を持っおくれるならば、サヌバヌ内での私のデヌタを盎接調べおください。
私自身がなにか䜜業を行っお、この画像の再アップロヌドなどの察凊をする予定はありたせん。もずもずこれらのデヌタには重耇ず刀断される芁玠は存圚しおいたせん。
私の以前のコメントに dropboxのリンクを茉せおあり、私がアップロヌドした写真(そしお、Web公開が保留されおいる写真)はそこから確認できたす。

今回の新機胜むンプリメントのための過枡期の期間はそんなに長くないはずです。
その期間に発生した党゚ラヌ(そしお、Webに公開しないずいう措眮がされおいるデヌタ)に぀いお、サヌバヌ偎で芋盎しを行うこずを、私は提案したす。

最埌の郚分のみ、英蚳しお付加したす。

In this way, I believe that the Mapillary system behaves the same as before, including the presence of existing anomalies.

However, I think that the operation of the entire Mapillary upload system (especially the server-side) was abnormal during the several days during which the new strict duplicate check was being introduced.
A sequence uploaded in the same way resulted in a duplicate error.
As mentioned in the previous comment (http://mapillary.trydiscourse.com/t/web-uploads-not-arrived-cut-at-999-images/4165/17?u=potaro67v ), as below, I received an email and it is not published on the Web yet.
I received an email titled “Mapillary images or videos you tried to import failed”.
There was a description such as “duplicate image file(s) detected,” in the contents.

I believe that the data uploaded by the desktop uploader should not fail with the “strict duplicate check” that implemented this time.
However, I have data that has been placed on NO web publishing due to a duplicate error.
Of course, other users were likely to have similar failures.

By my policy, the log files are “deleted when the upload is completed because the amount of log files is huge”, so it is impossible for me to check the log file.
If a Mapillary engineer would be interested in this anomaly, please look directly at my data on the server.
I have no plans to do anything to re-upload these images. Originally, there was no element judged to be duplicate in these data.
I’ve included a dropbox link in my previous comment, and you can see the photos I uploaded (and photos pending web publishing).

The transition period for implementing this new feature should not be that long.
I suggest doing a server-side review of all errors (and any data that you’re prepared not to publish on the web) that occurred during that period.