I’ve made the PS suitable for everybody. It can process 360 videos (GoPro Max 1 and 2) for Mapillary (automated processing of the video to pictures) and for other programs (mp4 with GPS data and a GPX file).
3 Likes
Hey! Where can I find the gopro labs firmware for the Max 2?
2 Likes
Doesn’t the timewarp at 5x get you around 6FPS at 8K? I just ordered a Max2 today and this was after I confirmed I could get the sub second frame rate with GoPro support.
I finally made a program (windows only) that extracts and corrects the mp4 and gpx files of the GoPro Max 2 which can be used with Street View Studio without any errors. Also there is a Mapillary function built with adjustable sample distance.
I made it into a single .exe file which has all the tools (Exiftool, ffmpeg, ffprobe and mapillary_tools) built in.
This tool is only compatible with the GoPro Max 2, not with the older versions. I will adapt the current program to use it also with the Max 1 but there are still some issues I have to overcome.
The issue was that the GPMF information of the Max 2 is totally different than that of the previous version (Way more advanced, face detection for example). StreetView Studio is much more a pain in the *ss than Mapillary and is rejecting everything what is not compliant with there (own) standards.
The fix will rewrite the GPX and video file headers to be in line with each other after all the extraction of these data.
https://www.dropbox.com/scl/fi/bvm387ruwbkgh8queajjm/GoProMax_Workflow.zip?rlkey=rurrv7hn1xvkpzcwnk6mak9aa&dl=0
6 Likes
You are a very good person TheWizard. I look forward to testing this out shortly but do I open the camera now or wait for Christmas?
Never wait, you can always put the empty box under the Christmas tree 
1 Like
We have made some adjustments, and GoPro Max 2 stitching works reasonably well now. Here are examples of sequences from Max 2 (.360 files) uploaded through Mapillary tools and stitched by our backend:
The mentioned 5952 x 1920 (which will be stitched into 7680 x 3840) will also work.
At the same time, we haven’t yet tested it thoroughly with longer captures. So, if you have something to upload and can verify that it works for you, that would be awesome.
cc @boris
–
BR, Yaro
3 Likes
Now 419 euro in Belgium.
1 Like
Thank you for making the platform accept and properly handle GoPro MAX2 .360 files directly. 
What tool do you use on the backend to re-project the video into equirectangular projection? Can you recommend a Linux tool?
Btw, I would appreciate it if you would avoid wrapping example links in facebook.com redirects. It is no good netiquette. 
Thanks!
1 Like
I have noticed some really strange stuff going on with image quality on some cube sides. 
I am not sure where this comes from. Is it in the original video stream? Is it a camera 
firmware bug? Or, is it due to the conversion or reprojection? This effect is not visible at the start of your example video. But, it becomes very apparent at the end. 
One can clearly see chunky artifacts (lower bit rates) on some sides of the cube video stream, especially on the cyclist. In fact, one can easily make out each cube side due to different levels of artifacting per side. I have highlighted the cube sides for your convenience.
1 Like
I will probably split my answer into multiple messages, as it is simpler for me to focus on one thing at a time. 
So, first things first: I removed the FB redirects—I’m not even sure where I got them, to be honest.
1 Like
.360 handling. The implementation is custom and home-made (implemented long time ago and not by me), and it involves multiple steps (I will describe them below).
From end-to-end (or near that) and open-source, I know about max2sphere, though I’m not sure whether it’s included by default in any repository. Also, I don’t remember if it takes video files or pairs of images as input, so it might require some additional scripting. That’s the closest tool I can think of.
In our case it happens like this:
First, we slice the video into images - always two matching images for GoPro, one from each camera. FFmpeg is a great open-source tool for that. Then, we read each rectangle (not a cube, but rectangles provided by GoPro .360), and generally, it is projected while honoring/blending overlapping parts. Alternatively, one can overlap the images to get a real cubemap, and there are plenty of tools (e.g., py360convert) to move forward from there.
2 Likes
About the artifacts: I’ve noticed that they were present in the initial frames, and they were more noticeable on one camera—specifically in the area that is to be blended. I’m not sure whether this is a hardware or software issue, and our blending process doesn’t resolve it. With my bare eyes, I would simply blend the images with emphasis on the clearer part, but the algorithm doesn’t recognize or account for this, at least for now.
There’s always a chance there’s more to it, but the artifacts were definitely present in the initial video I had. Here is a good example, on the branches.
P.S. That’s an extremely sharp eye you have - I would have spent years trying to find it
2 Likes
There is more to it. This looks really 
weird! 
Once you know what to look for, you kind of cannot unsee 
it again.
Well, I guess everybody is more or less susceptible to slightly different things. 
Perhaps knowing how JPEG works may also have something to do with it because then one tends to look for specific things? 
But anyhow, the differences in quality per cube side are imho such obviously apparent in your examples that I cannot but believe that there is a bug somewhere in the chain of processing. I am just unable to tell where it might be. In fact, GoPro’s 360° cameras process six sides internally, employing Google’s Equal Area Cubemap projection. All of this makes me scratch my head because if I recall correctly, @TheWizard’s imagery does not exhibit such anomaly.
This paragraph in GoPro’s blog post makes me think:
Finally, another big bitrate advantage is that we are now doing Electronic Rolling Shutter (ERS) compensation on the camera before it hits the encoder; unlike Fusion, which did this all in a post-processing step. This ERS compensation is a key factor in stabilization. It is the anti-vibration stabilization applied in camera. This means even better compression for the encoder since like pixels move less between frames, and this has proven to significantly reduce compression artifacts alone.
Especially the last sentence “This means even better compression for the encoder since like pixels move less between frames” reads to me like if you may have captured your video with HyperSmooth off, which may have caused these different levels of compression artifacts per side. It is just a guess.
Yeah, this stitching/blending artifact can also be observed in GoPro produced JPEG images and is kind of to be expected. Thus, I do not find it as distracting or apparent as these awful compression artifacts per cube side. A blending strip area is to be expected and most people are also probably okay with it. Although, this blending halo may also be one reason for why some contributors prefer to turn the camera sideways.
Absolutely, FFmpeg is a great tool! 
Am I correctly assuming that you use the lenscorrection and/or lensfun video filters?
You can clearly observe the same strange effect here too:
This sequence has also been processed just recently.
This is so apparent. 
Looks like a bug in the camera’s firmware. @Yaro @lewin76 please check your anti-flicker (effectively rolling shutter compensation) setting first. It should be set to 60 Hz for 30 fps recording. If this does not solve the issue then please set anti-flicker to 50 Hz and recording speed to 25 fps.
Looks like Michelangelostraat to me.
1 Like
Great, now we know! Thank you for solving this mystery for us.
Thank you for pointing out max2sphere. I cannot find it in my distro nor on Flathub either. Nevertheless, I have built it on my machine and took a look at it. It works okay but it could use some efficiency improvements. I would be ready to improve the code but not for free. Open source is great and can be a lot of fun but people need to eat. My days working for free are long time over.