Great, this is very helpful @GITNE !
I was able to replicate the issue with the last image in a capture having issues with automatic horizon alignment and re-orientation. We’ll have a fix out in an upcoming release (so disregard my earlier message, no need to work on this @caglarpmeta)
In terms of the roll axis, that does look like a small SfM reconstruction accuracy issue, unfortunately not something we can easily address in the short term. However that looks to be a fairly minor one.
Thanks. A key reason why I upgraded from an Insta360 X3 to a newer 360-degree camera was the X3’s inability to keep the horizon level. Now, looking back at older X3 footage following the Mapillary update, it is clear that I can once again put this camera to productive use thanks to this innovation. Even though the X3’s resolution doesn’t match that of modern cameras, the footage remains impressive—especially in good lighting. The heart of Insta.
Recently I recorded a sequence with a GoPro Hero 5 Black and I noticed that the automatic horizon leveling performed automatic horizon unleveling on it. It was leveled on the car and didn’t need much corrections. The sequence is still processing though. I’m just sharing this to give you one more point of data.
Man, this is close to the ground! Did you mount the camera to the bumper? 
Generally speaking, I suppose most people do not care if the imagery is rolled, pitched, or yawed during processing (after all, this is SfM’s job) with intermediate results being visible in the viewer. But, they do care about the quality of the end result. Most people tend to notice when the view is even slightly slanted or rolled in the end result, especially since we are aiming for this perfect level alignment with the AHL feature. The margin of error, especially for roll, virtually does not exist here. I would argue that this roll issue is not negligible at all! Why? Because due to how the human sense of balance works and the fact that humans naturally basically always keep their head (or eye‑line) level with the horizon, unless they actually deliberately force themselves to do otherwise, humans are extremely susceptible to even the slightest deviation. It goes so far that some people get seasick just by looking at slanted pictures, while others just feel discomfort. Hence, this issue’s importance is not something that can be measured by SfM’s (even though relatively tiny) alignment threshold but should be measured by the level of discomfort of or impact on viewers. 
Now, multiply this by the number of views per day on the Mapillary platform!
The camera is mounted on the hood of my car. It’s the highest place where no part of the car is in the frame 
I guess I could have used an extension to make it higher, but for various reasons (like pedestrian safety during an accident) I choose not to. Normally I use a 360 camera on the roof anyways.
This is a good example, thank you @Fixtor - I was able to isolate the issue and we’ll have a fix out in an upcoming release.
The fix is out now, your capture should now be correctly leveled @Fixtor .
Thanks again!
Roll looks to be fixed for all the mentioned (planar and fish‑eye) examples. 
However, your initial 360° example continues to suffer from roll.
Yet, the last image in that sequence has now received correct AHL.
Does the fix require full SfM recompute or can you limit recompute to images with roll?
One more thing; when comparing images in time‑travel mode AHL is lost or disabled. Is this intended?