In my opinion it is impossible to deduce height of a photographed object based solely on one image without further information. You would need a zoom level, lens information, focal length and such then.
There are apps out there, where you can place two points and the app tells you the distance. That is in essence what you are trying to do? But that app has the image info needed to do that (nifty!) calculation…
When there are two images, at different places with the same object(s) in the images and you know the latitude and longitude, you know the distance between the two images, then you could look for object matches/corners etc and with some extremely nifty programming you could deduce height then?
(I think I just described SfM? 
)
[edit]
Auch, I was a bit quick to respond, I visited your (test) page. One has to enter distance and angle now (I was thinking way more complex I guess).
I missed the “360” part in your post about the required image. Would one such an image have enough for you to go on? You just might!? Next step click on two points in the image? You got me curious!
You made me think of my debug/test page: http://www.geoarchief.nl/mapillary.html
When one moves around with the mouse, the Mapillary JS gives a latitude/longitude. Maybe you could reverse engineer that calculation?
Maybe you could convert your concept to an Mapillary JS extension.
Drop a line when you’re further in devolpement!