Shooting 360°x180° Images – from a Helicopter

This is a small demo.  Please note: this works (properly) only with HTML5 compliant browser – such as Safari on a Mac. It works on iPad2 (i.e. Mobile Safari) but it does NOT work with iPad 1st gen. nor iPhone (except – I was told – it does work with iPhone 4S) – due to non-optimized media and subsequently high processing load. Thats’ why it is called a demo i.e. it is not a finished product – it doesn’t have to be perfect… :-)

I have been presenting it couple of times now (in different formats) when I have been talking about the possibilities of visual storytelling and multimedia  – and the question always comes: “How the hell did you do that…?”

I also want to dedicate this post  to my good friend  Peter “Hopper” Stone, who is working in Hollywood / LA and who has been getting into this. I showed him this when I was working for it couple of weeks ago and I promised  to write a blog post “one of these days….”  Sort of quick tips / pointers as what to take into account when doing something like this.  Well, it’s been couple of weeks instead of days – sorry about it, dude –  but here we go.

Panorama Basics

Basically, when you shoot panoramic images you shoot several images, typically with an extreme wide-angle such as 8mm/15mm and stitch them together  with a specific stitcher program. I use PTGui – professional and yields excellent results but I have to warn you: it does have a learning curve.

360° degrees is very easy – but when you add the other dimension i.e. 180° vertical, it gets a bit more complicated. You typically also shoot so-called Zenith (totally up) and Nadir (totally down) images which you include into your project. The most important thing is that the camera rotates around so-called  Nodal point – which is the one point within the lens where all the beams of light cross. You could call it “an absolute zero”  inside the lens if you want.  You either have to know this point or you have to know how to measure it (it’s not that hard, I might write a post describing it one of these days), typically it is done with either a laser or e.g. aligning several spotlights on an imaginary line and calibrating your camera with them.

When working in tight surroundings – such as inside a car – knowing the nodal within +/- 1mm is crucial. Outdoors it is much more forgiving. It all comes down to what is called the “parallax-error” ie. in neighboring images objects which are near align differently to objects which are far away, if the camera is not rotated around the nodal.

To present a panorama, flash (as in “programming language”, not as in “strobe”… ;-) ) is typically used.  QTVR used to be common, but it is dying. The most modern way of doing them is HTML5 3D transformations (which is used here). It has it’s problems but also advantages – the most important of which is that i plays in iDevices. But problems include that the majority of browsers are not HTML5 compliant – yet.

Well, that’s the basics as briefly as I can present them. Add a helicopter and it becomes a bit more complicated… :-)

The Assignment

The client asked me if this could be done. I said “sure, no problem”.

Yeah, right…

What they did NOT ask me was “Do you know how to do this?”  Thank God, because I had no idea… But I think I figured it out.

Obviously, the problem is “how to rotate the camera around the nodal” when flying c. 400ft above the city of Helsinki. I thought about it for a long time and came up with two solutions: you either get the camera under the chopper in some sort of rigging and rotate it there (or you go yourself under the chopper witht the camera (hmnnn… no, I don’t think so…) ) – or then you rotate the chopper.

I did both – and both worked – but the demo presented here is done with rotating the camera. The Zenith is shot on the ground and the Nadir (which was not actually used on this example) by asking the pilot (Rene Koivisto – what a great pilot to work with!) to slide the chopper sideways so I could get the ground below – and then I corrected the image for perspective in post. If you are shooting straight down from c. 150m you can pretty much forget the parallax (unless there are tall buildings right below you) because the scenery under you is basically 2D.

For another image (not presented here) I did ask the pilot to turn the chopper around while I was shooting (we had removed the door) and I asked him to try to rotate around the spot I was shooting from. It worked…but it was pretty nerve wrecking as there was 12-15m/s gusts of wind in that height and being sideways to the wind was not easy.

Challenges

I do NOT like heights...

Goes without saying that there are lots of factors which have to be taken into account and which can go wrong.

Start with the fact that it’s cold and windy up there with no door. You cannot attach the camera under chopper beforehand (as the chopper is standing on ground). You cannot use radio control on the remotes. Each piece of equipment has to be secured – yourself included. I wore a climbing harness and all the equipment was secured with climbing runners. You don’t want to drop any gear on somebody’s head 200m below you… might ruin his/her day, not to mention your equipment…

I ended up using a carbon fibre sound pole (Gitzo) onto which I attached the camera with cable remote. It was lightweight to handle and the locks are solid and reliable – plus you could set a security leash inside of it.  Trouble is: extended it is c. 5 meters. And if you have ever been in a four seater chopper…. it’s kind of tight quarters, you know ;-) .

So you have to extend it when airborne – and make sure you have properly measured the security leashes, the remote cable  etc.  Even more challenging is collapsing your rig – as c. 1m above you are the rotor blades turning and keeping you in the air – and you do not want to touch them with your rig.

To make things more challenging still: you work with gloves on (it’s cold) and you have to secure the runners, the cable, the focus ring, even your safety belt with tape. You manipulate your rig with two hands. Thus, you need a third hand to trigger the camera. I used a sort or makeshift thing between my knees – but a release switch  operated with your teeth would have been great (same kind of things as parachute shooters use).

You worry about shake, vibration from the chassis, exposure, adequate shutter speeds, leveling with the horizon, focus, not breaking your equipment when you are setting it under the chopper, not falling out yourself, not dropping anything…

Basically, you are worried and scared beyond belief… at least I was. It’s not that my hands were shaking, it was my legs doing it. Seriously.

Sure, in an optimal case – theoretically -I would have had an assistant next to me checking the focus, exposure, horizon level etc. with an iPad… Well, this time was not optimal… ;-)

Presentation

Click Image to See

(For those of you who have manage to read this far….)

The most interesting part for me of this demo is the presentation: it’s pure HTML5,  which means no flash. Not in the panoramas, not in the presentation with animations/transitions as a whole.

But this comes with a downside: you have to view it with a compliant browser as I mentioned in the beginning. If you are on a mac, just use safari and you’re okay. Windows: Google Chrome at least, don’t know about the IE9 etc.

It is also iPad compatible. But: as these panoramas are not optimized for small processors, I realized that e.g. my iPad 1st gen. did crash constantly.

BUT: IF you own a iPad2 I (or iPhone 4s) urge you to try it out.

It’s turned into a web app, so if you save it to your home screen, you notice the button and the fact that there is no safari navigation bar.

The panoramas also have a gyroscope function built into them. So:  you can manipulate them with controls, with your finger… or by just tilting and turning your iPad. I was personally blown away when I saw this….

But then again, I really do not know anything about programming. No, this is not brilliant, it’s not multimedia – as I said, it’s a demo.

But I just think it looks kinda cool…. ;-) ?

PS (29.11.2011)

I was happy (and proud) to notice I made it to Rob Galbraith’s  great website. He said I did not mention how I erased the helicopter. I did but maybe not clearly enough:  the image straight up is called Zenith and it is shot after landing the chopper. Blending  it together with the rest of the images was not hard at all as the sky was uniform blue.

Also – as I started to make explanations ;-) – the image with the chopper was not intended for the client in the first place. Thus the stitching is not of the precision it should be in this image.

19 Comments

  1. Rami Saarikorpi
    November 27, 2011

    Aika hyvin onnistuit :)
    Ainut huono homma on tossa esityksessä se, että siinä on joku (ilmeisesti iPad-katseluun tarkoitettu kiihtyvyysanturin tunnistus päällä) ja koneella katsellessa kuva pyrkii kokoajan kääntymään alaspäin…

    Reply
    • kkuukka
      November 27, 2011

      Rami – millä kokoonpanolla katsot “koneella”? Mulla pyörii hienosti kolmessa eri mäkissä ja iPad2:ssa (OS Lion ja Snowleopard). Yes, siinä on kiihtyvyys gyroskooppi aktivoituna, mutta en tajua miten se vois ns. kick in tavallisella koneella katsellessa, ellei sun kone ole viritetty jotenkin simuloimaan iDevices tms….? Pädissä joo, se on tarkoituskin. iPad 1st gen sais mukaan sillä, että vain pienentäisi cube faces 1024px, mutta en viitsinyt ruveta veivaamaan.
      Kuten sanottu, tää on demo jota käytän opetuksessa ja vastaavissa.

      Reply
      • Rami Saarikorpi
        November 27, 2011

        Mac Book Pro ja Leo sydeemillä pyörittelin ja siinä se vähän tökki, mutta tällä iPadilla se on hyvä! :)

        Reply
        • kkuukka
          November 27, 2011

          Joo, se on totta. Se vähän nykii, sitä ei oikein voi hiirellä sujuvasti siirrellä, vaan on käytettävä dashboardia. Luulen – kuten sanoitkin – että se voi olla se kiihdytys tunnistin aktivaatio joka siinä tökkii.
          Jes, tämä ei ole mainstream vielä – mutta jos tekee vaikka flash version koneille ja html5 vers. pädille, niin johan pelittää sujuvasti. Onko toi teidän tuleva kirja 360 kuvausksesta- oletan että on?

          Reply
  2. Rami Saarikorpi
    November 27, 2011

    Ai nii ja ton “nodalpointin” löytymiselle on tammikuun viimeinen päivä ilmestymässä Kennethiltä ja multa kirja. Toki siinä kaikkea muutakin on, mutta mm. ohjeet ko pisteen löytämiselle myös…

    Reply
  3. ramin
    November 28, 2011

    Ainakin Lion+Chrome yhdistelmällä tuo toimi ihan ongelmitta, vaikkakin 2009 alkuvuoden MacBookin prossu alkoikin huutaa. Mutta erittäin mukava nähdä, että nämäkin tekniikan alkavat toimia ilman flashiä.

    Reply
    • Juha
      November 28, 2011

      4 v vanha perus-pc vakionäytönohjaimella, Vistalla ja Chromella: reagoi miellyttävästi hiireen, rullazoomaus myöskin juohevaa. En tiedä johtuuko osin kohteiden etäisyydestä, mutta minusta kaiken kaikkiaan miellyttävät intuitiiviset kontrollit hiirellä verrattuna esimerkiksi tähän (http://kkuukka.wordpress.com/2011/11/27/shooting-360×180-images-from-a-helicopter/#more-2979) flashiin, jossa kaikki liikkeet mun hiiren perussäädöillä on niin kuin kumilangalla vetelisi. Chromella tässä helikopteridemossa tartuntapiste pitää ja seuraa hiiren liikettä melkein viiveettä.

      Firefox toimii myös hyvin, paitsi mulla rullazoom on raivostuttavan hidas verrattuna Chromen “välityksiin”.

      Ie9 vinkuu ensin muutaman varoituksen ja jumittuu sen jälkeen latausnäyttöön “Loading 0%”

      Niin kuin aina, eri yhdistelmät (koneet+selaimet+säädöt) tuottaa ennalta arvaamattomia tuloksia, mutta hyvään suuntaan ollaan menossa. Hyvältä näyttää html5 tulevaisuus tämän esimerkin valossa flashiin verrattuna. Hieno demo – uutta materiaalia odottaessa!

      Reply
      • Juha
        November 28, 2011

        äh kopioin väärän linkin, oli tarkoitus verrata tähän flashiin: http://docimages.fi/vancouver/panoramas/

        Reply
      • kkuukka
        November 28, 2011

        Hetken ihmettelin… joo, tämä koneiden, ohjelmien ja niiden käyttäjien sekasotku aiheuttaa aina pientä päänvaivaa. Toi Vancouver juttu oli tosiaan FPP/FFC flash ja toimii mun maceissa ihan siedettävästi, tosin inertiaa voisi olla vähän enemmän, eli että kuva pysähtyisi helpommin.

        Helikopteri on täysin kokeellisesti tehty HTML5:lla – tietyllä tavalla juuri että saisi tällaista (kiitos…) arvokasta palautetta. Jos olisin tehnyt tuon asiakkaalle, niin olisin pudottanut kaikki JS animaatiot pois, tehnyt perusflashin ja iPad version html5 1024px osakuvilla.

        Mutta; tämä on tällaista hakemista ja kokeilua. Ja kokemuksen (syvällä) rintaäänellä voin sanoa, että ne kokeilut pitää tehdä oikeassa ympäristössä (ie. tässä tilanteessa helikopteri ja näyttää julkisesti vs. kuvata kotona olohuoneessa ja näyttää vaan omalta koneelta rouvalle…) että niillä olisi mitään arvoa.

        Koska tulevaisuuden suunnasta ei liene mitään epäselvyyttä (i.e. pädi, jossain muodossa, luultavasti kuitenkin iPad 70-80% globaalilla markkinaosuudella), niin on tärkeää katsella mitä tuolla hötömölö vitosella saa aikaiseksi.

        Kiitos arvokkaasta palautteesta.

        K

        Reply
  4. James
    November 28, 2011

    Just wanted to let you know that this worked perfectly in safari on the iPhone 4S!

    Reply
    • kkuukka
      November 28, 2011

      Sweet – cheers. I tried with 3s and 4 but obviously there was not enough processing power. Nice to know that it works – and that the used technology basically is ok, just have to downsize the physical size of the panoramic media.

      K

      Reply
  5. JP
    November 29, 2011

    Tämä juttu oli jo päätynyt Rob Galbraithin uutisiin saakka

    http://www.robgalbraith.com/bins/content_page.asp?cid=7-11667-12139

    Reply
  6. Brian
    November 30, 2011

    alternatively one might also use kite photography to capture such panos: http://scotthaefner.com/360panos/kap/

    Reply
    • kkuukka
      November 30, 2011

      Brian hi –

      yes, I did consider using a balloon or a set of balloons (winds are too unstable this time of year and this country -and considering it’s a tight urban setting with railroad tracks right next to it – I decided against it.

      Also: it was a paid gig for a client, who needed exactly these coordinates (X-Y-Z +/- c. 10 meters) – so a chopper (or a remote chopper) was the only option. And the client needed couple of other shots as well around the city (cannot show them, sorry) – and they needed them like… now. (If you work professionally, these days you might have noticed that clients usually want their stuff … like yesterday ;-)

      Something one also does not automatically think: this is Finland, not sunny California: daylight like couple of hours only and this time of year possibility of rain/extreme strong winds/ low visibility is really high. So lots of things to factor in.

      I was sort of discouraged against doing anything too experimental by a FB posting a friend of mine – a professional film maker – made couple of days earlier: “I consider it a good day, when you don’t have to call the Helsinki-Vantaa airport traffic control and warn them about some balloons with camera equipment set loose in their air-space…”

      He offered to lend me some balloons, but I politely declined… ;-)

      I checked Scotts Haefner’s site quickly, there is some great info over there, thank you for sharing, I’ll be going back there one of these days.

      K

      Reply
  7. Sami M.
    December 12, 2011

    Works fine with Windows XP and Firefox 8.0, doesn’t work with Internet Explorer 8.0.

    Reply
  8. Tim P
    March 17, 2012

    Hey guys, I just found this site full of similar 360 degree images taken from all over the city of Tampere from Finland: http://www.virtualtampere.com
    Image quality seems great and barely no stitching “errors” visible. Works wit iPad too. Do you have something to do with this and isn’t it pretty expensive to fly around with helicopter taking these?

    Reply
  9. Rami Saarikorpi
    April 12, 2012

    Tim P. That Tampere 360 system is made with UVA not with real helicopter. Something like this:
    http://www.aerialimaging.co.nz/assets/Photos/X6-in-flightL.jpg

    Reply

Leave a Reply