Creating a Multimedia Interface

Multimedia Interface

The Multimedia Interface

I have been working on this for weeks now – and finally it is operational. And that’s good news, really good news and very important to me.

But the bad news (well, not really bad, but annoying): the project I used for creating this is sent to a client for evaluation and I cannot show it at this stage and  walk you thru it. So unfortunately no  WOW-factor this time, only my words. But you get a pretty good sense of what is possible by taking an intelligent and a critical peek at the Sölden Multimedia or Helsinki International Horseshow 2009 Multimedia I have mentioned earlier in this blog. Combine what you see to what I say (both were used as pilot projects).

My goal has been very simple: for the viewer, I want to offer non-linear, visually driven multimedia, incorporating images, video, panoramic images, sound, graphics and text all into a single piece of multimediawork. Intuitively accessible for the viewer and visually interesting and challenging, something they would like to come back to. Simple and yet a professional  feel to it.

MULTIMEDIA: Linear versus Non-Linear approach

mediastorm

The MediaStorm website

The majority of the so called “multimedia” which is published is simply linear A-to-B storytelling (if you see eg. what giants like The New York Times or MediaStorm produce). I strongly feel that if you are to call it MULTImedia, you should have a non-linear approach, something along the lines the BFC guys are doing.

Besides, I am thinking about the readers of traditional newspapers (my clients), even trying to profile their needs, habits and desires: newspapers are inherently non-linear (whereas TV is linear). Thus, if you want to appeal those audiences who in the past preferred the printed page, the approach has to be more than a simple video “A-to-B” . The approach has to be non-linear – and it has to be a good quality one.

For the designer (ie. this time myself), I needed to create a designing workflow which was GUI-based, with minimal hand-coding necessary.  To make it have  flash and html-combined, with the emphasis more towards flash where possible, no AS 2/3 -skills needed. The workflow had to be so smooth that you could accomplish a (simple) piece within more or less decent timeframe  (hours, not days) in order to meet a deadline. And  you had to maintain a professional look, even if you were pressed with deadlines.

That is lot to ask – as I am a photographer, not a designer. But, I realized that I had  knowledge of and access to some interesting programs, which when used – or should I say mis-used – appropriately, would produce interesting results. So combining some 10-12 programs into a workflow,  I managed to come up with pretty fluid solution. Several of these programs are meant for panoramic work, the other programs include FCP, PS, PM, Soundtrack, Flash, etc.

2D VS. 3D

The key concept of my thinking is: normally  visual content (“newscontent” as I am a pressphotographer) is displayed on a 2D-space, ie. with  x-y – coordinates. Typically an image or collection of several images – or maybe a slideshow on a webpage, along with text. A graphic – maybe animated – maybe even a small video.

When working with panoramas, you have the z-axis as well.  And  that makes it a totally new ballgame.

But – and I can hear the counter-argument immediately – what if you don’t want to include panoramas? What if you don’t know how to shoot them? What if you really could not care less about them?

I realized that panoramas are actually not necessary, even though I personally like to work with them. What one really needs, is the 3D- projection plane panoramas are usually projected into. Understanding that and mastering the necessary geometry/software gives you a full control of x-y-z -coordinates on a cylindrical or spherical projection plane – or both – along with the standard 2d projection plane used. In laymans terms, think of it as a projection space consisting of a screen, interior of an oildrum and the interior of a ball, all blended together, forming a perfect 3D designing platform.

This gives you limitless possibilities for displaying your media. First of  all, all the visual media content (images, slideshows, graphics, video) can be either static or dynamic – ie. they can be aligned to the underlying 3D-plane/space or to the 2D-window you have open. They can be presented straight or distorted in the projection space and they can be scaleable or non-scaleable.

nauhurit

Some of my favorite tools for recording audio: Zoom H4n, M-Audio microtrack 24/96 and Marantz PMD 660

Audio can be either on autoplay or have manual controls. It can be static or panned in space. Live-mixing with several interacting sounds is totally possible (ie. eg. a situation where ambient audio changes depending on the viewing angle of the plane and there is voice-over with controls super-imposed – a VO which you can listen if you choose to).

An event (ie. loading an action, image, sound, video, graphic… or  loading another projection plane, etc.)  – can be triggered by  e.g. entering, leaving, clicking, etc. on a hotspot  – or by entering a preset area in the presentation. And as presentation can be multilayered (ie. contain several projection planes), there is practically no limit to what one can or cannot do.

One can also use a preset timer – which can be reset dynamically, multiple times, if one wants to. Events can include visual changes, audio effects, narratives, links to websites, basically anything. The media/action which is called up by an event can be aligned  either statically to your window or aligned to your  projection plane.

And as all this is accomplished within flash-framework: there is a possibility for fullScreen-viewing and scaling up and down of your presentation. Supported and preferred format for media in this workflow are jpg’s, png’s, flv’s, swf’s, mp3’s, among others.

HTML-code is used minimally, basically only to trigger outside media resources/assets into the publishing frame  and for providing that frame. Most of that – if not all – could be done within the flash-framework, but that would make the presentation heavy and sluggish, especially if one were to introduce loads of  heavy video content (maybe even distorted such). And it would increase the designing load more than time would allow in certain cases. The whole workflow is about 90-95%  GUI driven and the remaining hand-coding is relatively simple.

That’s about it…. I will show examples later on, especially the one  which prompted me to  cook this all up – and others, as they come along.

This week, my teaching and another major project I am working on is demanding my attention. But I am slowly shifting my focus towards Vancouver and starting to plan the stuff (including multimedia) I am supposed to deliver from there – given the 10-hour time difference against us. It’ll be a true challenge – and I welcome it as such.

**************

(UPDATE 3.11.2009: Just saw Julian Treasure’s presentation in TED on soundscapes – and I warmly recommend it . Pause and contemplate for a moment: there is enormous relevance in his words for this line of multimedia work.)

3 Comments

  1. Cheri
    November 3, 2009

    Welcome to WordPress. I look forward to visiting your blog and reading about what you do, especially in photography, a subject I love.
    More later, after work.

    Reply
  2. kkuukka
    November 3, 2009

    Thank you Cheri for taking the time to write. I have been following you and Andreas talking quite often and I have really enjoyed it. Got some influence from things you’ve said into my own teaching… just the other day I quoted to a student. “The only place where you find success…”.
    Besides, I could just copy-paste your About Me into my blog… except that I do not teach writing and do not live in Northern California, and I do not attend Stanford. Wine, pasta etc. all the rest fits like a shoe.

    Reply
  3. DocImages – multimedia production « Kari Kuukka Photography – a blog
    November 20, 2009

    […] process of creating this is explained in my earlier post – and also in the multimedia itself – so I won’t get more into it here. The idea […]

    Reply

Leave a Reply