Project sketches so far

Mar 6, 2011

I’ve gotten a bit further on my exam project. If you’re new here, you can read about it here.

I’ve tried mapping some simple knobs to the playhead position of a clip, thus visualizing the position of the knob:

But that stuff is easy, and doesn’t hold any conceptual significance. What’s more interesting is that I’ve come a long way towards figuring out how to make the performance more available to the audience, and visualize what is being played live and not. I’ve come to call it “virtual artists”, meaning that additional “band members”, playing the pre-arranged parts can be projected into the performance.

At first I had the idea of making the Virtual artists based on primitive shapes, or some kind of minimalist characters of some kind. But I’m afraid it would turn the whole project into a character building workshop, since making such figures emote can be really hard. Right now I’m working on projecting “copies” of the actual artist on stage, so you’ll have one physical and several other virtual copies of the same guy, playing as a band. So this way the virtual band members will be “playing the playback” very visually, instead of hiding it, enforcing transparency and honesty in the performance.
Additionally, the virtual band members themselves can be controlled just like the music, so, as an example, one parameter on some instrument can correspond to the head rotation of one of the projections. Or anything, really.

I spent some days experimenting with 3D, making objects that move around in space, trying to see how they could interact with the physical objects. But when you have the 3D-glasses on, everything that’s not 3D gets thrown out of whack, and it can be quite hard on the eyes at length. It’s also very dependent on the audience’s position in space, to draw the picture out correctly. So now I’m trying to find ways of getting a three-dimensional effect, without all the stereoscopic humbug.

Another element with the performance I’m trying to figure out, is the table. Most electronic instruments need to sit on a table in front of the artist, like a DJ. But this table creates a distance to the audience I really don’t like.
One idea I’ve had is to break up the table into smaller squares, to create space between them. Also, setting them up diagonally towards the audience makes them a passage, not a blockage.

Through the use of fancy lights, table parts can become “invisible” during the performance, shifting focus from one part to the other. So if one of the virtual band members are playing a really nice solo, I can shut out the entire rest of the “band” for a bit. This can be done either simply by muting the lights on areas, or by using some semi-transparent materials and such, combined with light sources inside the tables themselves.
The tables can also be used as direct interface elements in the music, using cameras and projectors together to make touch sensitive areas. This way the side of the table could become a big fader for example, or the square top can switch to become a X/Y-pad in a special breakdown.

The sky is really the limit here, since I’m only making a proof of concept, and don’t have to think about budgets too much. Implementing these ideas in an actual setting making it really interactive and not pre-rendered is only a matter of code though. After my exam, I would love to take this project further, to use it in an actual setting instead of it being mainly theoretical. So scalability is key here.

I have partnered up with Njål Paulsberg for the music, so I know I’ll have some awesome tunes to work with. You might know Njål from his solo act Njaal, his band project Put Your Hands Up For Neo-Tokyo, or as key member of the super-group Young Dreams.
We are now going through tons of his old song sketches, to find something to fit the project neatly. I’m telling you, he’s got some gold on that computer of his.

What I have outlined here is basically the framework for a performance. Going forward, I’ll be working more on the what, not the how.I can barely wait.

Thanks for reading! If this triggers anything in your own head, I’ll be thrilled to hear about them in the comments, don’t be afraid to share!

Starting my exam project

Feb 9, 2011

I just started the first work on my final exam project, and I’ll post regularly about it as it moves forward.

In the project, I try to explore ways to bring the audience of an electronic concert closer to an understanding of the music, through visualizing and interpreting the performance.
In electronic music, most artist stand on stage (many even sit on chairs while playing, what kind of signal is that) look like they’re checking their email. Electronic instruments are ubiquitous objects, usually incomprehensible to the audience. So I try to engage the audience more, and give them a larger understanding of what happens, through visual communications.

Right now I’m kind of stressed out, since the definitions of the product itself is still completely unknown to me, and I have a mountain of work to do. But it’s fun work, so I’m not really worried. Here are some of my initial inspiration, if you would like to watch where my head is at right now. I’ve also set up a process reference blog at javalightmagic.tumblr.com, and I’ll post my own sketches and further work right here. See you soon!

Hiroaki Umeda | Duo | while going to a condition from www.labomatique.com on Vimeo.

a story of light from Jerome Monnot on Vimeo.