Tension
From Bob's Basement
A generic control surface for live multimedia performances
Project Leader: Naxxtor
Status: Planning
Contents |
Problem
To make a live performance of electronic music, DJing or VJing entertaining, the performer must react to his audience and shape the performance in real time. Increasingly performers are using PC based hardware to generate their media, using applications such as freej or LiVES, or using a custom built application. Most often these applications are controlled usnig normal keyboard input, mouse movements or increasingly joystick input.
These devices are not made for live performances, and I can tell you that using a joystick to control 4 different video feeds is impractical at best. Also, support for joysticks and other devices varies between operating systems greatly. Their support in software is often limiting, requiring software hacks to map joystick movements to, say, a keyboard or mouse input. Other control surfaces exist, but only have a limited implementation in software and platform.
One of the worst offenders for this is what M-audio call "iControl". It is a control surface with sliders, lots and lots of buttons and even rotary encoders. But it's impossible (or at least ridiculously implausible) to use it with anything other than GarageBand on a Mac.
Solution
A generic device which has all the required input devices, which can be used on almost any platform, with any software, with little to no problem.
This device will include all the controls that a live performer (e.g. a DJ or VJ) may require in their performance, and will emulate their output into devices which are very widely supported already.
Inputs
The control surface (Tension) needs to have a minimum of 3 different types of input.
- Momentary Button
Simply a momentary push to make switch to trigger or toggle an event (e.g. trigger a drum sound, start a video loop)
- Infinite encoder
Usualy rotary, often used for "scratching" or scrubbing audio or video in real time, or for making a selection, or for chaging a parameter, or for ... anything you can think of!
- Finite analog encoder
Such as a crossfader in a DJ's mixer, or a fader on a vision mixer (like this, seen right of center, bottom), which allows one parameter to be altered, or blend two or more parameters together between two fixed points (minimum and maximum)
Other types of input can also be used, for example an ADC connected to two electrodes placed in a tank of water. The level of the water dictates the level of the output, allowing you to control a parameter by sloshing the water around. You can go really crazy with your input devices if you want to. The selection of input controls also allows the use of passive feedback, or even active feedback (through for example haptic feedback using electric motors) which is controlled by the host (see D'Groove).
Outputs
In order to make this very, very portable, I suggest that we use some generic keyboard and mouse USB controllers, possibly even using their sensors too. It will in effect be a concept keyboard (except the concept part is up to whoever is using it :p) and a very disfigured mouse, and a mini usb hub thrown to tidy things up.
So the output will be a USB connection to the host. Providing the host recognises generic usb HID's, then it should recognise this no problem.
It will then be handled in whatever way the OS handles normal keyboard and mouse devices. Under Windows, signals from two keyboards will be merged when they get to userland. I'm not entirely sure how Linux or *BSD handle multiple keyboards, but I believe that you can seperate them out if required, or use a merged metadevice like /dev/mice.
Development
This project could be developed in lots of different ways.
Touchscreen
To aid the performer, the momentary buttons could be triggered from a programmable touchscreen, which contains different icons depending on what it is being used for. You could also switch between multiple 'profiles' on the fly, to allow for more flexibility of the device. However it should be carefully considered what the tactile response would be like, and it's durability. In particular with momentary buttons, performers may give it quite a beating (when laying down a heavy drum beat, for instance). It wouldn't be terribly satisfying to feel like you're tapping a beat into a unresponsive sheet of wood.
New interface
As mentioned before, there are many many more kinds of analog inputs that could be used. A canadian company Phidgets make USB development boards which make making such devices quite simple, and the project is open source. They have an interface board which can take 8 analog inputs, which could be attached to either one of their sensors or another device of our ... um, devising. This could in fact entirely replace the keyboard and mouse controllers - however since we can modify the (multi platform) drivers as we see fit, we could still emulate a keyboard if required.
Other kinds of interface exist, including an IP protocol, OSC (Open Sound Control). Since the protocol is fully open, it would be reletively easy to write our own software to use it.
Velocity measurement
In drum machines, momentary buttons to trigger sound samples also have a velocity value attached to each press. This allows the drum beats to have a dynamic range depending on how hard the button was hit. I'm really not sure how you'd implement this in the generic model, though.
Tactile Feedback
As mentioned earlier, it might be interesting to provide the user with active feedback - for example when scrubbing through an audio track, peaks are recognised and some torque is applied to the rotary encoder to provide resistance. Also other forms of tactile feedback such as buttons resisting being pressed or "clicking" when a certain criteria is met when they are pressed.
Manufacture and Progress
I think we could very easily knock up a basic prototype in an hour or two. Simply gut a usb keyboard and mouse, rewire some keys to momentary push-to-make switches, mount a stick on the mouse scrollwheel and that'd be it. Where we'd go from there depends on the outcome of the test....
