Dave Perrett

Using a Gaming Joystick as a MIDI Pitch Bend Wheel

This past January I traveled to Worcester, MA to perform at The Firehouse's yearly event "Noise Brunch". The material I wanted to debut required a good deal of gratuitious soloing and noodling on a keyboard MIDI controller (and when I solo, I tend to lean quite heavily on the pitch bend wheel). I was taking the bus to Worcester and wanted to travel light, but my 25-key Akai MPK Mini, which fit nicely in my backpack, did not have a pitch wheel.

So I started to think: since MIDI controllers work by simply sending event messages (note on, note off, etc.) to the device they are controlling, there shouldn't be any reason why I couldn't send messages to the synth from two separate devices - note on and note off messages from the MPK Mini, and the pitch bend messages from somewhere else. It hit me: what about a gaming joystick? The action of a joystick is similar to that of a pitch wheel - both return to a center resting position when you are not actively bending. Plus, bending the pitch with a joystick would look cool and add a fun performative element.

I quickly scanned Craigslist and found someone located in Coney Island selling a USB joystick for ten dollars - perfect. I headed to Coney later that night to make the pickup, and then headed home to start hacking.

I decided to use ChucK as the language for this project, as I'd enjoyed working with it in the past and I knew that it had some built-in classes for dealing with human interactive devices (i.e. joysticks).

Constructing MIDI Messages

A little background: MIDI messages are structured as three-byte words. They’re a little bit like system calls in assembly language, in that the first byte - known as the status byte - determines which type of message is being sent (note on, note off, program change, etc.), and the second and third bytes act as the parameters.

Before actually dealing with the joystick, I decided to write some test code to make sure it would be possible to manually build MIDI messages and send them to a soft-synth.

// instantiate a MidiOutput object and open a connection on port 0.
MidiOutput midiOut;
midiOut.open(0);

Running the above code caused a new MIDI output device to appear under qjackctl's MIDI tab. Sweet!

After spending some time squinting at this handy table of MIDI messages, I determined the relevant status bytes for the types of messages I wanted to use:

To actually build the MIDI message, to be sent via the MidiOut object, I needed to instantiate a MidiMsg object.

MidiMsg msg;

144 => msg.data1;  // status byte - note on, channel 1
30  => msg.data2;  // a low note
127 => msg.data3;  // highest possible velocity

midiOut.send(msg); // send the message

Success - the soft-synth played a note!

Interacting with the Joystick

The next step was to see if I could get the joystick to work. Thankfully, it worked right away and there was no unnecessary drama with drivers or loading additional kernel modules. Running cat /dev/input/js1 in the terminal and moving the joystick around a bit printed some promising gibberish to the screen.

Now confident that the joystick worked and was talking to my computer, I wrote a quick ChucK script to capture joystick events and print the output (based on this example from the ChucK documentation), in order to see what kind of output - eventually to be transformed into MIDI pitch bend messages - I would be dealing with.

msg.axisPosition() returned float values ranging from -1.000 to 1.000 representing the current position of the joystick, and msg.which() returned an integer representing the axis on which the joystick was moving: 0 for motion along the x-axis, and 1 for motion along the y-axis.

I decided that since I wanted to mimic the action of a pitch bend wheel on a keyboard (up and down, no side to side), I would only pay attention to y-axis values, and ignore any messages where msg.which() returns 0.

I noticed that msg.axisPosition() returned a negative value when the joystick was in quadrants I or II (pushing forwards / down) and a positive value when it was in quadrants III or IV (pulling up / backwards). It felt more intuitive to me for the pitch to go up when I pushed the joystick forward, and vice versa, so I needed to make sure to account for this in my arithmetic and 'flip' the output.

Transforming the Raw Output

In the MIDI protocol, a peculiarity of pitch bend messages is that they only take one argument instead of two: a fourteen-bit wide value indicating the position of the pitch wheel. In decimal, this is a value ranging from 0 to 16383, where 8192 is the wheel at resting position.

The reason for this is that 128 discrete values (the amount provided by seven bits of data) do not provide enough granularity for the pitch to sound like it is bending smoothly - the listener will hear the individual "steps" as the pitch moves. Fourteen bits of data provide enough granularity between steps to trick the human ear into hearing a smooth bend.

The pitch bend message still needs to take the format of a status byte followed by two parameter bytes, so the fourteen-bit wide position value needs to be split into two separate seven-bit wide values (one containing the seven high-order bits, and one containing the seven low-order bits) in order to be passed as a MIDI message.

This was an opportunity for me to trot out ChucK's bitwise operators. To get the seven low-order bits, I needed to use the and operator (&) to mask the joystick's position value with 127 (or 00000001111111). This zeroed out the seven high-order bits and left me with the low-order half.

In order to isolate the seven high-order bits, I needed to shift (>>) the original position value seven bits to the right (the empty positions created to the left are padded with zeroes).

// mask bendValue with 1111111 to get the least significant bits
bendValue & 127 => lsb;

// shift bendValue 7 bits right to get the most significant bits
bendValue >> 7 => msb;

The next step was to implement the arithmetic needed to transform the output of msg.axisPosition() to values that would be useful to me. If msg.axisPosition() returned negative, I would need to take its absolute value, then multiply by 8191, and then offset it by adding 8192, since 'resting position' needed to be 8192 rather than zero. And if the return value was positive, it would be the same as above except I'd need to multiply the entire result by -1.

if(hidMsg.which == 1 && hidMsg.axisPosition <= 0) {
  ( (Std.fabs(hidMsg.axisPosition) * 8191) + 8192)         \
    $ int => bendValue;
}
else if(hidMsg.which == 1 && hidMsg.axisPosition > 0) {
  ((-1 * (Std.fabs(hidMsg.axisPosition) * 8191)) + 8192)   \
    $ int => bendValue;
}

Putting it All Together

To wrap things up, I wrote a helper function to build and transmit the MIDI messages. sendMsg() takes the following parameters: a reference to a MidiOutput object, a reference to a MidiMsg object, and the three bytes of the message.

fun void sendMsg(MidiOut midiOut, MidiMsg msg,
                 int data1, int data2, int data3) {

  data1 => msg.data1;
  data2 => msg.data2;
  data3 => msg.data3;
  midiOut.send(msg);

}

Lastly, I placed all of this inside of an event loop continuously capturing the output of the joystick, and I was good to go. I was making sweet music with one hand on the keyboard, and one hand on the joystick.

You can see the full code here. Thanks for reading!