I had 2 projects this year for Christmas for the City – first, the online streaming, and second, to take my idea of a giant walking piano (like in the movie Big), and turn it into reality. The main purpose of this was twofold – first, to give kids a feature that uses interactive technology to bring a smile to their faces, and second, for me to learn what works and doesn’t work well when it comes to interactive displays for projects down the road.
The basic idea is this: Kids walk across the keys of the piano, activating some kind of a pressure sensor, and the keys light up and the key’s sound plays. Sounds fairly simple, right? In all reality, it’s a lot harder than it looks in the movies. 🙂 I decided to go with a 16-key piano (to give me 2 octaves), and to make the design a lot simpler, just to skip the black keys altogether. While I didn’t expect to end up with something as elaborate as this picture, I did intend to create the same experience for a person walking across it.
First, the material for the “keys” needed to be lightweight but strong, and readily available. I settled on a Lexan material from Lowe’s. They come in 18” x 24” rectangles, which, when cut in half, give you a 9” x 24” piano key. 2 feet seemed to be wide enough for my purposes, so the next step is to find the best way to let it ride above some kind of a switch, and let it hit the switch when a few pounds of pressure are added to it (i.e. a child’s foot).
The picture to the right is when I was testing it in my living room. I had a curved plastic stand that some time ago held a cable modem or access point or something, and I experimented with a couple of different types of switches.
The most readily available supply of switches came from Radio Shack, but like a lot of things from Radio Shack, not all of them were all that great. The general idea here is that the switches would be recessed into the piano wood just enough to allow them to be pushed, but not damage them.
The first contender was a Radio Shack Mini SPST Momentary Pushbutton Switch (275-1556). At less than $3 a piece, this appeared to be what I was looking for, but was actually a massive fail. You have to push these down so hard, and even when pushed down, the “momentary contact” isn’t even that consistent, so I wasn’t comfortable trying to make this actually work.
The next type of switch I tried was a SPST Detect Switch (275-008). It’s a little pie shaped switch that gets pushed down into the casing to activate the momentary contact. This had the response I was looking for and took very little travel before the contact was connected. I decided to go ahead and roll with this. The only problem was, most Radio Shacks only stock about 3 of these at a time. It took about a week of visiting Radio Shacks in 3 cities to accumulate enough of these things to wire up 16 keys. At one Radio Shack, someone had stolen all of them and left the plastic packaging behind. Nice.
Now, it was time to make…
The idea here was pretty simple. Get a few 14-ft 2×4’s, and space 2×4 supports every 9” where the edges of the keys rest on them. Mount the sensors in the supports and off we go, piece of cake.
I learned a valuable lesson here – measurements are never always exact. I tried to build the frame first, measure it out, attach it, and then put the keys on, expecting everything to line up perfectly. There is apparently a law of woodworking, and a law of Lowe’s lexan products, that measurements don’t always come out like they do on paper. Add in a little bit of warping wood, and I rebuilt this frame twice before I figured out that I should build and fit the frame around the keys, instead of trying to stay in my idealistic happy bubble where everything works out like it does on paper and fairies dance around singing duets with unicorns.
Here is where I got the first few keys fitted, and it starts to look like the top of a keyboard.
To hold the keys in place, I used molding from Lowe’s. Cheap stuff and worked great for keeping the keys from shifting around.
Wiring It Up
The next step, while the framing was fairly accessible, was to go ahead and get the wiring harness in place. I started out using 1-conductor solid wire, running 2 wires to each key, and after 3 keys, realized this would be a complete and unmanageable mess. I switched over to using A/V 2-conductor + ground install cable (XLR audio cable) that I had left over from an old install. This made things a lot more manageable by keeping it to 1 cable per key.
This added some thickness to the cable that I didn’t really anticipate, so a few extra holes and it worked out fine.
For the actual switches, I carved out a slot in each key support with a Dremel router bit. I left just enough clearance on top for the sensor switch to stick up above the top of the wood. I also found that chunks of foam weatherstrip (the kind you would stick around your home windows to seal them) worked great for the keys to ride on. It held the weight of the lexan above the switch, but also compressed enough for it to hit the switch with no problem.
Also, these switches had a hole on the left side of them that I used to tack a short nail through. This held it in place quite nicely.
Once these were in place, I soldered the key cables up to the switches, which completed the wiring for each key. Once stapled and labeled, it ended up looking pretty clean. This part of the design was very solid and worked extremely well.
Once I attached the keys, it ended up looking like this…
This piano was to be computer controlled. The main challenge here is how to make the piano talk to the computer. I wanted to do this via USB… so the plan was:
- Get enough inputs together on one microcontroller to sense on/off for 16 piano keys.
- Write a simple .NET Micro Framework controller OS that will sense these and notify the computer via USB.
- Write a host application on the computer that will talk to said microcontroller and take actions (play sound, turn on lights).
I went down this road, looking at Digi Connect, Tahoe, Arduino, etc, and after longer than I care to admit, realized something… what’s a microcontroller that speaks USB, just has a bunch of on/off contacts, is cheap, and readily available? A computer keyboard.
Things just got a LOT simpler, with a $12 Logitech keyboard from Wal-Mart… or so I thought. I didn’t have a whole lot of time to finish this, so this decision was final, and could probably be considered one of the best from a cost/simplicity standpoint, and one of the worst from a design standpoint.
A computer keyboard is actually really, really simple. When you type, your keys simply push two plastic sheets together that are less than a millimeter apart. When certain spots (where the keys are) connect, this connects 2 pins together on the keyboard circuit board. It’s this combination of pins that determines which key value to send to the computer over USB.
The new plan was:
- Rip out the controller, and solder cables to the controller contacts.
- Wire the piano key switches to a combination of contacts.
- In the computer, map certain keys to notes and lights.
There are 2 parts of a keyboard controller that really made this plan suck:
- These boards aren’t designed to be soldered. The contact surfaces are really, really close together, making it darn near impossible to get soldered cleanly without shorting two contacts together.
- These boards are cheap and fragile. In all reality, they don’t have to be robust – a plastic sheet lays against them and that’s pretty much it. But once I soldered cables on, the slightest pull and the copper contacts would simply peel off.
It was too late for me to change it by the time I realized these 2 major points (okay, it was the night before the event, because I thought this would be super easy and left it to the last minute), so I rolled with it. After soldering on 20 wires, out of a possible 108 key combinations, I ended up with about 22 working key combinations. Some of these were special keys (tab, shift, etc) that I didn’t want to use, but I ended up with exactly 16 letters. Score. Once connected, I could step on piano keys, and type random letters into Notepad. I was home free on the “input” side of things at this point.
Now on to the software…
I used MIDI to make the sounds on the piano. I used an open-source library called NAudio to drive MIDI sounds. NAudio is a wrapper for DirectSound and runs as a 32-bit target. There are tons of tutorials on controlling MIDI, so I won’t cover that here.
In the hollow space below the keys of the piano, I filled it with Christmas lights. At Wal-Mart, a pack of 100 Christmas Lights was about $1.50, so they became a cost effective lighting medium. I connected these up to DMX-controlled dimmers. For those not familiar with DMX, it stands for Digital MultipleX protocol, and is the standard communication protocol used in production lighting systems today. It is really RS-485 (in essence, RS-232 that can daisy chain to up to 32 devices together and travel 1000 feet without loss) running a constant stream of byte values that represent sequential channels and their current control/intensity value.
Most programmers today don’t often come into contact with lighting control protocols used on large intelligent lighting and dimming systems, so there’s not a whole lot of code out there to work with it. There are really 2 ways of having a normal computer talk to lighting systems:
- Through a DMX-over-Ethernet protocol called ARTNet. This requires an Ethernet network, and a converter that will translate Ethernet DMX packets into RS-485 DMX traffic to get to the dimmers. The converters usually run between $300 up to $5,000.
- Through a USB DMX device.
My office lights are already controlled via ARTNet, and I already had code for it, but the converter cost was a deterrent for this project. Keeping costs to a minimum, I went the USB DMX device route with a really cheap device – the Enttec Open DMX USB. Enttec has simply taken a chip from FTDI (232BM) that is a USB to RS-485 converter.
The major pain point here is that FTDI only offers unmanaged code DLLs to work with, and a C# wrapper for it that was mediocre and halfway-working at best.
Low-level, byte-by-byte programming using non-managed Interop is pretty much not my cup of tea, but to make a long story short, I decided to stick it out and coded a serial communication wrapper in C# based on the DMX protocol spec, and 6 hours and lots of frustration later, some lights were coming on!
Putting It All Together with Software
The lighting code took a LOT longer than I thought, and truth be told, I still hadn’t written the software to put this all together and actually make the piano come alive an hour before the doors opened for the event.
Using a simple .NET dataset, I constructed a XML file format that looked like this:
I made a very quick Windows Forms application that took the keyboard input (key down / key up events), adjusted the DMX output to light up the key, and played/stopped the specified MIDI note value. This application looked like this:
One of the problems I had to solve was key repeat – when you hold a key down in Windows, Windows will automatically repeat the key every few milliseconds. Each repeat is actually fired as a separate KeyDown event on the form. A simple array of “currently pressed keys” solved this pretty quickly, but there was somewhat of a limit on how many keys could be processed at a time because of this.
By the time I finished tweaking, it was literally minutes until the doors opened. The application actually ran in Visual Studio debug mode the first night, until I got it transferred over to a netbook to run the other 2 nights. One of the unintentional but nice side effects of running it straight out of Visual Studio became the ability to update the code whenever a wave of people left the room. 🙂 So, about halfway through the first night, I added a little timer to change the instrument every 30 seconds. Interesting side note here – MIDI bagpipes are a great way to clear out the room quickly to make room for the next wave to come in. There’s not much more annoying than a high-pitched bagpipe note from a kid sitting on one of the keys and not moving.
Below is the video of the finished product in action:
- Find a better microcontroller than a computer keyboard with small solder points.
- A giant piano made out of 2×4’s, lexan, and dimmers weighs in at over 350lbs and is very awkward. (Thanks goes to the 8 people who helped get this thing up and down the stairs since it wouldn’t fit in the elevator.)
- The weather strip started to wear out after a few nights of beating and some had to be supplemented when keys would start to stick. I would use larger strips next time.
The piano was a great success. It did exactly what it was supposed to do, and brought smiles to a lot of kids faces. Can’t wait to use the experiences gained from this project to make something else.