The Web-Connected MIDI Trumpet Robot
Raspberry Pi, Teensy and Web MIDI — How I Hacked Together an Interactive Musical Demo for the Chrome Dev Summit
Since we couldn’t get the artificial latex lips of the robot to work reliably, I went with the backup plan — using a small speaker to play the sounds, and plug this speaker right into the trumpet’s mouthpiece, to get the sound travel through the trumpet and come out of its bell.
The Chrome Dev Summit will be happening tomorrow, and I finally got all the pieces of the projects working together. Hooray!
So how does it work?
High Level Overview
Lines 12 and 14 are where the actual MIDI messages sent to the robot, each consists of three values (a command, note index, and velocity — how low the note should be played). These values are then received by the robot which interprets them.
The robot has a 3D-printed finger mechanism which is driven by servo motors. This mechanism allows the robot to depress the trumpet’s keys in response to the MIDI commands. You can learn how I designed this mechanism in my “Designing 3D Printable Mechanisms in OpenSCAD” blog post.
The robot also has to translate the incoming MIDI messages into sound output. It does it using a synthesizer code I hacked together using the Web Audio API:
The Robot connects to a computer (or a smartphone) through USB, and identifies itself as a MIDI device. A small Teensy 3.2 board takes care of speaking USB and decoding the incoming MIDI messages. It runs a bunch of Arduino code that controls the servos for the finger mechanism, and also forwards the MIDI messages to a Raspberry Pi over UART.
The purpose of the Raspberry Pi is to synthesize the audio for the robot. It runs a small Node.js server that opens a Web Socket and forwards all the incoming MIDI messages from the Pi’s UART interface to the Web Socket. It uses the serialport module for the UART part, and socket.io for Web Sockets.
The Raspberry Pi also runs headless chrome that loads a web page. This web page listens for MIDI messages coming from the Web Socket, and then uses the Web-Audio trumpet synthesizer that I mentioned earlier to create the sound for the trumpet.
The sounds comes out of the Pi’s audio jack, amplified using a TDA2030 external amplifier (I found mine as a DIY kit in a local electronics shop, you can find one too by googling for “TDA2030 kit”), and fed to the speaker.
So in short, the computer sends a MIDI message through USB. It is received by the Teensy device which updates the Servo motors and also send a copy of the message to the Raspberry Pi through UART. The node.js process that runs on the Raspberry Pi receives this message, and sends it to the web page running on headless Chrome through a web socket, which in turn uses the web audio API to synthesize the sound. Pretty complex setup, isn’t it?
Ground Loop Noises
When I put everything together for the first time, the amplifier would not only amplify the trumpet sound — it would also amplify some annoying noise as soon as I plugged the Raspberry Pi. This noise seemed to correlate with CPU and WiFi activity.
After much googling (and frustration), I found out that this was caused by a Ground Loop — basically, using the same power supply both for the Raspberry Pi and the amplifier. Google also revealed that I would need an Isolation transformer to fix this. You can order them for about $1.5 USD from ebay, but finding one in Israel a day before leaving for the conference was a tough challenge. Luckily, some friend in the local maker community had one piece and saved the day!
I wrote this editor app in the plane on my way to the bay area. It is an Angular application. It uses Monaco Editor, the engine behind Visual Studio Code for the editor part, and Firestore for storing the code snippets created by the users.
Monaco Editor turned out to be amazing! It takes just a few lines of code to get syntax highlighting, code auto-completion, typescript error detection, code formatting and code transpilation — basically, everything you’d expect from a modern, powerful source code editor.
Today is The Big Day! 🎺
I’m just about to head to the Chrome Dev Summit. I can’t wait to see how people will interact with the Robot and what kind of creative code snippets they will come up with. Fingers crossed for everything to work well! 🤞