About 10 years ago I offered my wife a M-Audio USB MIDI Keyboard and installed Ubuntu Studio on a computer so she could play some piano at home. She was so amazed with the possibility to generate music sheet while playing that almost accepted the idea of using Linux… almost 🙂
I remember that at that time I used timidity++ as a software MIDI synthesizer, tuned ALSA (one of the several Linux sound systems, perhaps the most generally used) and the preemptive kernel to work almost flawlessly with the Creative Labs sound card. My wife didn’t enjoy the KDE experience, Firefox was OK for her but OpenOffice were terribly with the spreadsheets she used and finally, when our first kid was born, she attended some English lessons at Wall Street Institute and we found out that the online lessons required an odd combination of an old version on Java, ActiveX and IE… so she returned to Windows XP and never looked back.
10 years is a LOT of time in computer systems but ALSA is still around, even on ev3dev. So I installed timidity++ and tried to play a MIDI file… to find that an ALSA module that is not currently available in ev3dev kernel is required just for MIDI.
I googled for alternatives and found fluidsynth with an unexpected bonus: there is a quite interesting python library, mingus, that works with fluidsynth. So I installed it in my Ubuntu laptop and in a few minutes I was playing harp – amazing!
sudo apt-get install fluidsynthsudo easy_install mingus
python
>>> from mingus.midi import fluidsynth
>>> from mingus.containers.note import Note
>>> fluidsynth.init("/usr/share/sounds/sf2/FluidR3_GM.sf2", "alsa")
>>> fluidsynth.set_instrument(1, 46)
>>> fluidsynth.play_Note(Note("C-3"))
In the previous example I just import the fluidsynth and Note parts of the library, initialize fluidsynth to work with ALSA loading the soundfount that cames with it, choose harp (instrument number 46) and play C3.
Well and polyphony? The correct way is to use a NoteContainer
from mingus.containers import NoteContainer fluidsynth.play_NoteContainer(NoteContainer(["B-3", "C-3", "F-3"]))
but the lazy way is… just play several notes in a fast sequence.
So, let’s do it in the ev3dev!
Oops, fluidsynth also needs an ALSA module not available in current ev3dev kernel.
I’m not a linux music expert. Not even a linux expert! So after some more googling I gave up and asked for help in ev3dev’ GitHub project. And once again David accepted to include ALSA MIDI suport in future kernels, so I’ll just wait a bit.
Oh, but I can’t wait…
And if I read the color sensors in ev3dev and play the music in my laptop?
ALSA, once again, suports something like client/server MIDI communication with “aseqnet” and “aconnect” commands and some people are already using it with Raspberry Pi!
Yeah, I should have guessed… “aconnect” requires an ALSA MIDI module that is not available in current ev3dev kernel.
OK, let’s use MQTT: configure my EV3 as a publisher and my Ubuntu laptop as a subscriber and just send some notes as messages.
On the EV3:
sudo apt-get install mosquitto sudo easy_install paho-mqtt
The publisher script is “harp-mqtt-pub.py”:
#!/usr/bin/env python
from ev3dev.auto import *
from time import sleep
import paho.mqtt.client as mqtt
DELAY = 0.01
# should have an auto-calibrate function
AMB_THRESHOLD = 9
sensor1 = ColorSensor('in1:i2c80:mux1')
sensor1.mode = 'COL-AMBIENT'
sensor2 = ColorSensor('in1:i2c81:mux2')
sensor2.mode = 'COL-AMBIENT'
sensor3 = ColorSensor('in1:i2c82:mux3')
sensor3.mode = 'COL-AMBIENT'
sensor4 = ColorSensor('in2')
sensor4.mode = 'COL-AMBIENT'
sensor5 = ColorSensor('in3')
sensor5.mode = 'COL-AMBIENT'
sensor6 = ColorSensor('in4')
sensor6.mode = 'COL-AMBIENT'
# there is no sensor7 yet, I need another MUX
s1 = 0
s2 = 0
s3 = 0
s4 = 0
s5 = 0
s6 = 0
s7 = 0
client = mqtt.Client()
client.connect("localhost",1883,60)
print 'Running...'
while True:
key_touched = False
s1 = sensor1.value(0)
s2 = sensor2.value(0)
s3 = sensor3.value(0)
s4 = sensor4.value(0)
s5 = sensor5.value(0)
s6 = sensor6.value(0)
# s7 = sensor7.value(0)
if s1 < AMB_THRESHOLD:
client.publish("topic/Harp", "C-3")
key_touched=True
if s2 < AMB_THRESHOLD:
client.publish("topic/Harp", "D-3")
key_touched=True
if s3 < AMB_THRESHOLD:
client.publish("topic/Harp", "E-3")
key_touched=True
if s4 < AMB_THRESHOLD:
client.publish("topic/Harp", "F-3")
key_touched=True
if s5 < AMB_THRESHOLD:
client.publish("topic/Harp", "G-3")
key_touched=True
if s6 < AMB_THRESHOLD:
client.publish("topic/Harp", "A-3")
key_touched=True
# if s7 < AMB_THRESHOLD:
# client.publish("topic/Harp", "B-3")
# key_touched=True
if key_touched == True:
sleep(DELAY)
On the Ubuntu laptop side:
sudo easy_install paho-mqtt
The subscriber script is “harp-mqtt-sub.py”
#!/usr/bin/env python
import paho.mqtt.client as mqtt
from mingus.midi import fluidsynth
from mingus.containers.note import Note
EV3_IP = "192.168.43.35"
SOUNDFONT = 'Concert_Harp.sf2'
INSTRUMENT = 46 # Harp
NOTES = ['C-3','D-3','E-3','F-3','G-3','A-3','B-3']
def on_connect(client, userdata, flags, rc):
print("Connected with result code "+str(rc))
client.subscribe("topic/Harp")
def on_message(client, userdata, msg):
global i
if (msg.payload in NOTES):
print msg.payload
fluidsynth.play_Note(Note(msg.payload))
client = mqtt.Client()
client.connect(EV3_IP,1883,60)
client.on_connect = on_connect
client.on_message = on_message
fluidsynth.init(SOUNDFONT, "alsa")
fluidsynth.set_instrument(1, INSTRUMENT)
client.loop_forever()
And guess what? It works!!! I just love linux and open source!
I will keep waiting for David Lechner to include ALSA MIDI support in ev3dev’ kernel. I’m not so sure if there is enough horsepower in the EV3 to load a soundfont and play it with acceptable latency but if I can at least use the MIDI client/server functionality I can drop MQTT.
An interesting possibility that this client/server design allows is to scale my harp easily: with just a second EV3 (2 MUX each) I can make a 13-string harp with almost no modification on my code.
