{"id":951,"date":"2016-08-23T14:50:06","date_gmt":"2016-08-23T13:50:06","guid":{"rendered":"http:\/\/ofalcao.pt\/blog\/?p=951"},"modified":"2016-08-23T15:06:10","modified_gmt":"2016-08-23T14:06:10","slug":"lego-laser-harp-part-ii","status":"publish","type":"post","link":"https:\/\/ofalcao.pt\/blog\/2016\/lego-laser-harp-part-ii","title":{"rendered":"LEGO laser harp – part II"},"content":{"rendered":"
This post is part 2 of 2 of \u00a0LEGO Laser Harp<\/a><\/div>

About 10 years ago I offered my wife a M-Audio USB MIDI Keyboard<\/a> and installed Ubuntu Studio<\/a> on a computer so she could play some piano at home. She was so amazed with the possibility to generate music sheet while playing that almost accepted the idea of using Linux… almost \ud83d\ude42<\/p>\n

I remember that at that time I used timidity++<\/a> as a software MIDI synthesizer, tuned ALSA<\/a> (one of the several Linux sound systems, perhaps the most generally used) and the preemptive kernel to work almost flawlessly with the Creative Labs sound card. My wife didn’t enjoy the KDE experience, Firefox was OK for her but OpenOffice were terribly with the spreadsheets she used and finally, when our first kid was born, she attended some English lessons at Wall Street Institute and we found out that the online lessons required an odd combination of an old version on Java, ActiveX and IE… so she returned to Windows XP and never looked back.<\/p>\n

10 years is a LOT of time in computer systems but ALSA is still around, even on ev3dev. So I installed timidity++ and tried to play a MIDI file… to find that an ALSA module that is not currently available in ev3dev kernel is required just for MIDI.<\/p>\n

I googled for alternatives and found fluidsynth<\/a> with an unexpected bonus: there is a quite interesting python library, mingus<\/a>, that works with fluidsynth. So I installed it in my Ubuntu laptop and in a few minutes I was playing harp – amazing!<\/p>\n

sudo apt-get install fluidsynthsudo easy_install mingus\r\npython\r\n>>> from mingus.midi import fluidsynth\r\n>>> from mingus.containers.note import Note\r\n>>> fluidsynth.init(\"\/usr\/share\/sounds\/sf2\/FluidR3_GM.sf2\", \"alsa\")\r\n>>> fluidsynth.set_instrument(1, 46)\r\n>>> fluidsynth.play_Note(Note(\"C-3\"))<\/pre>\n
In the previous example I just import the fluidsynth and Note parts of the library, initialize fluidsynth to work with ALSA loading the soundfount that cames with it, choose harp (instrument number 46) and play C3.<\/p>\n
Well and polyphony? The correct way is to use a NoteContainer<\/p>\n
from mingus.containers import NoteContainer\r\nfluidsynth.play_NoteContainer(NoteContainer([\"B-3\", \"C-3\", \"F-3\"]))<\/pre>\n
but the lazy way is… just play several notes in a fast sequence.<\/p>\n
So, let’s do it in the ev3dev!<\/p>\n
Oops, fluidsynth also needs an ALSA module not available in current ev3dev kernel.<\/p>\n
I’m not a linux music expert. Not even a linux expert! So after some more googling I gave up and asked for help<\/a> in ev3dev’ GitHub project. And once again David accepted to include ALSA MIDI suport in future kernels, so I’ll just wait a bit.<\/p>\n
Oh, but I can’t wait…<\/p>\n
And if I read the color sensors in ev3dev and play the music in my laptop?<\/p>\n
ALSA, once again, suports something like client\/server MIDI communication with “aseqnet” and “aconnect” commands and some people are already using it with Raspberry Pi<\/a>!<\/p>\n
Yeah, I should have guessed… “aconnect” requires an ALSA MIDI module that is not available in current ev3dev kernel.<\/p>\n
OK, let’s use MQTT: configure my EV3 as a publisher and my Ubuntu laptop as a subscriber and just send some notes as messages.<\/p>\n
On the EV3:<\/p>\n
sudo apt-get install mosquitto\r\nsudo easy_install paho-mqtt<\/pre>\n
The publisher script is “harp-mqtt-pub.py”:<\/p>\n
#!\/usr\/bin\/env python\r\n\r\nfrom ev3dev.auto import *\r\nfrom time import sleep\r\nimport paho.mqtt.client as mqtt\r\n\r\nDELAY = 0.01\r\n\r\n# should have an auto-calibrate function\r\nAMB_THRESHOLD = 9\r\n\r\nsensor1 = ColorSensor('in1:i2c80:mux1')\r\nsensor1.mode = 'COL-AMBIENT'\r\nsensor2 = ColorSensor('in1:i2c81:mux2')\r\nsensor2.mode = 'COL-AMBIENT'\r\nsensor3 = ColorSensor('in1:i2c82:mux3')\r\nsensor3.mode = 'COL-AMBIENT'\r\nsensor4 = ColorSensor('in2')\r\nsensor4.mode = 'COL-AMBIENT'\r\nsensor5 = ColorSensor('in3')\r\nsensor5.mode = 'COL-AMBIENT'\r\nsensor6 = ColorSensor('in4')\r\nsensor6.mode = 'COL-AMBIENT'\r\n\r\n# there is no sensor7 yet, I need another MUX\r\n\r\ns1 = 0\r\ns2 = 0\r\ns3 = 0\r\ns4 = 0\r\ns5 = 0\r\ns6 = 0\r\ns7 = 0\r\n\r\nclient = mqtt.Client()\r\nclient.connect(\"localhost\",1883,60)\r\n\r\nprint 'Running...'\r\n\r\nwhile True:\r\n\u00a0\u00a0\u00a0 key_touched = False\r\n\u00a0\u00a0\u00a0 s1 = sensor1.value(0)\r\n\u00a0\u00a0\u00a0 s2 = sensor2.value(0)\r\n\u00a0\u00a0\u00a0 s3 = sensor3.value(0)\r\n\u00a0\u00a0\u00a0 s4 = sensor4.value(0)\r\n\u00a0\u00a0\u00a0 s5 = sensor5.value(0)\r\n\u00a0\u00a0\u00a0 s6 = sensor6.value(0)\r\n#\u00a0\u00a0\u00a0 s7 = sensor7.value(0)\r\n\r\n\u00a0\u00a0\u00a0 if s1 < AMB_THRESHOLD:\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 client.publish(\"topic\/Harp\", \"C-3\")\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 key_touched=True\r\n\u00a0\u00a0\u00a0 if s2 < AMB_THRESHOLD:\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 client.publish(\"topic\/Harp\", \"D-3\")\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 key_touched=True\r\n\u00a0\u00a0\u00a0 if s3 < AMB_THRESHOLD:\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 client.publish(\"topic\/Harp\", \"E-3\")\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 key_touched=True\r\n\u00a0\u00a0\u00a0 if s4 < AMB_THRESHOLD:\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 client.publish(\"topic\/Harp\", \"F-3\")\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 key_touched=True\r\n\u00a0\u00a0\u00a0 if s5 < AMB_THRESHOLD:\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 client.publish(\"topic\/Harp\", \"G-3\")\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 key_touched=True\r\n\u00a0\u00a0\u00a0 if s6 < AMB_THRESHOLD:\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 client.publish(\"topic\/Harp\", \"A-3\")\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 key_touched=True\r\n#\u00a0\u00a0\u00a0 if s7 < AMB_THRESHOLD:\r\n#\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 client.publish(\"topic\/Harp\", \"B-3\")\r\n#\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 key_touched=True\r\n\r\n\u00a0\u00a0\u00a0 if key_touched == True:\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 sleep(DELAY)<\/pre>\n
On the Ubuntu laptop side:<\/p>\n
sudo easy_install paho-mqtt<\/pre>\n
The subscriber script is “harp-mqtt-sub.py”<\/p>\n
#!\/usr\/bin\/env python\r\n\r\nimport paho.mqtt.client as mqtt\r\nfrom mingus.midi import fluidsynth\r\nfrom mingus.containers.note import Note\r\n\r\nEV3_IP = \"192.168.43.35\"\r\n\r\nSOUNDFONT = 'Concert_Harp.sf2'\r\nINSTRUMENT = 46 # Harp\r\n\r\nNOTES = ['C-3','D-3','E-3','F-3','G-3','A-3','B-3']\r\n\r\ndef on_connect(client, userdata, flags, rc):\r\n\u00a0\u00a0\u00a0 print(\"Connected with result code \"+str(rc))\r\n\u00a0\u00a0\u00a0 client.subscribe(\"topic\/Harp\")\r\n\r\ndef on_message(client, userdata, msg):\r\n\u00a0\u00a0\u00a0 global i\r\n\u00a0\u00a0\u00a0 if (msg.payload in NOTES):\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 print msg.payload\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 fluidsynth.play_Note(Note(msg.payload))\r\n\u00a0\u00a0 \u00a0\r\nclient = mqtt.Client()\r\nclient.connect(EV3_IP,1883,60)\r\n\r\nclient.on_connect = on_connect\r\nclient.on_message = on_message\r\n\r\nfluidsynth.init(SOUNDFONT, \"alsa\")\r\nfluidsynth.set_instrument(1, INSTRUMENT)\r\n\r\nclient.loop_forever()<\/pre>\n
And guess what? It works!!! I just love linux and open source!<\/p>\n