I had some LEGO Move Hubs with me with a new Pybricks firmware image and a new version of ‘pybricksdev’ python library that works with them.
Move Hub has an internal IMU sensor. And batteries. And wireless communication. So I could make a kind of wireless game controller.
But Facebook keeps hitting me with ads of wireless MIDI drumsticks. I can do this with LEGO:
It wasn’t difficult – the Pybricks script is very short, inspired on an Arduino project from Andrea Richetta (‘Air Drum with Arduino Nano 33 IoT‘) – I just calculate the size of the acceleration vector (in fact it’s square since Move Hub doesn’t support float numbers so no complex math functions) and print something out whenever it changes ‘suddenly’:
from pybricks.hubs import MoveHub
from pybricks.tools import wait
ACC_THRESHOLD = 170
ACC_MARGIN = 50
TIME = 40
KEY = '1'
hub = MoveHub()
max_v = 0
x,y,z = hub.imu.acceleration()
v = x*x+y+y+z+z
if v > ACC_THRESHOLD:
max_v = v
elif v < max_v-ACC_MARGIN:
max_v = 0
Then I just used ‘pybricksdev’ library in a pyhton script on my laptop to capture the output of the Move Hub and redirect it with ‘xdotool’ to a MIDI controller (VMPK) that generates MIDI events usable by Hydrogen (a MIDI drum engine).
I could have used something on python to directly generate the MIDI events instead of using VMPK… but I’m a lazy guy on holidays (or call it ‘Agile’ 😀 )
I found out that everytime I tested Tuxie McPython the two Technic Hubs were getting hot. Very hot.
Also batteries were draining fast. I hate changing batteries on stationary robots/automata. For my MINDSTORMS EV3 I tend to use the Li-Ion battery as a PSU, leaving it connected to a 12V/1.5A wall power adapter.
So I ordered 2x USB Power Box from PV Productions. They fit on the Technic Hub in the place of the 6xAA battery holder and they can withstand 9V/2A so it seamed a good and clean method.
Less than 3 days after they arrived I melted one 🙁
The 4 Large Angular motors were consuming too much current. To proper close the holes I was forcing the motors to strongly press the fingers against the chanter but never thought that would require 2A (probably even more).
So I changed the code to reduce the fingers strength and opted for other fingertips also made with protective foam but a different one, apparently less “foamy” and a bit thinner. I glued over the EVA foam fingertips I already tried before and cut them a bit larger so the hole would get more covered (no fotos but can be seen used on last video).
Even so, rechargeable NiMH batteries were still draining too fast (and the hubs were still getting hot). So I picked some alligator cables and 2.1 mm barrel adapters and used 2 of my wall power adapters to power the Hubs. I set them for 9V (measured 9.1 with no load) and connected them – it worked:
but while playing I got a blinking LED and very often one of the hubs just powered off. The adapters are rated 1.5A… so the motors were still demanding too much current.
As I said before… I don’t have other proper motors – the smaller version of this motors, sold with the 51515 Robot Inventor, are not easy to find and it would be very expensive to buy 8 of them on Bricklink. I could use the BOOST external motors but I don’t have 8 of them and I am not so sure they have enough power to properly close the holes. Same for Technic motors (except maybe the larger ones) and none of them have internal zero reference (not a major blocker but its something that helps).
So I have 8 motors that demand too much current and two hubs that get too hot and shutdown… I have no other proper motors… but I do have other two identical hubs.
So last version of Tuxie McPython now uses 4 Technic Hubs, each controlling two fingers. Code is essentially the same, just less “if…then…elif” and a shorter “command vocabulary”:
So I now send just 4 different commands (‘0’, ‘1’, ‘2’ or ‘3’) to each hub instead of previous 16 but from the controller side have to send those messages to 4 hubs instead of just 2. My only concern was that my laptop couldn’t proper handle 4 simultaneous BLE sessions but thankfully it works fine.
With 2 motors per hub I can now use NiMH batteries without blinking LEDs. The hubs still get hot (not so hot as before but still very warm) and the batteries will probably run down after a few songs but for that I already ordered 9V/2A power adapters and more alligator cables.
Programming the LEGO Technic Hubs with Pybricks is not difficult (much, much more easier than using assembly with PIC controllers in the 90’s). But one thing is making motors move the way you want and other is making a choreography that sounds like music.
The chanter has 8 holes – 7 are placed above, the other one is downside. There are several sites on the web showing the proper fingertips positions and more than a few adopted a notation of black and white circles (representing closed and open holes). Much like binary notation.
So I chose to use ‘0’ for motor in rest position (closing the hole) and ‘1’ for motor in some other position that opens the hole. So a single byte can represent any possible note (and of course also lots of other combinations).
Using the most significant bit for the higher pitch hole (the one downside) and the less significant bit for the low pitch hole (the one near the end tip of the chanter) this is the result:
As I am (was) using two hubs, it seemed logic to group each representation in two nibbles of 4 bits so the same program running on both Hubs could position its 4 fingers just by receiving a char with a value between ‘0’ (all holes closed) and ‘F’ (all holes open):
The main program loop would be in fact just a bunch of if-then-elif’s based on the key pressed (if using Pybricks Chrome-based IDE) or the character received (if using pybricksdev or accessing the Nordic UART Service directly):
char = stdin.read(1)
if char == '0': #0000
elif char == '1': #0001
and this really worked – pressing some keys on Chrome made the fingers move.
But that’s not enough for playing music. What movements should be done and for how long for this apparently random fingering to produce something that doesn’t sound like me stepping on a cat’s tail?
XenonJohn was gentle enough to include his Arduino code with the Instructables for his Bagpipe Playing Robot. He used a 3-char notation for his music where the first char represents an embellishment (like a gracenote or a doubling), the second char represents the main note and the last char represents the duration of the note. For instance:
means playin a ‘G’ for 1/4 of the time of a full duration note (I believe this means a ‘quarter note’ or a ‘crotchet’) with no embellishment.
And he also included two songs in his source code: ‘Amazing Grace’ and ‘Scotland The Brave’.
Now this is something I can use.
Instead of implementing his whole code with Pybricks I just adapted his song sequence. I opted for not representing embellishments since they can be represented by (fast) sequences of finger positions (assuming that there is enough bandwidth between the “controller” and each hub to send everything and the hub processes everything fast enough… if I find that is not the case I might return to XenonJohn’s notation).
So this is my way of representing a song:
sequence = [
where the first character is the note and the second character is the duration (still using Ardu McDuino, too lazy to change that for now):
j = 1 = whole note (semibreve) Z = 1/2 = half note (minim) R = 1/4 = quarter note (crotchet) N = 1/8 = eight note (quaver) L = 1/16 = sixteenth note (semiquaver) K = 1/32 = thirty-second note (demisemiquaver)
But for gracenotes this doesn’t work good since a grace note is played for 1/32 and the next note should be reduced from this value. So I had three timing notations for the 3 cases where a gracenote is used in Ardu McDuino songs (before a note with a timing of R, Z or N) reducing these timing by 1/32:
x = R – 1/32 = 1/4 – 1/32 = 1/4.57 y = N – 1/32 = 1/8 – 1/32 = 1/10.67 z = Z – 1/32 = 1/2 – 1/32 = 1/2.13
these look strange but are easy to use in my code since I define a tempo for my song and each timing is just the multiplication of this tempo by this numbers.
First attempt looked promising but not quite right, then I find a silly math error and now I finally have something that sounds familiar:
After some attempts, decided to gave up (at least for now) using LEGO parts as fingertips. Even the LEGO rubber pads were too hard and narrow to close enough the holes for the chanter to play something that reminds music – for instance I could not notice any difference between a G (all holes closed) and a A (opening just the hole near the chanter endpoint).
I tried Technic rubber links and even rubber tires… no luck.
So I started with small circles of EVA foam glued to LEGO 2×2 dishes attached to the same Technic connectors I was using with the LEGO rubber pads. It looked much like those ear pads used on headphones… and then I remembered that Neil Fraser used exactly that for his Robotic Bagpipe Chanter:
These new fingertips improved sound quality but only for the higher notes.. still having problems with G, A, B…
Adding adhesive felt pads didn’t make it better.
But replacing EVA foam with packaging protective foam (not sure what material it is made off) made a big difference:
Great, I can finally produce distinguishable notes! Still far from perfection because air pressure inside the balloon has a strong influence on note pitch and sometimes the reed sounds very bad (and sometimes doesn’t sound at all)… but it’s still a major milestone:
A little progress has been made. And a lot have been learnt.
So my fingertips worked… sort of. But the ‘microphones’ were not large enough to close the holes. And rigid plastic is not good, rubber or something else soft is better to cover the holes. I am now using the 1×1 rubber pads used on LEGO Technic tracks (like in the BOOST set):
Still not perfect but better.
Also found out that rubber balloons have some non-linear properties. After reaching some internal pressure level, if I keep inflating it the pressure drops drastically. So my idea of using a pressure sensor to regulate air volume is not so easy to achieve. Elastic hysteresis doesn’t help either…
I increased the LEGO air compressor air flow by changing the 2x Power Function ‘L’ motors with ‘XL’ motors and gearing it faster. But still not enough for long tests and still too noisy. So for now I am using a small air pump used on Arduino and other projects. At 5V it’s almost strong enough to ditch the “bag” (the balloon).
This is now the current state of the project:
LEGO MINDSTORMS EV3
Wi-Fi Power Plug
Mini air pump
Mindsensors pressure sensor for EV3
MINDSTORMS EV3 medium motor
LEGO custom air valve
LEGO Powered Up Technic Hub (2x)
LEGO Powered Up Large Angular motor (8x)
The air pump is powered by a power wall adapter connected to a Wi-Fi Power Plug. The plug state (ON/OFF) is controlled through UDP messages sent from the EV3.
The EV3 also reads the pressure inside the pneumatic circuit with a Mindsensore pressure sensor and tries to keep pressure steady by turning the air pump ON whenever pressure drops bellow a threshold.
The air valve is used only at start, when we need to inflate the balloon. The EV3 medium motor is used to close the valve (squeezing the aquarium air tube between the balloon and the chanter) and when there is enough air in the balloon the valve is opened.
Each Technic Hub is responsible for controlling 4 LEGO fingers with 4 Powered Up Large Angular motors. Probably smaller and faster motors would be better since they have much more strength than needed (if I don’t take care, they bend the “fingers” easily)… but that’s what I have with internal zero reference, it helps coding (but I did I had a problem with two of them, looks like they came with a large offset, near 100º… had to replace them).
The whole setup can now be described as an IoT project. I don’t have everything integrated yet so the EV3 functions are still independent of the Technic Hub functions:
the EV3 can be controlled manually and/or with MQTT messages
the Technic Hub can be controlled with Jupyter Notebook (using pybricksdev) or with two Chrome browsers running the Pybricks IDE
I have a video of current stage in action – please be warned, terrible noise:
In this video the reed has an elastic ring applied to it (I ordered a bag with the chanter). It does reduce the noise a bit but it also changes the required air pressure to play it so I ended up with a very large balloon… and that also changes the reed’s sound. Will have to make further tests to choose the best balloon size and try to adjust my code for it.
In the meantime, wainting for two orders to arrive:
some other practice reeeds to check if that makes any difference in the sound (I feel that my reed sounds like a cracked cane… my coding is bad, my setup is not the best but it does sound odd)
a pair of USB adapters for the Technic Hubs, I am tired of replacing batteries
So confination has been tough. One year at home, one year of remodeling and repairing on my building – a 9 floor building with 36 apartments and it seems many neighbors saw the major repairs as an opportunity to make their own remodeling.
So I almost went crazy (or maybe I already am) and found myself thinking on playing bagpipes on the roof, scaring the falcons that appear at spring near my village (one even made a nest on this building, some years ago). And of course, retaliating to all the noise.
Of course, I don’t know how to play bagpipes (or any other instrument). I don’t even own a bagpipe – just a toy bagpipe that we bought on Scotland on our honeymoon:
It’s not a proper bagpipe, the drones are false and the bag looses air but the chanter (the long tube at right with a reed inside that makes a hell of a noise when air passes through it) works and is also detachable so I use it as a sort of practice chanter (the reed inside is also replaceable).
Almost 2 years ago I already tried to fill this bagpipes with my brute-force LEGO air compressor:
but it needs a lot of air and it also needs somone to squeeze the bag. I am too lazy for that.
But I read a lot. And found people making their own bagpipes with plastic bags, rubber or latex gloves and even balloons… like these dudes that use a melodica and 2 human slaves to pump air into the balloon:
So I decided to try my own robotic LEGO bagpiper proof of concept. I just needed enough air, a balloon and a chanter so I ordered a practice chanter (more affordable than an usual bagpipes chanter and a bit easier to play because usally it requires less air pressure).
And the first attempt looked promising:
and then all the usual problems on a proof of concept project started 🙂
First thing I found was that LEGO pneumatic tubes aren’t large enough to supply enough air fast enough for the reed inside the chanter to play. So the last tube, connecting the balloon to the chanter’s mouth piece, had to be replaced by aquarium tube (1 or 2 mm wider than LEGO’s).
I also needed a larger balloon – softer than the first I used so less pressure needed from the air compressor side.
And finally a needed a better air compressor. 4 MINDSTORMS motors acting over 8 large Technic pneumatic pumps wasn’t enough (and also makes a lot of noise – I want most noise to be from the bagpipes).
Luckily I already had a LPEpower SYS Inline 3 engine I used on my last LUG event. It’s an air engine – you inject compressed air and get mechanic rotation – but it can be reverse used (probably with less efficiency).
So my first attempt evolved to this:
The SYS Inline 3 engine was using a LEGO Power Functions L motor but I felt that I could obtain better results if I applied more power… so I added a second motor and used gears to combine the power (and also rotate it a bit more faster). That resulted in almost a minute of “playing”, not bad. But also showed up a new problem: too much air pressure on the reed makes it block, not generating any sound and also making the balloon accumulate even more air. For now, as I am not yet controllong the flow, my only option is manually reducing the air flow from the air compressor.
With a powerful compressor I added the first “fingers” using stick tack to close the chanter holes:
Not good but it was a start.
So I searched my LEGO spare parts after something that could close the hole and I found 2 similar parts (images are from Bricklink site):
The microphone is a bit smaller than the towball so I used it for the smaller holes on the chanter (the higher pitch notes). Also used a motor to controlling the finger for the first time – so finally we have some Python to explain McPython’s name:
It seemed OK so I added the rest of the fingers (7 fingers upside, 1 thumb downside). With 8 Large Powered Up motors (from Volvo set or SPIKE / Robotic Inventor sets) and 2 Control+ hubs running Pybricks Tuxie McPython has reached Proof of Concept level:
So for Alpha Release level this is the roadmap:
coordinate the fingers – that requires me to control the 2 Control+ hubs so I will use Jupyter Notebook
control the air flow – that requires a custom valve to squeeze the aquarium tube, a pressure sensor to check the balloon’s current state and some sort of relay to turn the compressor power ON or OFF (so a MINDSTORMS EV3 will join soon, explaining the Tuxie name)
learn basic bagpipes music – by far the toughest part (but my wife already started helping me converting music scores to notes and temporizations)
Not sure if I deal with it now but also a stronger air compressor will be needed on the long run that allows Tuxie McPython to play for at least 2 minutes (time enough to drive everyone crazy before the police arrives)
Second LEGO DMX fixture: a Fire Machine 🙂 Similar script on EV3, listening to DMX channel 1 and reacting to just two values:
255 turns the BIC lighter ON
0 turns it OFF again
Working quite good with QLC+ as a DMX controller (at 9 Hz DMX frame rate) and other DMX fixtures on the loop (in this video a Fonestar PAR-18L set to DMX channel 2):
from ev3dev2.motor import MediumMotor, OUTPUT_A, SpeedPercent
from time import sleep
port = pyftdi.serialext.serial_for_url('ftdi://ftdi:232:AG0KAEQB/1', baudrate=250000)
#works good with QLC+ at 9 Hz
port.timeout = 0.0226
# clear input buffer (not sure if everything needed)
channels = *512
CH_FIRE = 1
fire = MediumMotor(OUTPUT_A)
fire.position_sp = 0
fire.speed_sp = 1560
fireON = False
dmx_data = port.read(514)
i = 0
error = False
for b in dmx_data:
if i == 0 or i == 1:
if b != 0:
error = True
channels[i-2] = b
i = i + 1
# first 2 bytes should be zero ("start-of-packet procedure" and "start frame")
if channels[CH_FIRE -1] == 255:
if fireON == False:
# turn Fire ON
print('Fire triggered ON')
fireON = True
fire.position_sp = 3050
elif channels[CH_FIRE -1] == 0:
if fireON == True:
# turn Fire OFF
print('Fire triggered OFF')
fireON = False
fire.position_sp = 0
print("FireON: ", fireON)
Installed QLC+ on my laptop. Great tool, giving up on bying a DMX controller.
QLC+ recognizes my FTDI cable adapter (and my Waveshare adapter):
Plugin Device Input Output Feedback
DMX USB 1:USB-RS485 Cable (S/N:FT4NMHF6) - X -
This plugin provides DMX output support for DMXKing ultraDMX range, Enttec DMX USB Pro, Enttec Open DMX USB, FTDI USB COM485 Plus1, Vince USB-DMX512 and compatible devices.
No input support available.
1: USB-RS485 Cable (S/N: FT4NMHF6)
Device is operating correctly.
Driver in use: libFTDI
Protocol: Open DMX USB
DMX Channels: 512
DMX Frame Frequency: 30Hz
System Timer Accuracy: Patch this widget to a universe to find out
In the plugin settings I can change Mode. Open TX (above) allows sending DMX messages, Open RX allows receiving. And it works.
Using the two USB adapters – one on my laptop and the other on EV3 – I used QLC+ to test my scripts on EV3:
using QLC+ as controller, my EV3 python scripts (pyftdi-based) can decode messages on any of the 512 channels… as long as I lower DMX frame frequency to 9 Hz.
using the Ev3 as controller (python scripts using dmx485) I can receive messages on all 512 channels on QLC+
using laptop as a controller (python dxm485 instead of QLC+) the same pyftdi-based scripts on EV3 no longer work… unless I force sender object to stop immediately after the first frame:
It seems dmx485 forces a DMX Frame Frequency too high for my decoding scripts to keep up – it seems to be a limitation of EV3 because reverting the scenario (EV3 as a controller with dmx485-based script and laptop as a receiver with pyftdi-based script) it works.
Good that QLC+ has an option to adjust it – 9 Hz isn’t good for light animations but is more than enough for what I want. Will use it as a DMX controller from now on.
Interesting, QLC+ also has a MIDI plugin. It can listen to MIDI channels and take actions… like activating a DMX fixture when a selected note is played.
So… MIDI brings me to DMX… DMX brings me back to MIDI. Such a small world.
So my second DMX fixture arrived today: a Fonestar PAR-18L.
I had ordered a few weeks ago with a cheap Waveshare USB-RS485 adapter but the order took ages so last week I went for the Ibiza PAR-MINI-RGB-3 and the FTDI USB-RS485-WE-1800-BT cable.
My adapter based on the FTDI cable can control both PARs
My cheap Cameo Control 6 controller can only control the Ibiza PAR
An adapter based on the Waveshare adapter cannot control the Ibiza PAR (sometimes I get a short flash of the intended color)
So my Cameo DMX controller was a bad starter choice, I will need a better controller to confirm my receiver code is working correctly. Will try to get a better one next month, with a few more channels and (if possible) some MIDI integration.
Now even more strange: the Waveshare adapter does work… but only with the new Fonestar PAR.
I changed the Ibiza address to 008 and kept Fonestar at 001 the joined both on the DMX chain and changed my python script to use both. With the FTDI cable both PARs work. With the Waveshare only the Fonestar work, no matter if I put it as the first or the last of the chain so it doesn’t seem to be a termination problem.
As the Cameo controller works with the Ibiza but not with the Fonestar AND the Cameo controller data can only be read by my scripts when I cute B (DMX-) wire… I suspect both the Cameo and the Ibiza have A/B (DMX+/DMX-) reversed or even stranger deviations from the DMX standard.
probably also first LEGO DMX fixture ever in the multiverse 🙂
So my Cameo Control 6 DMX controller is not probably full DMX compliant – I can only read 6 channels and even so only when cutting one of the data wires off. But who cares? (I do, but this month’s budget is already exceeded so I will only get a more decent DMX controller for Easter).
But using PyFtdi on EV3 works:
pip3 install ftdi
I could not confirm my FTDI url with ‘ftdi_urls.py’ like I did on my laptop but used same url (‘ftdi://ftdi:232:FT4NMHF6/1’) and it worked:
from ev3dev2.motor import MediumMotor, LargeMotor, DcMotor, OUTPUT_A, OUTPUT_B, OUTPUT_C, SpeedPercent
from ev3dev2.port import LegoPort
from time import sleep
lights_port = LegoPort(OUTPUT_C)
lights_port.mode = 'dc-motor'
port = pyftdi.serialext.serial_for_url('ftdi://ftdi:232:FT4NMHF6/1', baudrate=250000)
channels = *6
pan = MediumMotor(OUTPUT_A)
pan.position = 0
pan.speed_sp = 90
tilt = LargeMotor(OUTPUT_B)
tilt.position = 0
tilt.speed_sp = 90
lights = DcMotor(OUTPUT_C)
lights.duty_cycle_sp = 0
dmx_data = port.read(8)
i = 0
error = False
for b in dmx_data:
if i == 0 or i == 1:
if b != 0:
error = True
channels[i-2] = b
i = i + 1
# print("Channels:", channels)
pan.on_to_position(SpeedPercent(25), channels * 90/255 - 45)
tilt.on_to_position(SpeedPercent(25), 45 - channels * 90/255)
lights.duty_cycle_sp = channels * (100/255)
Now it’s time to plan a few DMX fixtures: a bubble machine, a fog machine, a spark machine, a confetti cannon, a flamethrower… 😀