LEGO laser harp – part I

This post is part 1 of 2 of  LEGO Laser Harp

This is an idea I’ve been postponing for several months but the time has finally come: an laser harp.

After tinkering with lasers, fog, sound, color sensors and python I found myself wondering how to give a proper use to all that. Then I remembered Jean-Michel Jarre and how his laser harp made such a big impression on me at late 80’s when I finally decided “hey, i wanna study Electronics!”

For a first version, let’s call it “a proof of concept”, I just want a simple 7-string harp that can play the basic 7 notes. Polyphony would be great but I doubt that the EV3 sound capabilities allow that (and I cannot afford the brute force solution of using 7 EV3 so that each one plays only a single note).

So in the last months I’ve been buying EV3 color sensors and I finally have 7. Since the EV3 only has 4 input ports I need some kind of sensor multiplexer but thanks to mindsensors.com I already have one EV3SensorMux (and a second one is on the way, from an european distributor – portuguese customs DO stink!)

With 2 MUX it’s possible to connect up to 8 sensors to the EV3. Since I just want 7 “strings” I am considering using an 8th sensor to control the amplitude of the notes. I’ll try an ultrasonic sensor but I’m not sure if it has enough “wideness” to cover the whole harp, let’s see.

So of course I’ll be using ev3dev and python.

Using the EV3SensorMux is easy: just plug it to an input port and ev3dev immediately recognizes it:

lego-port port8: Registered 'in1:i2c80:mux1' on '3-0050'.
lego-port port8: Added new device 'in1:i2c80:mux1:lego-ev3-color'
lego-sensor sensor0: Registered 'ms-ev3-smux' on 'in1:i2c80'.
lego-port port9: Registered 'in1:i2c81:mux2' on '3-0051'.
lego-port port9: Added new device 'in1:i2c81:mux2:lego-ev3-color'
lego-sensor sensor1: Registered 'ms-ev3-smux' on 'in1:i2c81'.
lego-port port10: Registered 'in1:i2c82:mux3' on '3-0052'.
lego-port port10: Added new device 'in1:i2c82:mux3:lego-ev3-color'
lego-sensor sensor2: Registered 'ms-ev3-smux' on 'in1:i2c82'.
lego-sensor sensor3: Registered 'lego-ev3-color' on 'in1:i2c80:mux1'.
lego-sensor sensor4: Registered 'lego-ev3-color' on 'in1:i2c81:mux2'.
lego-sensor sensor5: Registered 'lego-ev3-color' on 'in1:i2c82:mux3'.

Even better: by default all 3 mux ports are configured for the EV3 color sensor, just as I wanted!

NOTE: as of today (kernel version ‘4.4.17-14-ev3dev-ev3’) my EV3 autodetection only works when booting with a non-default configuration:

sudo nano /etc/default/flash-kernel

 LINUX_KERNEL_CMDLINE="console=ttyS1,115200"

sudo flash-kernel
sudo reboot

this was suggested to me by David Lechner in another issue, hope will be fixed soon.

To use the color sensors in python I just need to know their ports. With the MUX in port ‘in1’ and 6 color sensors connected, these are the ports to use:

in1:i2c80:mux1
in1:i2c80:mux2
in1:i2c80:mux3
in2
in3
in4

And to play a note in python I just need to know it’s frequency to use with Sound.tone() function, so:

C3 = [(130.81, TONE_LENGHT)] 
D3 = [(146.83, TONE_LENGHT)] 
E3 = [(164.81, TONE_LENGHT)] 
F3 = [(174.61, TONE_LENGHT)] 
G3 = [(196.00, TONE_LENGHT)] 
A3 = [(220.00, TONE_LENGHT)] 
B3 = [(246.94, TONE_LENGHT)]

And so this was the first script for my harp:

#!/usr/bin/env python

from ev3dev.auto import *

TONE_LENGHT = 150

C4 = [(261.64, TONE_LENGHT)]   #Do4
D4 = [(293.66, TONE_LENGHT)]   #Re4
E4 = [(329.63, TONE_LENGHT)]   #Mi4
F4 = [(349.23, TONE_LENGHT)]   #Fa4
G4 = [(392.00, TONE_LENGHT)]   #Sol4
A4 = [(440.00, TONE_LENGHT)]   #La4
B4 = [(493.88, TONE_LENGHT)]   #Si4

AMB_THRESHOLD = 9

sensor1 = ColorSensor('in1:i2c80:mux1')
sensor1.mode = 'COL-AMBIENT'
sensor2 = ColorSensor('in1:i2c81:mux2')
sensor2.mode = 'COL-AMBIENT'
sensor3 = ColorSensor('in1:i2c82:mux3')
sensor3.mode = 'COL-AMBIENT'
sensor4 = ColorSensor('in2')
sensor4.mode = 'COL-AMBIENT'
sensor5 = ColorSensor('in3')
sensor5.mode = 'COL-AMBIENT'
sensor6 = ColorSensor('in4')
sensor6.mode = 'COL-AMBIENT'

# there is no sensor7 yet, I need another MUX

s1 = 0
s2 = 0
s3 = 0
s4 = 0
s5 = 0
s6 = 0
s7 = 0

while True:
    s1 = sensor1.value(0)
    s2 = sensor2.value(0)
    s3 = sensor3.value(0)
    s4 = sensor4.value(0)
    s5 = sensor5.value(0)
    s6 = sensor6.value(0)
#    s7 = sensor7.value(0)
  
    if s1 < AMB_THRESHOLD:
        Sound.tone(C4).wait()
    if s2 < AMB_THRESHOLD:
        Sound.tone(D4).wait()
    if s3 < AMB_THRESHOLD:
        Sound.tone(E4).wait()
    if s4 < AMB_THRESHOLD:
        Sound.tone(F4).wait()
    if s5 < AMB_THRESHOLD:
        Sound.tone(G4).wait()
    if s6 < AMB_THRESHOLD:
        Sound.tone(A4).wait()
#    if s7 < AMB_THRESHOLD:
#        Sound.tone(B4).wait()

So whenever the light level over one of the color sensor drops bellow AMB_THRESHOLD a note will play for TONE_LENGHT milliseconds.

Unfortunately the sound is monophonic (just one note can be played at a time) and it doesn’t sound like an harp at all – it sounds more like my BASIC games on the ZX Spectrum in the 80’s.

So I tried Sound.play(Wave File) instead. Found some harp samples, converted them to .wav files at 44100 Hz and it really sounds much better… but the length of the samples I found is to big so the “artist” have to wait for the note to stop playing before moving the hand to another “string”. Not good and also not polyphonic.

Next post I’ll show a better approach for both quality and polyphony: MIDI.

ev3dev – using IRLink with python

I got myself a HiTechnic IRLink sensor.

As of today (August 2016) ev3dev already recognizes the IRLink as a nxt-i2c sensor but there’s no language support for it. David Lechner suggested me using the “direct” attribute to communicate directly with the IRLink at I2C level.

Last time I wrote something mildly related to I2C was about 20 years ago for a Microchip PIC project but well… why not?

So after lots of trial and error, reading several times the LEGO Power Functions RC Protocol and shamelessly copying code from Mike Hatton (“Parax”), Xander Soldaat and Lawrie Griffiths I found on GitHub, RobotC forum and LeJOS forum I fanally managed to control a PF motor in ComboPWM mode.

In the following video, I’m increasing the motor speed (all 7 steps) then decreasing it again until it stops:

This is the python script running in the EV3:

#!/usr/bin/python

#
# based mainly on RobotC code from Mike Hatton ("Parax") and Xander Soldaat
# but also on LeJOS code from Lawrie Griffiths
#

# assumes IRLink at Input 1 as sensor0

import sys
from time import sleep

# channel: 0..3
# motorA, motorB: 0..7

channel = 0
for motorA in (1,1,2,2,3,3,4,4,5,5,6,6,7,7,6,6,5,5,4,4,3,3,2,2,1,1,0,0):

  motorB = motorA

  iBufferSize=2
  iBuffer = bytearray(iBufferSize)

  iBuffer[0] = ((0x04 | channel) << 4) | motorB
  iBuffer[1] = motorA << 4
  check = 0xF ^ (0x04 | channel) ^ motorB ^ motorA
  iBuffer[1] = iBuffer[1] | check

  oBufferSize=14
  oBuffer = bytearray(oBufferSize)

  # clear all positions
  for i in range (0,oBufferSize):
    oBuffer[i]=0

  oBuffer[0]=0x80    # Start Bit

  oBufferIdx = 0

  for iBufferByte in range (0,2):
    for iBufferIdx in range (0,8):
      oBuffer[1 + (oBufferIdx / 8)] |= (0x80 >> (oBufferIdx % 8) )
      if ( ( ( iBuffer[iBufferByte] ) & (0x80 >> (iBufferIdx % 8) ) ) != 0 ) :
        oBufferIdx = oBufferIdx + 6
      else:
        oBufferIdx = oBufferIdx + 3

# Stop bit
  oBuffer[1+ (oBufferIdx / 8)] |= (0x80 >> (oBufferIdx % 8) )

  tailIdx = 1 + (oBufferIdx / 8) + 1

  # Tail


  if (tailIdx == 10):
    oBuffer[tailIdx]= 0x10 # IRLink message payload length
    register = 0x43
  else:
    oBuffer[tailIdx]= 0x11
    register = 0x42

  oBuffer[tailIdx+1]= 0x02 # IRLink in Power Functions Mode
  oBuffer[tailIdx+2]= 0x01 # IRLInk Start transmission 


# clear IRLink (not sure if correct but seems to improve)

  fd = open("/sys/class/lego-sensor/sensor0/direct", 'wb',0)
  fd.seek(0x41)
  fd.write(chr(0x46))
  fd.write(chr(0x44))
  fd.write(chr(0x4C))
  fd.write(chr(0x50))
  fd.close()
  sleep(0.1)

  for i in range(0,5):
    fd = open("/sys/class/lego-sensor/sensor0/direct", 'wb',0)
    fd.seek(register)
    for oBufferIdx in range (0,oBufferSize):
      fd.write(chr(oBuffer[oBufferIdx]))
    fd.close()

    # Power Functions timings (for a 5-command burst)
    if (i==1):
      sleep(0.064)
    elif (i==5):
      sleep(0.096)
    else:
      sleep(0.080)

 

Running ev3dev on a Raspberry Pi 3

A few days ago the ev3dev project launched a great feature: nightly image builds. Right after that I got a received a notice that they included in the image for Raspberry Pi 2/3 support for onboard the Bluetooth and needed to test it.

So I did test it. And found out that onboard Bluetooth indeed works… as also onboard Wi-Fi… as also the Brick Pi, no need to disable BT. Yeah, no more USB dongles!

The procedure is very simple – the really important step is freeing the hardware serial port for the BrickPi (both the onboard Bluetooth and the BrickPi need a UART so a soft UART (“miniuart”) is used for BT instead of the default one.

  • get the latest nightly image build for the Pi2/Pi3 (mine was 26 July 2016) and restore it to a microSD card
  • insert the card in the Pi3
  • connect an USB keyboard and a HDMI display to the Pi3
  • power up the Pi
  • login (robot + maker) – if you cannot see the login prompt change to the proper console with Alt+F1 or Alt+F2 or Alt+F[n]
  • run ‘sudo connmanctl’ to configure BT and Wi-Fi (see this tutorial on how to configure Wi-Fi from command line; for BT just run ‘sudo connmanctl enable bluetooth’)
  • edit the ‘/boot/flash/config.txt’ and uncomment these 4 lines:
    • dtoverlay=brickpi
    • init_uart_clock=32000000
    • dtoverlay=pi3-miniuart-bt
    • core_freq=250
  • sudo reboot
  • remove the display and the keyboard and from now on just connect through Wi-Fi

To test that both Bluetooth and the BrickPi work properly I used a python script to read the NXT ultrasonic sensor (in the first input port) and change the color of my WeDo 2.0 Smart Hub from green to red:

#!/usr/bin/python

# run with sudo
# assumes NXT Ultrasonic at INPUT #1

from ev3dev.auto import *
from gattlib import GATTRequester
from time import sleep

BTdevice = "hci0"       # BlueTooth 4.0 BLE capable device

WeDo2HubAddress  = "A0:E6:F8:1E:58:57"

InputCommand_hnd = 0x3a
OutputCommand_hnd  = 0x3d

RGBAbsoluteMode_cmd = str(bytearray([01,02,06,17,01,01,00,00,00,02,01]))
RGBAbsoluteOutput_cmd = str(bytearray([06,04,03]))  # or "\x06\x04\x03"

DELAY      = 0.3

# DO NOT FORGET TO CONFIG FOR US sensor:
# sudo echo nxt-i2c > /sys/class/lego-port/port0/mode
# sudo echo "lego-nxt-us 0x01" > /sys/class/lego-port/port0/set_device
#
us = UltrasonicSensor('ttyAMA0:S1:i2c1')
assert us.connected

req = GATTRequester(WeDo2HubAddress,True,BTdevice)
sleep(DELAY)

# configure RBG LED to Absolute Mode (accepts 3 bytes for RGB instead of default Index Mode)
req.write_by_handle(InputCommand_hnd,RGBAbsoluteMode_cmd)

while(True):
  if (us.value() < 10):
    print("!")
    req.write_by_handle(OutputCommand_hnd, RGBAbsoluteOutput_cmd+chr(255)+chr(0)+chr(0))
    sleep(DELAY)
  else:
    print("-")
    req.write_by_handle(OutputCommand_hnd, RGBAbsoluteOutput_cmd+chr(0)+chr(255)+chr(0))
    sleep(DELAY)

My script need the gattlib library to talk with Bluetooth Low Energy devices. You can install this library with ‘pip’ but first need to install some dependencies:

sudo apt-get install pkg-config libboost-python-dev libboost-thread-dev libbluetooth-dev libglib2.0-dev python-dev

then

sudo pip install gattlib

WeDo 2.0 – reverse engineering

This post is part 1 of 6 of  WeDo 2.0 - reverse engineering

This post tries to gather all the information I collected in the last weeks related to the WeDo 2.0.

I’m not a programmer but I’m a very stubborn guy so after I managed to get my linux systems (my Ubuntu laptop, some Raspberry Pi’s and my two Mindstorms EV3) controlling the SBrick I told to myself: I’m gonna make the same with this new WeDo 2.0 system no matter what it takes.

So first things first: let’s use my Android mobile to inspect the WeDo 2.0 Hub. Nordic has a very good (and free) app that I like to use: nRF Master Control Panel (recently renamed to nRF Connect). After connecting to the Hub we find 6 services:

BLE services

Some of this services, like “Battery Service” are known BLE services so [hopefully] it will be easy to use.

There’s also a very important finding at the top:

“U0001 A0:E6:F8:1E:58:57

“A0:E6:F8:1E:58:57” is the Bluetooth address of my Hub (similar to the MAC Address of every network device). It will be used *a lot* in the rest of this post.

And “U0001” is the friendly name that the Hub advertises – that’s the name that shows up in the LEGO Education WeDo 2.0 App when connecting to the Hub:

WeDo 2.0 making a connection

A note about this name: before I finally managed to make the LEGO Education WeDo 2.0 App work in my Android phone, my Hub advertised itself as “LPF2 Smart Hub 2 I/O”. So the LEGO App changed it to “u0001”, probably at the first time it connected to it (but since my Hub was first used by 3 other AFOL’s at Paredes de Coura I’m not sure if the process is automatic or the user is given some kind of option).

So the default (factory set) name of the Hub is “LPF2 Smart Hub 2 I/O” – LEGO Power Functions 2 Smart Hub 2 I/O”. Not much to speculate here: LEGO announced that Power Functions and Mindstorms will adopt a new plug type so new devices are expected, this is just the first one. But “Smart Hub 2 I/O” is interesting… does that means that there will be other Smart Hubs? Perhaps even a “Smart Hub 4 I/O”? That would answer some of the points I have been discussing with Fernando Conchas like “what’s the use for a 4.5 Volt device in LEGO Technic unless there’s also another device with better power features just waiting to come out”?

Now let’s look deeper to those BLE services…

I can use the nRF App and take a lot of screenshots but now that I know the BT address I will switch to my Ubuntu laptop and use 2 of the available BlueZ (the native Linux Bluetooth stack) functions, ‘hcitool’ and ‘gatttool’

$ sudo hcitool -i hci0 lescan
LE Scan ...
A0:E6:F8:1E:58:57 (unknown)
A0:E6:F8:1E:58:57 LPF2 Smart Hub 2 I/O
sudo gatttool -i hci0 -b A0:E6:F8:1E:58:57 --primary
attr handle = 0x0001, end grp handle = 0x0007 uuid: 00001800-0000-1000-8000-00805f9b34fb
attr handle = 0x0008, end grp handle = 0x000b uuid: 00001801-0000-1000-8000-00805f9b34fb
attr handle = 0x000c, end grp handle = 0x002f uuid: 00001523-1212-efde-1523-785feabcd123
attr handle = 0x0030, end grp handle = 0x003e uuid: 00004f0e-1212-efde-1523-785feabcd123
attr handle = 0x003f, end grp handle = 0x0045 uuid: 0000180a-0000-1000-8000-00805f9b34fb
attr handle = 0x0046, end grp handle = 0xffff uuid: 0000180f-0000-1000-8000-00805f9b34fb

The ‘gatttool’ command is a powerfull tool for BLE – in the past, without a proper BLE library for python, I used it (through python system calls) to talk with the SBrick. Clumsy but… hey, I said I’m not a programmer 😉

The ‘gatttool’ can run in an interactive mode that allows us to establish a connection and keep it until we disconnect instead of making a new connection each time we want to test a command:

$ sudo gatttool -i hci0 -I
[                 ][LE]> connect A0:E6:F8:1E:58:57
Attempting to connect to A0:E6:F8:1E:58:57
Connection successful
[A0:E6:F8:1E:58:57][LE]>

In this “interactive” session we just send ‘primary’ to get the same output as using the command with ‘–primary’ option [but sometimes the commands differ a bit, so use ‘help’ and ‘–help’ to know what to use.

So the ‘primary’ command gets a list of the primary services offered by the WeDo 2.0 Hub. Of course, those are the same 6 services found by the Nordic app but that screenshot looks much better as Nordic developers added lots of intelligence to it.

First service:

attr handle = 0x0001, end grp handle = 0x0007 uuid: 00001800-0000-1000-8000-00805f9b34fb

The Nordic app shows just:

Generic Access
UUID: 0x1800
PRIMARY SERVICE

So this service has 7 handles assigned (from 0x0001 to 0x0007) and serves a well know service (the ‘Generic Access‘) so it’s UUID is shortened to just 0x1800 instead of ‘00001800-0000-1000-8000-00805f9b34fb’.

Bluetooth specification for ‘Generic Access’ defines 5 properties:

  • Device Name (0x2A00)
  • Appearance (0x2A01)
  • Peripheral Privacy Flag (0x2A02)
  • Reconnection Address (0x2A03)
  • Peripheral Preferred Connection Parameters (0x2A04)

From these list, only ‘Device Name’ and ‘Appearance’ are defined as ‘Mandatory’.

With the gatttool we read these properties with command ‘char-read-uuid’:

[A0:E6:F8:1E:58:57][LE]> char-read-uuid 0x2A00
handle: 0x0003      value: 75 30 30 30 31 
[A0:E6:F8:1E:58:57][LE]> char-read-uuid 0x2A01
handle: 0x0005      value: 00 00 
[A0:E6:F8:1E:58:57][LE]> char-read-uuid 0x2A02
Error: Read characteristics by UUID failed: No attribute found within the given range
[A0:E6:F8:1E:58:57][LE]> char-read-uuid 0x2A03
Error: Read characteristics by UUID failed: No attribute found within the given range
[A0:E6:F8:1E:58:57][LE]> char-read-uuid 0x2A04
handle: 0x0007      value: 50 00 a0 00 00 00 e8 03

So ‘Peripheral Privacy Flag’ and ‘Reconnection Address’ were not implemented by LEGO. But what’s the meaning of this hexadecimal values?

Device Name‘ is a string so we just convert it to ASCII (we can use an online tool like RapidTables):

75 30 30 30 31 -> u0001

Ah-ha!

Appearance‘ is two-byte value “composed of a category (10-bits) and sub-categories (6-bits)” that classifies the device. Since it is ‘0’, it’s not classified in any known category (so it is ‘Unknown’).

Peripheral Preferred Connection Parameters‘ is an 8-byte value containing 4 parameters (each one is 2-byte):

Minimum Connection Interval
Maximum Connection Interval
Slave Latency
Connection Supervision Timeout Multiplier

Each value is written in reverse order so

Minimum Connection Interval =005h = 80d
Maximum Connection Interval = 000ah = 160
Slave Latency = 0
Connection Supervision Timeout Multiplier = 03e8h=1000d

According to definition, the Min/Max Connection Interval values should be multiplied by 1.25 so the range is in fact 100~200 [ms].

So let’s return to the Android and validate these:

WeDo 2.0 'Generic Access' primary service

Yes, it certainly looks good!

Next post we’ll see another well-know service: the ‘Battery Service’.