{"id":1041,"date":"2017-02-19T23:57:36","date_gmt":"2017-02-19T23:57:36","guid":{"rendered":"http:\/\/ofalcao.pt\/blog\/?p=1041"},"modified":"2017-02-20T01:03:59","modified_gmt":"2017-02-20T01:03:59","slug":"lego-voice-control-ev3","status":"publish","type":"post","link":"https:\/\/ofalcao.pt\/blog\/2017\/lego-voice-control-ev3","title":{"rendered":"LEGO Voice Control &#8211; EV3"},"content":{"rendered":"<div class=\"seriesmeta\">This post is part 2 of 2 of \u00a0<a href=\"https:\/\/ofalcao.pt\/blog\/series\/lego-voice-control\" class=\"series-288\" title=\"LEGO Voice Control\">LEGO Voice Control<\/a><\/div><p>And now the big test &#8211; will it work with EV3?<\/p>\n<p>So, ev3dev updated:<\/p>\n<pre>Linux ev3dev 4.4.47-19-ev3dev-ev3 #1 PREEMPT Wed Feb 8 14:15:28 CST 2017 armv5tejl GNU\/Linux<\/pre>\n<p>I can&#8217;t find any microphone at the moment so I&#8217;ll use the mic of my Logitech C270 webcam &#8211; ev3dev sees it as an UVC device as you can see with dmesg:<\/p>\n<pre>...\r\n[ 1343.702215] usb 1-1.2: new full-speed USB device number 7 using ohci\r\n[ 1343.949201] usb 1-1.2: New USB device found, idVendor=046d, idProduct=0825\r\n[ 1343.949288] usb 1-1.2: New USB device strings: Mfr=0, Product=0, SerialNumber=2\r\n[ 1343.949342] usb 1-1.2: SerialNumber: F1E48D60\r\n[ 1344.106161] usb 1-1.2: set resolution quirk: cval-&gt;res = 384\r\n[ 1344.500684] Linux video capture interface: v2.00\r\n[ 1344.720788] uvcvideo: Found UVC 1.00 device &lt;unnamed&gt; (046d:0825)\r\n[ 1344.749629] input: UVC Camera (046d:0825) as \/devices\/platform\/ohci.0\/usb1\/1-1\/1-1.2\/1-1.2:1.0\/input\/input3\r\n[ 1344.772321] usbcore: registered new interface driver uvcvideo\r\n[ 1344.772372] USB Video Class driver (1.1.1)\r\n[ 1352.171498] usb 1-1.2: reset full-speed USB device number 7 using ohci\r\n...<\/pre>\n<p>and we can check with &#8220;alsamixer&#8221; that ALSA works fine with the internal microphone:<\/p>\n<p>First press F6 to select sound card (the webcam is a sound card for ALSA)<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/ofalcao.pt\/blog\/wp-content\/uploads\/2017\/02\/alsamixer-01.png\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-1042\" src=\"https:\/\/i0.wp.com\/ofalcao.pt\/blog\/wp-content\/uploads\/2017\/02\/alsamixer-01.png?resize=494%2C291\" alt=\"\" width=\"494\" height=\"291\" \/><\/a>Then press F5 to view all sound devices &#8211; there is just one, the mic:<\/p>\n<p><a href=\"https:\/\/i0.wp.com\/ofalcao.pt\/blog\/wp-content\/uploads\/2017\/02\/alsamixer-02.png\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-1043\" src=\"https:\/\/i0.wp.com\/ofalcao.pt\/blog\/wp-content\/uploads\/2017\/02\/alsamixer-02.png?resize=494%2C291\" alt=\"\" width=\"494\" height=\"291\" \/><\/a><\/p>\n<p>We also need to know how ALSA addresses the mic:<\/p>\n<pre>arecord -l\r\n**** List of CAPTURE Hardware Devices ****\r\ncard 1: U0x46d0x825 [USB Device 0x46d:0x825], device 0: USB Audio [USB Audio]\r\n\u00a0 Subdevices: 1\/1\r\n\u00a0 Subdevice #0: subdevice #0<\/pre>\n<p>Card 1, Device 0 means we should use &#8216;hw:1,0&#8217;<\/p>\n<p>Now we just follow the same process we used with Ubuntu. First we install pocketsphinx:<\/p>\n<pre>sudo apt install pocketsphinx\r\n...\r\nThe following extra packages will be installed:\r\n\u00a0 javascript-common libblas-common libblas3 libjs-jquery liblapack3 libpocketsphinx1 libsphinxbase1\r\n\u00a0 pocketsphinx-hmm-en-hub4wsj pocketsphinx-lm-en-hub4\r\nSuggested packages:\r\n\u00a0 apache2 lighttpd httpd\r\nThe following NEW packages will be installed:\r\n\u00a0 javascript-common libblas-common libblas3 libjs-jquery liblapack3 libpocketsphinx1 libsphinxbase1\r\n\u00a0 pocketsphinx pocketsphinx-hmm-en-hub4wsj pocketsphinx-lm-en-hub4\r\n0 upgraded, 10 newly installed, 0 to remove and 0 not upgraded.\r\nNeed to get 8910 kB of archives.\r\nAfter this operation, 30.0 MB of additional disk space will be used.\r\n..<\/pre>\n<p>Although Ubuntu and Debian packages seem to be the same, the maintaners made some differente choices because in Ubuntu the &#8216;pocketsphinx-hmm-en-hub4wsj&#8217; and &#8216;pocketsphinx-lm-en-hub4&#8217; packages are missing.<\/p>\n<p>So we copy 3 files from our previous work in Ubuntu:<\/p>\n<ul>\n<li>keyphrase_list.txt<\/li>\n<li>0773.lm<\/li>\n<li>0772.dic<\/li>\n<\/ul>\n<p>And we test it:<\/p>\n<pre>pocketsphinx_continuous -kws keyphrase_list.txt -adcdev hw:1,0 -lm 0772.lm -dict 0772.dic -inmic yes -logfn \/dev\/null<\/pre>\n<p>We get a &#8220;Warning: Could not find Capture element&#8221; but&#8230; yes, it works!<\/p>\n<p>Of course it is slow&#8230; we see a big delay while starting until it displays &#8220;READY&#8230;.&#8221; and also a big delay between each &#8220;Listening&#8230;&#8221; cycle. But it works! Isn&#8217;t open source great?<\/p>\n<p>So we install expect to use our pipe again:<\/p>\n<pre>sudo apt install expect\r\nmkfifo pipe<\/pre>\n<p>and we rewrite our &#8216;transmit.sh&#8217; to command two EV3 motors (let&#8217;s call it &#8220;controller.sh&#8221; this time):<\/p>\n<pre>#!\/bin\/bash\r\n\r\nwhile read -a words\r\ndo\r\ncase \"${words[1]}\" in\r\n\r\n\u00a0 move)\r\n\u00a0\u00a0\u00a0 if [ \"${words[2]}\" = \"forward\" ]; then\r\n\u00a0\u00a0\u00a0\u00a0\u00a0 echo \"FRONT\"\r\n\u00a0\u00a0\u00a0\u00a0\u00a0 echo run-timed &gt; \/sys\/class\/tacho-motor\/motor0\/command\r\n\u00a0\u00a0\u00a0\u00a0\u00a0 echo run-timed &gt; \/sys\/class\/tacho-motor\/motor1\/command\r\n\u00a0\u00a0\u00a0\u00a0\u00a0 sleep 0.2\r\n\u00a0\u00a0\u00a0 fi\r\n\r\n\u00a0\u00a0\u00a0 if [ \"${words[2]}\" = \"backward\" ]; then\r\n\u00a0\u00a0\u00a0\u00a0\u00a0 echo \"BACK\"\r\n\u00a0\u00a0\u00a0\u00a0\u00a0 sleep 0.2\r\n\u00a0\u00a0\u00a0 fi\r\n\u00a0\u00a0\u00a0 ;;\r\n\r\n\u00a0 turn)\r\n\u00a0\u00a0\u00a0 if [ \"${words[2]}\" = \"left\" ]; then\r\n\u00a0\u00a0\u00a0\u00a0\u00a0 echo \"LEFT\"\r\n\u00a0\u00a0\u00a0\u00a0\u00a0 echo run-timed &gt; \/sys\/class\/tacho-motor\/motor1\/command\r\n\u00a0\u00a0\u00a0\u00a0\u00a0 sleep 0.2\r\n\u00a0\u00a0\u00a0 fi\r\n\r\n\u00a0\u00a0\u00a0 if [ \"${words[2]}\" = \"right\" ]; then\r\n\u00a0\u00a0\u00a0\u00a0\u00a0 echo \"RIGHT\"\r\n\u00a0\u00a0\u00a0\u00a0\u00a0 echo run-timed &gt; \/sys\/class\/tacho-motor\/motor0\/command\r\n\u00a0\u00a0\u00a0\u00a0\u00a0 sleep 0.2\r\n\u00a0\u00a0\u00a0 fi\u00a0\u00a0 \u00a0\r\n\u00a0\u00a0\u00a0 ;;\r\n\r\n\u00a0 stop)\r\n\u00a0\u00a0\u00a0 echo \"STOP\"\r\n\u00a0\u00a0\u00a0 ;;\r\n\r\n\u00a0 *)\r\n\u00a0\u00a0\u00a0 echo \"?\"\r\n\u00a0\u00a0\u00a0 echo \"${words[1]}\"\r\n\u00a0\u00a0\u00a0 echo \"${words[2]}\"\r\n\u00a0\u00a0\u00a0 ;;\r\nesac\r\ndone<\/pre>\n<p>For some reason I don&#8217;t yet understand I had to change 2 things that worked fine with Ubuntu:<\/p>\n<ul>\n<li>increase the index of the arguments (&#8220;${words[1]&#8221; and &#8220;${words[2]&#8221; instead of &#8220;${words[0]&#8221; and &#8220;${words[1]&#8221;<\/li>\n<li>use capital letters for the keywords<\/li>\n<\/ul>\n<p>This script sends &#8220;run-timed&#8221; commands to the motor file descriptors (you can read a good explanation on this ev3dev tutorial: <a href=\"http:\/\/www.ev3dev.org\/docs\/tutorials\/tacho-motors\/\">&#8216;Using the Tacho-Motor Class&#8217;<\/a>). I didn&#8217;t write commands for &#8220;move backward&#8221; this time (it would require extra lines to change direction, not difficult but I don&#8217;t want to increase the script to much).<\/p>\n<p>Before we can use this script, we need to initialize the motors so we can use this other script, &#8220;init.sh&#8221;<\/p>\n<pre>#!\/bin\/bash\r\n\r\necho 1050 &gt; \/sys\/class\/tacho-motor\/motor0\/speed_sp\r\necho 200 &gt; \/sys\/class\/tacho-motor\/motor0\/time_sp\r\necho 1050 &gt; \/sys\/class\/tacho-motor\/motor1\/speed_sp\r\necho 200 &gt; \/sys\/class\/tacho-motor\/motor1\/time_sp<\/pre>\n<p>(it just sets maximum speed to motor0 and motor1 and the timer to 200 ms for the duration of each &#8220;run-timed&#8221; command).<\/p>\n<p>So we open two a second ssh session to our EV3 and we ran in the first session:<\/p>\n<pre>unbuffer pocketsphinx_continuous -kws keyphrase_list.txt -adcdev hw:1,0 -lm 0772.lm -dict 0772.dic -inmic yes -logfn \/dev\/null &gt; pipe<\/pre>\n<p>and in the second session:<\/p>\n<pre>cat pipe | .\/controller.sh<\/pre>\n<p>And presto!<\/p>\n<div class=\"jetpack-video-wrapper\"><iframe loading=\"lazy\" title=\"LEGO Mindstorms EV3 with voice control\" width=\"840\" height=\"473\" src=\"https:\/\/www.youtube.com\/embed\/_c-nk69x3H8?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/div>\n<p>The robot is a <a href=\"http:\/\/www.damienkee.com\/home\/2013\/8\/2\/rileyrover-ev3-classroom-robot-design.html\">RileyRover<\/a>, a &#8220;very quick to build&#8221; design from Damien Kee.<\/p>\n","protected":false},"excerpt":{"rendered":"<div class=\"seriesmeta\">This post is part 2  of 2 of \u00a0<a href=\"https:\/\/ofalcao.pt\/blog\/series\/lego-voice-control\" class=\"series-288\" title=\"LEGO Voice Control\">LEGO Voice Control<\/a><\/div><p>And now the big test &#8211; will it work with EV3? So, ev3dev updated: Linux ev3dev 4.4.47-19-ev3dev-ev3 #1 PREEMPT Wed Feb 8 14:15:28 CST 2017 armv5tejl GNU\/Linux I can&#8217;t find any microphone at the moment so I&#8217;ll use the mic of my Logitech C270 webcam &#8211; ev3dev sees it as an UVC device as you &hellip; <a href=\"https:\/\/ofalcao.pt\/blog\/2017\/lego-voice-control-ev3\" class=\"more-link\">Continuar a ler<span class=\"screen-reader-text\"> &#8220;LEGO Voice Control &#8211; EV3&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false},"version":2}},"categories":[1],"tags":[290,289],"series":[288],"class_list":["post-1041","post","type-post","status-publish","format-standard","hentry","category-sem-categoria","tag-speech-recognition","tag-voice-control","series-lego-voice-control"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p2Mhyv-gN","_links":{"self":[{"href":"https:\/\/ofalcao.pt\/blog\/wp-json\/wp\/v2\/posts\/1041","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ofalcao.pt\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ofalcao.pt\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ofalcao.pt\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ofalcao.pt\/blog\/wp-json\/wp\/v2\/comments?post=1041"}],"version-history":[{"count":0,"href":"https:\/\/ofalcao.pt\/blog\/wp-json\/wp\/v2\/posts\/1041\/revisions"}],"wp:attachment":[{"href":"https:\/\/ofalcao.pt\/blog\/wp-json\/wp\/v2\/media?parent=1041"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ofalcao.pt\/blog\/wp-json\/wp\/v2\/categories?post=1041"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ofalcao.pt\/blog\/wp-json\/wp\/v2\/tags?post=1041"},{"taxonomy":"series","embeddable":true,"href":"https:\/\/ofalcao.pt\/blog\/wp-json\/wp\/v2\/series?post=1041"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}