ConnectCore Voice | ConnectCore 8M Nano

Digi ConnectCore Voice brings full voice processing at the IoT edge to any device integrating a ConnectCore module. It integrates the vicCONTROL Industrial application from voice INTER connect with the Digi ConnectCore platform, providing speech recognition and voice control capabilities for any ConnectCore-enabled product.

ConnectCore Voice provides a customizable vocabulary, customizable wake word, and text-to-speech output, enabling zero-touch user interaction with the device. It does not require hardware-based AI/ML accelerators to operate, enabling you to add voice-processing capabilities without additional hardware costs.

The speech-recognition technology of ConnectCore Voice operates locally on the device and does not require an Internet connection. This makes it particularly suitable for applications that must function without Internet access, applications that require fast response time to voice commands, or have special data protection requirements. It recognizes natural speech, comparable to cloud-based speech recognition services such as Amazon’s Alexa or Google Speech-to-Text Services. Additionally, it is available in 30 national languages and enables semantic understanding of what is being said using machine-learning and artificial intelligence technologies.

Architecture

ConnectCore Voice consists of two main parts:

The vicSDC webtool to design and compile the speech application
An offline dialog manager, which runs on the ConnectCore platform

Get started with the Digi ConnectCore Voice demo

Follow these steps to get started using the Digi ConnectCore Voice demo with your ConnectCore 8M Nano Development Kit.

Requirements

ConnectCore 8M Nano Development Kit

USB headset, or microphone and headset connected to the audio jack connector

The Sennheiser PC 8 USB is the recommended headset for getting the most out of the ConnectCore 8M Nano speech recognition demo, but you can also use a microphone and a headset connected to the audio jack connector.

HDMI display for the graphical user interface in the demos
A mouse to switch between demos in the graphical user interface

Step 1: Set up the hardware

Follow these steps to set up your ConnectCore 8M Nano Development Kit hardware:

Connect the microAB USB cable to the USB CONSOLE connector on the board and to your host computer. The operating system will detect the board as two new serial ports. One serial port is the Cortex-A53 console (system default console), and the other one is the Cortex-M7 console. Identify the new serial port for the Cortex-A53 console on your operating system (/dev/ttyUSBx in Linux, COMx in Windows), as you will need to open a console at that port in the following step.

If the operating system does not recognize your board you must manually install the drivers, which are available on the Prolific website.
Connect an HDMI cable (not included) to the HDMI connector and then to an HDMI-compatible display.
Plug the USB headset into the USB connector (or the microphone and headset into the audio jack connector).

If you are going to use a USB headset, make sure it is plugged before powering on the device. Otherwise, the demo will route the audio by default to the audio jacks on the development board.
Make sure the S1 micro-switches are all in the OFF position.
Connect the power supply to the power barrel connector.

Step 2: Program the speech recognition image

The prebuilt ConnectCore Voice image includes a demo that simulates a washing machine that you control with your voice. You can use voice commands for the washer to follow using voice control and natural language recognition, such as “extra rinse” or “wash delicates quickly in cold water”. You can also request information from the washer, such as “Hey washer, how long is the rinse cycle?” to test the text-to-speech feature. Digi recommends you program the firmware images directly in your device’s eMMC, but you can also boot from the uSD card.

Make sure the USB headset is connected before you proceed with programming the firmware. If the headset is disconnected when the board is powered on, you must reboot the device before programming the firmware.

Establish a serial connection with your device

Open a serial connection with the device. You can use any serial terminal program such as Tera Term, Minicom, CoolTerm, or HyperTerminal.

PuTTY terminal program is not recommended due to an incompatibility with the USB/serial driver.

Open a serial connection with the following settings:

Parameter	Value
Port	Serial port where the device is connected
Baud rate	115200
Data bits	8
Parity	None
Stop bits	1
Flow control	None

Parameter

Value

Port

Serial port where the device is connected

Baud rate

115200

Data bits

Parity

None

Stop bits

Flow control

None

Download the firmware

To access firmware images, please fill out the following form to provide some information so we can better understand your project needs.

If the form doesn’t load after a few seconds, you may need to refresh the page. Contact Digi Support Services if you are not able to access the firmware images.

Program the firmware

Digi recommends you Use the Universal Update Utility (UUU) through a USB connection to your development PC to program the images.

Decompress the zip file in a directory of your choice.
Download the latest Universal Update Utility (UUU), a software tool from NXP.

UUU version should be v1.5.11 or higher.
Place the uuu executable in a directory in your PATH, such as /usr/bin.

Install udev rules to allow any user to run uuu to access the USB port:

$ sudo sh -c "uuu -udev >> /etc/udev/rules.d/70-uuu.rules"
$ sudo udevadm control --reload

Connect a micro USB cable to your development PC and the other end to the target USB recovery connector.

Reset the device by pressing the reset button on the board and immediately press any key in the serial terminal to stop the auto-boot process. The U-Boot bootloader prompt displays:

U-Boot SPL dub-2020.04-r6.1 (Sep 19 2023 - 07:18:57 +0000)
DDRINFO: start DRAM init
DDRINFO: DRAM rate 3000MTS
DDRINFO:ddrphy calibration done
DDRINFO: ddrmix config done
Normal Boot
Trying to boot from BOOTROM
image offset 0x0, pagesize 0x200, ivt offset 0x0
NOTICE:  BL31: v2.4(release):lf-5.10.72-2.2.3-2-gb91c6a890
NOTICE:  BL31: Built : 07:19:22, Sep 19 2023


U-Boot dub-2020.04-r6.1 (Sep 19 2023 - 07:18:57 +0000)

CPU:   i.MX8MNano Quad rev1.0 1400 MHz (running at 1200 MHz)
CPU:   Industrial temperature grade (-40C to 105C) at 36C
Reset cause: POR
DRAM:  1 GiB
MCA:   HW_VER=1  FW_VER=1.01
MMC:   FSL_SDHC: 1, FSL_SDHC: 0
In:    serial
Out:   serial
Err:   serial
Model: Digi ConnectCore 8M Nano Development Kit
ConnectCore 8M Nano SOM variant 0x01: 1 GiB LPDDR4, Wi-Fi, Bluetooth, MCA, Crypto-auth
  Board version 3, ID undefined
Boot:  MMC2

 BuildInfo:
  - ATF b91c6a8
  - U-Boot dub-2020.04-r6.1

flash target is MMC:0
Net:   eth0: ethernet@30be0000
Fastboot: Normal
Normal Boot
Hit any key to stop autoboot:  0
=>

Information in the console log may vary.

On the target, in U-Boot, execute the following command:
```
=> fastboot 0
```
On your development PC, change to the directory where you extracted the ZIP file:
```
$ cd <zip-folder>
```
To program the ConnectCore Voice demo image, run:
```
$ ./install_linux_fw_uuu.sh -i dey-image-voice-recognition
```
Once the firmware is installed, the device boots automatically. The default username is root with no password.
Remove the micro USB cable from the recovery connector of the ConnectCore 8M Nano Development Kit.
Reboot the device. This step is necessary if you use a USB headset because the USB port will not connect if a cable is connected to the recovery USB port of the ConnectCore 8M Nano Development Kit. See USB interfaces on the ConnectCore 8M Nano.

While the recommended boot method is eMMC, you can also boot the speech recognition kit images from a microSD card using the *.sdcard image file on the downloaded ZIP file. See Boot from microSD card for instructions on programming a microSD card and booting from it.

You must use a microSD with a minimum capacity of 8 GB.

Step 3: Test applications

Speech recognition sample application

The speech recognition sample application allows you to use voice commands for the washer to follow using voice control and natural language recognition. You can also test the text-to-speech feature by requesting information from the washer. The demo starts automatically once the target boots, but you can also launch it manually with the following command:

# vicctl start demo-washer

The demo kit is programmed with a wake-up phrase of “Hey washer”. After it detects the wake-up phrase, the application waits 10 seconds for a verbal command.

The available actions are:

Change the fabric cycle
Change the temperature
Change the spin speed
Set a timer

ConnectCore Voice washing machine demo screenshot

To test the application, follow these steps:

To communicate with the demo, say the wake-up phrase "Hey washer" into the microphone. If the application recognizes the wake-up phrase, the device plays a sound and the characters at the top of the application turn blue.

Speak an action to perform, for example: “extra rinse” or “wash delicates quickly in cold water”.

Don’t worry about how you phrase the sentence. The demo program uses natural language understanding and will try to figure out what you said.

Available actions are:

Change the fabric cycle (cotton, delicate, wool…)
Change the temperature (cold, warm, hot, 30 degrees…)
Change the spin speed
Set a timer

The application changes the selections to match your spoken command.

Try speaking different actions to see how the indicators on the washer change.

Text-to-speech sample application

The demo also has text-to-speech (TTS) capabilities that can reply with audible responses to your spoken questions. To test TTS, you need to have a headset or speakers connected to hear the audio. This turns application menu interfaces into lifelike speech responses.

For example, if you say: "Hey washer, how long does the program last?" it verbally returns the time the current program takes.

Speech synthesis sample application

The speech synthesis demo allows you to select a phrase from a list and have the software reproduce it in a selected language.

Select Speech Synthesis from the main menu of the vicCONTROL Industrial application.
Select a language and a quote to say. For example, you can select the English USA language and the quote “The flow rate is 60 liters per minute”.
Click Say to reproduce the phrase in the speaker.
Try different combinations of languages and quotes.

Request full ConnectCore Voice license key

To continue exploring the ConnectCore Voice solution and get access to additional demos or build your own custom voice-enabled application, complete the following form to request a full ConnectCore Voice development license. Please make sure you enter a valid email address and phone number. Digi will contact you to provide the license key.