28. ROS2 AI Voice Interaction Courses

28.1 Introduction to WonderEcho Pro and Installation

28.1.1 Introduction to WonderEcho Pro

WonderEcho Pro, also known as the AI Voice Interaction Box, is equipped with a high-performance noise-canceling microphone and a high-fidelity speaker. Featuring a USB-to-audio module, it offers plug-and-play functionality without the need for drivers and is compatible with multiple systems for both playback and recording.

This device integrates various speech processing modules and uses advanced noise suppression algorithms to effectively filter out background noise, supporting a seamless process from wake-up to speech recognition and interaction. With its modular design, each functional component—such as wake-up, detection, recognition, and synthesis—can be developed and tested independently.

  • Features & Parameters

(1) The device is equipped with an onboard microphone and speaker interface, supporting both audio input and output.

(2) It is driver-free, offering seamless plug-and-play functionality with broad system compatibility, including Windows, macOS, Linux, and Android. The listen/speak dual function ensures versatility across various platforms.

(3) It uses a standard USB 2.0 interface.

(4) Control Interface: USB

(5) Voice Chip Model: Cl1302

(6) Speaker Output: 3.0W per channel (4Ω BTL)

(7) Power Supply Voltage: 5V

  • Recording and Playback Test

Note

The following steps demonstrate the process using the Raspberry Pi 5, but the connection and testing procedures are equally applicable to other compatible devices, such as NVIDIA Jetson controllers.

28.1.2 Connection Setup and Verification

For Raspberry Pi, you can establish a remote connection to the system using VNC by following the 3 Remote Desktop Tool Installation and Connection / 3.1 Remote Tool Installation and Connection guide. Once connected, check the top-right corner of the desktop for the microphone and speaker icons, as shown below. If they are present, the connection has been successfully established.

For Jetson-based systems, you can use the Nomachine remote connection tool. Once connected, verify the presence of the speaker icon in the top-right corner of the system.

28.1.3 Recording Test

(1) Next, open a new command-line terminal and enter the following command to check for the use of the -l (lowercase L) option, and verify that the card number (card) is 0. This is just an example; please ensure to check your actual query results.

arecord -l

(2) Then, use the following command to start recording, replacing the highlighted card number (in red) with the one obtained from the previous query:

arecord -D hw:0,0 -f S16_LE -r 16000 -c 2 test.wav

(3) A test.wav file will be generated in the current folder.

(4) You can record for about 5 seconds, then press Ctrl+C to stop the recording.

28.1.4 Playback Test

(1) After completing the recording, you can check if the file was successfully recorded by running the following command in the current directory:

ls

(2) If test.wav appears, the recording was successful. To play the recorded file, use the command below, ensuring you replace the card number with the one obtained from your previous query:

aplay test.wav

28.1.5 WonderEcho Pro Installation

Connect the WonderEcho Pro to the port 2.

28.1.6 Firmware Flashing Method

In this lesson, we will learn how to flash firmware onto the WonderEcho Pro.

  • Flashing the Firmware

(1) Begin by connecting the WonderEcho Pro to your computer using a Type-C data cable.

(2) Navigate to Firmware Flashing Tool and open the PACK_UPDATE_TOOL.exe file. Select the CI1302 chip from the list, then click Firmware Upgrade to begin the process.

Note

For this example, we will use the hello hiwonder firmware (Wake word: hello hiwonder).

(3) Select the firmware by locating the hello hiwonder firmware (Wake word: hello hiwonder).bin file under the Appendix path.

(4) Identify the correct COM port and select it.

(5) Press the RST button on the WonderEcho Pro to initiate the flashing process. Wait for the process to complete successfully.

  • Testing the Wake Word

After flashing the firmware, follow the steps in Serial Port Debugging Tool to install the serial port debugging tool. Then, follow the steps below to test whether the firmware is functioning correctly.

(1) Connect the WonderEcho Pro to your computer’s USB port using a Type-C to USB adapter.

(2) Open the Serial Port Utility, select the connected COM port (including CH340), and set the baud rate to 115200, as shown in the image below.

(3) Say the wake word hello hiwonder as configured in the firmware settings. The corresponding flag information will appear on the right side (in hexadecimal format), confirming that the wake word was successfully recognized.

28.1.7 Firmware Development

In this lesson, you’ll learn how to create and flash firmware for the WonderEcho Pro module.

  • Firmware Creation and Flashing

Note

We’ll use HIWONDER as an example to demonstrate the firmware creation process. The same procedure also applies to creating firmware with Chinese wake words. Note: English wake words must be entered in all uppercase letters to function properly.

(1) Open the official firmware creation website: https://aiplatform.chipintelli.com/home/index.html; The platform interface is in Chinese. If needed, use your browser’s translation feature to switch to English for easier navigation.

(2) First, visit the ChipIntelli Voice AI Platform by clicking on Platform Features in the top menu, and then select In-depth development of Product Firmware & SDK under the Product Development section.

(3) You will be prompted to log in. (Note: Currently, registration is only available for mainland China mobile numbers. For international users, we provide a shared account: Username: 18033085495 Password: hiwonder

After successfully logging in, click In-depth development of Product Firmware & SDK again to access the development section.

(4) To create a new product, click the option in the left sidebar. You will be asked to provide a Product Name and a Description, both of which can be customized. For the other fields, follow the options shown in the red box. Specifically, choose Product Type: 通用 (General) → 智能中控 (Smart Central Control). After completing these steps, click Create.

(5) Next, you will need to set the project language. Since we are configuring for English voice recognition, select English under Language Type. If you were configuring for Chinese recognition, you would select Chinese for both Voice Type and Acoustic Model. Once you’ve selected the appropriate options, click Continue to proceed.

(6) In the firmware configuration interface, you will make some important changes to the settings. Start by enabling the Echo Cancellation feature under Algorithm Settings.

(7) Under Hardware Settings, set the oscillator source to Internal RC and disable the Baud Rate Calibration function.

(8) In the UART Print Settings, configure UART1 to Open-Drain Mode, supporting external 5V pull-up.

(9) For Communication UART Settings, set the Baud Rate to 115200 and configure UART0 to Open-Drain Mode, also supporting external 5V pull-up. After adjusting these settings, click Continue to move forward.

(10) The next step involves configuring the command words. You will need to choose a voice output for the wake-up command. For this example, select Dane – English Male Voice as the output voice. (If you’re using Chinese as the language, you may choose the voice option Chengcheng – Standard Boy Ver.1. The selection between Chinese and English voices is determined by the language settings configured in Step 5.)

(11) Next, upload the command word configuration file. Navigate to the folder Wake Word Protocol List and locate the spreadsheet titled Command Phrase and Playback Protocol List V3 – English Template. Simply drag and drop the file into the upload area on the webpage.

Note

If you wish to change the wake word, open the spreadsheet and replace the default wake word HIWONDER with your preferred word. Keep in mind that the English version of the firmware only supports English wake words, and mixed-language wake words are not allowed. Additionally, English wake words must be written in uppercase letters to function properly.

(12) After uploading the file, you will see the command data populated in the table below.

(13) Click Submit and wait a few minutes for the firmware generation process to complete. Once finished, click Download to obtain your customized firmware.

28.2 Voice Recognition

28.2.1 Preparation

Refer to 28.1.1 Introduction to WonderEcho Pro to flash the firmware onto the WonderEcho Pro module. Once the firmware has been successfully burned, install the module onto the robot.

By default, the wake-up word for the WonderEcho Pro factory firmware is Hello, HiWonder. If you wish to change the wake-up word to PuppyPi, please refer to the tutorial titled 28.1.6 Firmware Flashing Method for step-by-step instructions.

28.2.2 Brief Game Instructions

The source code for this program is located in the Docker container at: ros2_ws/src/example/example/puppypi_with_sensor/asr_detect_demo.py

The program communicates with the voice recognition sensor via a serial port. It interprets different voice commands and changes the RGB LED color accordingly.

28.2.3 Starting and Stopping the Game

Note

When entering commands, ensure that case sensitivity is strictly followed. You can use the Tab key to auto-complete keywords.

(1) Power on the robot, then follow the steps in 3.3 Docker Container Introduction and Entry and 3.4 ROS Version Switch Tool Guide to connect via the VNC remote control software and switch to the ROS 2 environment.

(2) On the Raspberry Pi desktop, double-click the Terminator icon to open the command-line terminal and enter the ROS2 development environment.

(3) Execute the following command to disable the ROS node.

~/.stop_ros.sh

(4) Enter the following command and press Enter to initiate the program:

ros2 launch example asr_detect_demo.launch.py

(5) To exit this feature, simply press Ctrl+C in the terminal. If the command does not stop the program, please try again.

28.2.4 How It Works

Note

The wake-up commands vary depending on the firmware installed. If you are using the Chinese Xiaohuan Xiaohuan firmware, please use the Chinese commands below. For PuppyPi or Hello HiWonder firmware, use the English commands provided.

User Command Terminal print Actions
puppypi AA 55 03 00 FB
RED-LIGHT AA 55 00 8A FB Red light on
GREEN-LIGHT AA 55 00 8B FB Green light on
BLUE-LIGHT AA 55 00 8C FB Blue light on
TURN-OFF-THE-LIGHT AA 55 00 09 FB Turn off the light

Note

Once the program is running, simply say Puppy to activate voice recognition. After activation, you can control the RGB LED on the Raspberry Pi expansion board by speaking color commands—for example, saying red light will change the LED to red, while green light or blue light will switch it to green or blue, respectively. If there’s no voice input for over 15 seconds, the system will automatically enter sleep mode. To wake it up, just say PuppyPi again.

28.2.5 Program Download

Source Code

28.2.6 Brief Program Analysis

  • Library Files Import

 4
 5
 6
 7
 8
 9
10
11
import os
import sys
import rclpy
import serial
import binascii
import signal
from rclpy.node import Node
from ros_robot_controller_msgs.msg import RGBState, RGBsState

  • RGB Light Settings

Turn off the RGB light

61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
    # 关闭RGB彩灯(turn off color light)    
    def turn_off_rgb(self):
        led1 = RGBState()
        led1.id = 1
        led1.r = 0
        led1.g = 0
        led1.b = 0
        
        led2 = RGBState()
        led2.id = 2
        led2.r = 0
        led2.g = 0
        led2.b = 0
        
        msg = RGBsState()
        msg.data = [led1,led2]
        self.rgb_pub.publish(msg)

Display the RGB light control function

78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
	# 设置RGB彩灯显示(set the RGB color light to display)    
    def set_rgb_show(self,r, g, b):
        led1 = RGBState()
        led1.id = 1
        led1.r = r
        led1.g = g
        led1.b = b       
        led2 = RGBState()
        led2.id = 2
        led2.r = r
        led2.g = g
        led2.b = b
        msg = RGBsState()
        msg.data = [led1,led2]
        self.rgb_pub.publish(msg)

  • Function Execution

The system selects the appropriate color based on the recognition result returned by the WonderEchoPro module.

43
44
45
46
47
48
49
50
51
52
53
54
55
	def parse_serial_data(self, data):
        hex_data = ' '.join(format(byte, '02X') for byte in data)
        self.get_logger().info(f"Received data: {hex_data}")
        if hex_data == "AA 55 00 8A FB": 
            self.set_rgb_show(255, 0, 0)
        elif hex_data == "AA 55 00 8B FB":
            self.set_rgb_show(0, 255, 0)
        elif hex_data == "AA 55 00 8C FB":
            self.set_rgb_show(0, 0, 255)
        elif hex_data == "AA 55 00 09 FB": 
            self.turn_off_rgb()
        elif hex_data == "AA 55 02 00 FB": 
            self.get_logger().info('restart wakeup!!!!!!!!!!!!!!!!!!')

28.2.7 Function Expansion

  • Change the Triggered Color

Note

The program uses red, green, and blue as the default color commands. If you’d like to change these, you can follow the steps below. This section uses replacing the red command with yellow as an example for demonstration.

(1) On the Raspberry Pi desktop, double-click the Terminator icon to open the command-line terminal and enter the ROS2 development environment.

(2) Enter the following command and press Enter to navigate to the directory where the program is located:

cd ~/ros2_ws/src/example/example/puppypi_with_sensor

(3) Locate the code section in the area highlighted in the image.

(4) Press “i” on the keyboard to enter edit mode, then update the values in the set_rgb_show function to 255, 255, 0 to represent the color yellow, as shown in the example below.

(5) Once the modification is complete, press the Esc key. Then, type the following command (note that a colon : precedes the wq) and press Enter to save the changes:

:wq

(6) Afterward, repeat the steps outlined in this document to run the program.

  • Change Recognition Command Word

(1) On the Raspberry Pi desktop, double-click the Terminator icon to open the command-line terminal and enter the ROS2 development environment.

(2) Enter the following command and press Enter to navigate to the directory where the program is located:

cd ~/ros2_ws/src/example/example/puppypi_with_sensor

(3) Run the following command to open the program file.

vim asr_detect_demo.py

(4) Locate the code section in the area highlighted in the image.

(5) Open the Command Word Protocol Playlist. For example, here we’ll show how to change the command from Red Light to Turn on the light.

(6) Execute the following command to access the program file.

vim asr_detect_demo.py

(7) Press “i” to enter edit mode, then update the protocol code to match the Turn on the light command, as illustrated below.

(8) Once the modification is complete, press the Esc key. Then, type the following command (note that a colon : precedes the wq) and press Enter to save the changes:

:wq

(9) Afterward, repeat the steps outlined in this document to run the program.

(10) For creating additional command words, please consult the 28.1.7 Firmware Development documentation for detailed instructions.

28.3 Sing and Dance

28.3.1 Preparation

Refer to 28.1 Introduction to WonderEcho Pro and Installation to flash the firmware onto the WonderEcho Pro module. Once the firmware has been successfully burned, install the module onto the robot. By default, the wake-up word for the WonderEcho Pro factory firmware is Hello, HiWonder. If you wish to change the wake-up word to PuppyPi, please refer to the tutorial titled 28.1.5 WonderEcho Pro Installation for step-by-step instructions.

28.3.2 Brief Game Instructions

The program plays .mp3 audio files using the WonderEcho Pro module’s sound card. Begin by creating a folder called MP3 within the program’s directory.

This program loads MP3 files from a specified directory, allowing users to select and play tracks. It simultaneously controls the robotic dog’s posture, gait, and movements to perform a variety of complex actions, including marching in place, multi-axis coordination, and sliding steps.

The source code is located inside the Docker container at: ros2_ws/src/example/example/puppypi_with_sensor/mp3_moonwalk_demo.py

28.3.3 Starting and Stopping the Game

Note

When entering commands, ensure that case sensitivity is strictly followed. You can use the Tab key to auto-complete keywords.

(1) Power on the robot, then follow the steps in 3.3 Docker Container Introduction and Entry and 3.4 ROS Version Switch Tool Guide to connect via the VNC remote control software and switch to the ROS 2 environment.

(2) On the Raspberry Pi desktop, double-click the Terminator icon to open the command-line terminal and enter the ROS2 development environment.

(3) Execute the following command to enable the underlying control service.

ros2 launch puppy_control puppy_control.launch.py

(4) Enter the following command and press Enter to navigate to the directory where the program is located:

cd ros2_ws/src/example/example/puppypi_with_sensor

(5) Open a terminal, enter the following command and press Enter to initiate the program:

python3 mp3_moonwalk_demo.py

(6) Please enter the number of the track you want to play, for example, 11.

(7) To exit this feature, simply press Ctrl+C in the terminal. If the command does not stop the program, please try again.

28.3.4 How It Works

After running the program, the WonderEcho Pro module will play the specified music and perform a dance routine.

28.3.5 Program Download

Source Code

28.3.6 Brief Program Analysis

  • Library Files Import

 4
 5
 6
 7
 8
 9
10
11
12
13
import math
import rclpy
import time
import signal
import os
import subprocess
from rclpy.node import Node
from speech import speech
from puppy_control_msgs.srv import SetRunActionName
from puppy_control_msgs.msg import Velocity, Pose, Gait

  • Acquire MP3 File

56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
 def get_mp3_files(self):
        """从目录中读取所有 MP3 文件"""
        try:
            files = [f for f in os.listdir(self.MP3_DIR) if f.endswith(".mp3")]
            self.mp3_files = {i + 1: os.path.join(self.MP3_DIR, f) for i, f in enumerate(files)}
            if not self.mp3_files:
                self.get_logger().error(f"目录 {self.MP3_DIR} 中没有找到 MP3 文件，请检查目录内容。")
                raise FileNotFoundError("No MP3 files found")
            
            self.get_logger().info("可用歌曲列表:")
            for num, path in self.mp3_files.items():
                self.get_logger().info(f"{num}: {os.path.basename(path)}")
        except Exception as e:
            self.get_logger().error(f"获取 MP3 文件失败: {str(e)}")
            raise

Retrieve all MP3 files from the specified directory and save their file paths in a dictionary.

  • Play the Audio

72
73
74
75
76
77
78
79
80
81
82
83
84
85
	def play_audio(self, file_path):
        """播放音频文件"""
        if not os.path.exists(file_path):
            self.get_logger().error(f"文件不存在: {file_path}")
            return
        try:
            self.mpg123_process = subprocess.Popen(
                ["mpg123", file_path],
                stdout=subprocess.DEVNULL,
                stderr=subprocess.DEVNULL
            )
            self.get_logger().info(f"正在播放音乐: {file_path}")
        except Exception as e:
            self.get_logger().error(f"播放音频失败: {str(e)}")

The program uses subprocess.Popen to run the mpg123 command and play a specified MP3 file. If the file is missing or mpg123 isn’t installed, it will display an error message.

  • Example Action

137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
	def linkage(self, times=1):
        """多轴联动"""
        try:
            for i in range(0, 15, 1):
                self.set_pose(roll=math.radians(i), run_time=30)
                time.sleep(0.03)
            for i in range(0, 15, 1):
                self.set_pose(pitch=math.radians(i), run_time=30)
                time.sleep(0.03)

            for _ in range(times):
                for i in range(15, -15, -1):
                    self.set_pose(roll=math.radians(i), pitch=math.radians(15), run_time=30)
                    time.sleep(0.03)
                for i in range(15, -15, -1):
                    self.set_pose(roll=math.radians(-15), pitch=math.radians(i), run_time=30)
                    time.sleep(0.03)
                for i in range(-15, 15, 1):
                    self.set_pose(roll=math.radians(i), pitch=math.radians(-15), run_time=30)
                    time.sleep(0.03)
                for i in range(-15, 15, 1):
                    self.set_pose(roll=math.radians(15), pitch=math.radians(i), run_time=30)
                    time.sleep(0.03)
        except Exception as e:
            self.get_logger().error(f"多轴联动出错: {str(e)}")

As an example, the robotic dog’s posture (roll and pitch) is adjusted to perform coordinated multi-axis movements.

  • Robot Motion Control

 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
	def mp3_moon_callback(self):
        """定时器回调函数，执行动作序列"""
        try:
            # 原地踏步
            self.set_move(x=0.01)
            time.sleep(3)
            self.set_move(x=0.0)
            time.sleep(1)

            # 多轴联动
            self.linkage(2)

            # 机器狗站立
            self.set_pose()

            # 向前走
            self.set_move(x=5.0)
            time.sleep(3)
            # 向后走
            self.set_move(x=-5.0)
            time.sleep(2)
            # 停止
            self.set_move(x=0.0)
            time.sleep(1)

            # 执行滑步动作组
            msg = SetRunActionName.Request()
            msg.name = 'moonwalk.d6ac'
            msg.wait = True
            self.run_action_group_srv.call_async(msg)
            time.sleep(0.1)
            self.run_action_group_srv.call_async(msg)
            time.sleep(0.1)

            # 机器狗站立
            self.set_pose()
            time.sleep(0.5)

            # 向前走
            self.set_move(x=5.0)
            time.sleep(3)
            # 向后走
            self.set_move(x=-5.0)
            time.sleep(2)
            # 停止
            self.set_move()
            time.sleep(1)
        except Exception as e:
            self.get_logger().error(f"执行动作序列时出错: {str(e)}")

It can control the robotic dog to carry out basic actions like standing, marching in place, multi-axis coordination, moving forward, and moving backward, as well as execute predefined action sequences such as the moonwalk.

28.3.7 Function Expansion

  • Modifying Dance Movements

(1) Double-click the Terminator icon on the Raspberry Pi desktop to launch the terminal and access the ROS2 development environment.

(2) Enter the following command and press Enter to navigate to the directory where the program is located:

cd ros2_ws/src/example/example/puppypi_with_sensor

(3) Run the following command to open the program file:

vim mp3_moonwalk_demo.py

(4) Locate the code section in the area highlighted in the image.

(5) Press the “i” key to enter edit mode. Modify the action group name at this location to match the action group from the ActionGroups file. For example, change it to jump.d6ac, as shown in the image.

(6) Once the modification is complete, press the Esc key. Then, type the following command (note that a colon : precedes the wq) and press Enter to save the changes:

:wq

(7) Afterward, repeat the steps outlined in this document to run the program.

28.4 Voice Interaction

28.4.1 Preparation

Refer to 28.1 Introduction to WonderEcho Pro and Installation to flash the firmware onto the WonderEcho Pro module. Once the firmware has been successfully burned, install the module onto the robot. By default, the wake-up word for the WonderEcho Pro factory firmware is Hello, HiWonder. If you wish to change the wake-up word to PuppyPi, please refer to the tutorial titled 28.1.5 WonderEcho Pro Installation for step-by-step instructions.

28.4.2 Brief Game Instructions

The source code for this program is located in the Docker container at:

ros2_ws/src/example/example/puppypi_with_sensor/voice_interaction_demo.py

This program receives voice commands through the serial port and uses the ROS 2 publisher mechanism to send control instructions to the quadruped robot, enabling voice-based interactive control.

28.4.3 Starting and Stopping the Game

Note

When entering commands, ensure that case sensitivity is strictly followed. You can use the Tab key to auto-complete keywords.

(1) Power on the robot, then follow the steps in 3.3 Docker Container Introduction and Entry and 3.4 ROS Version Switch Tool Guide to connect via the VNC remote control software and switch to the ROS 2 environment.

(2) On the Raspberry Pi desktop, double-click the Terminator icon to open the command-line terminal and enter the ROS2 development environment.

(3) Execute the following command to disable the app service:

ros2 launch example voice_interaction_demo.launch.py

(4) To exit this feature, simply press Ctrl+C in the terminal. If the command does not stop the program, please try again.

28.4.4 How It Works

Note

The wake-up commands vary depending on the firmware installed. If you are using the Chinese Xiaohuan Xiaohuan firmware, please use the Chinese commands below. For PuppyPi or Hello HiWonder firmware, use the English commands provided.

User Command Terminal print Actions
puppypi AA 55 03 00 FB
STAND-STILL AA 55 00 76 FB March on the spot
LOOK-UP AA 55 00 8D FB Look up
GET-DOWN AA 55 00 0B FB Lie down
TATTENTION AA 55 00 0A FB Stand at attention

Note

Once the program is running, the robotic dog will stay in a standing position. To activate voice control, simply say PuppyPi After activation, you can issue various voice commands—for example, say LOOK-UP and the dog will lift its head, GET-DOWN to switch to a resting posture, ATTENTION to return to standing, or STAND-STILL to start stepping on the spot. If no commands are received for over 15 seconds, the system will automatically enter sleep mode. Just say PuppyPi again to wake it up.

28.4.5 Program Download

Source Code

28.4.6 Brief Program Analysis

  • Library Files Import

 4
 5
 6
 7
 8
 9
10
11
12
import os
import sys
import rclpy
import math
import signal
import serial
import time
from rclpy.node import Node
from puppy_control_msgs.msg import Velocity, Pose, Gait

  • Robot Movement and Posture Settings

27
28
29
30
31
GaitConfig = {'overlap_time':0.2, 'swing_time':0.2, 'clearance_time':0.0, 'z_clearance':3.0}
# overlap_time:4脚全部着地的时间，单位秒(the time when all four legs touch the ground, measured in seconds)
	# swing_time：2脚离地时间，单位秒(the time duration when legs are off the ground, measured in second)
	# clearance_time：前后脚间隔时间，单位秒(the time interval between the front and rear legs, measured in seconds)
	# z_clearance：走路时，脚抬高的距离，单位cm(the distance the paw needs to be raised during walking, measured in centimeters)

① overlap_time: The duration when all four legs are in contact with the ground.

② swing_time: The duration during which two legs are lifted off the ground.

③ clearance_time: The time interval between the movement of the front and hind legs.

④ z_clearance: The height by which the legs are lifted during walking.

  • Serial Communication

40
	        self.ser = serial.Serial('/dev/ttyUSB0', 115200, timeout=1)  

64
65
66
67
	    if self.ser.in_waiting > 0:        
            data = self.ser.read(5) 
            hex_data = ' '.join(format(byte, '02X') for byte in data)           
            self.get_logger().info(f"Received data: {hex_data}")

Monitor the serial port for incoming data. When data is available, read 5 bytes and convert them into a hexadecimal string.

  • Node Initialization

40
41
42
43
	    self.ser = serial.Serial('/dev/ttyUSB0', 115200, timeout=1)  
        self.pose_publisher = self.create_publisher(Pose, '/puppy_control/pose',10)
        self.gait_publisher = self.create_publisher(Gait, '/puppy_control/gait', 10)
        self.velocity_publisher = self.create_publisher(Velocity, '/puppy_control/velocity',10)

Create three publishers to send control commands for the robot dog’s posture, gait, and speed.

  • Voice Command Processing and Control

62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
	def asr_callback(self):
         
        if self.ser.in_waiting > 0:        
            data = self.ser.read(5) 
            hex_data = ' '.join(format(byte, '02X') for byte in data)           
            self.get_logger().info(f"Received data: {hex_data}")
            
            if hex_data == "AA 55 00 8D FB": # 抬头(raise the head)               
                self.pose_publisher.publish(Pose(stance_x=self.PuppyPose['stance_x'], stance_y=self.PuppyPose['stance_y'], x_shift=self.PuppyPose['x_shift'],
                                       height=self.PuppyPose['height'], roll=self.PuppyPose['roll'], pitch=math.radians(20), yaw=self.PuppyPose['yaw'], run_time=500))
            elif hex_data == "AA 55 00 0B FB":# 趴下(lie down)
                self.pose_publisher.publish(Pose(stance_x=self.PuppyPose['stance_x'], stance_y=self.PuppyPose['stance_y'], x_shift=self.PuppyPose['x_shift'],
                                       height=self.PuppyPose['height'] + 4, roll=self.PuppyPose['roll'], pitch=self.PuppyPose['pitch'], yaw=self.PuppyPose['yaw'], run_time=500))
            elif hex_data == "AA 55 00 0A FB":# 立正(stand at attention)
                self.pose_publisher.publish(Pose(stance_x=self.PuppyPose['stance_x'], stance_y=self.PuppyPose['stance_y'], x_shift=self.PuppyPose['x_shift'],
                                       height=self.PuppyPose['height'], roll=self.PuppyPose['roll'], pitch=self.PuppyPose['pitch'], yaw=self.PuppyPose['yaw'], run_time=500))
            elif hex_data == "AA 55 00 76 FB":# 原地踏步(stepping in the place)
                self.pose_publisher.publish(Pose(stance_x=self.PuppyPose['stance_x'], stance_y=self.PuppyPose['stance_y'], x_shift=self.PuppyPose['x_shift'],
                                       height=self.PuppyPose['height'], roll=self.PuppyPose['roll'], pitch=self.PuppyPose['pitch'], yaw=self.PuppyPose['yaw'], run_time=500))
                time.sleep(0.5)
                self.velocity_publisher.publish(Velocity(x=0.1, y=0.0, yaw_rate=0.0))
                time.sleep(2)
                self.velocity_publisher.publish(Velocity(x=0.0, y=0.0, yaw_rate=0.0))
            elif hex_data == "AA 55 02 00 FB": 
                self.get_logger().info('restart wakeup!!!!!!!!!!!!!!!!!!')
            else:
                time.sleep(0.02)
                                       
        else:
            time.sleep(0.02)

Perform actions according to the received voice commands.