5. Python Programming Projects

5.1 Introduction to Hiwonder Python Editor

In this section, we will learn about the connection methods and specific functions of the Hiwonder Python Editor.

Note

If the editor fails to open, please change the editor’s name to English only, such as ‘Hiwonder’.

5.1.1 Function Introduction

The editor interface is divided into 5 sections as below:

Each area has its corresponding functions, as shown in the table below:

No.	Area Name	Function Description
1	Menu Bar	Contains File, Edit, View, Connect, Run, and Help
2	Toolbar	Includes some commonly used shortcut keys that have the same effect as certain keys in the menu bar
3	File List	Divided into multiple project files in the device and locally, allowing you to view the contents of project files (folders, source code, etc.)
4	Code Editing Area	Allows you to view and write code
5	Terminal	Displays message logs and debugging information. When no device is connected, only the message logs can be viewed

5.1.2 Operation Instructions

• Importing Local Projects

(1) When importing a local project for the first time, left-click on ‘Local Projects’ to open the file selection list. (For subsequent imports, right-click on ‘Local Projects’ and select ‘Switch Project Path’).

(2) Select Appendix/Program Collection/Python Project Programs and click the ‘Select Folder’ button.

(3) The files in the folder will be automatically added to the local projects, and you will be able to see them under ‘Local Projects’.

Note

Importing a local project brings files from your computer into the editor, not downloading them to the ESP32 core board.

• Viewing Imported Files/Programs

Here, we can double-click on the program file in the file list to view the detailed code. For example, let’s take Smart_fan.py:

Similarly, after downloading the program file to the ESP32 core board, you can double-click it under the ‘Devices’ list to view the program file.

• Code Writing and Storage

The code editing area on the right side of the interface supports functions such as creating, viewing, editing, modifying, and saving code. Before you start writing code, please read the following notes:

(1) Users cannot directly create files within the ‘Devices’ tab, and files in the ‘Devices’ can only be saved and modified by downloading. If you need to back up, please copy them to the local projects first.

(2) Do not modify action group files with the ‘.rob’ extension within the editor to avoid unknown formatting errors. If you need to modify action group files, please do so on the host machine.

(3) Among the provided low-level program files, ‘main.py’ is the main program for the device. All functionalities of the robot need to be initiated through this file, meaning that both reset and power-on operations require this program to be executed. If it becomes unresponsive, subsequent operations cannot proceed. Therefore, if the user needs to add functions to this file, it is recommended to rename the program as a precaution. If ‘main.py’ is renamed, even if a special freeze occurs during debugging (when the shortcuts ‘Ctrl+C’ and ‘Ctrl+D’ become unresponsive), you only need to reset the control board and delete and re-download the required program.

• Program Download and Execution

Downloading a program is an interactive action between the editor and the device. Using ‘smart_fan.py’ as an example:

(1) After selecting the ‘smart_fan.py’ file in the ‘Local Projects’ tab, click in the toolbar or right-click the file and choose ‘Download and Run’

(2) You can view the download progress and completion status in the terminal interface. Since ‘Download and Run’ was selected in the previous step, you can also observe the program’s running effect.

(3) Once the download is complete, the program will appear in the file list under the ‘Devices’ tab.

(4) Finally, delete the original ‘main.py’ file of Tonybot and rename the downloaded ‘smart_fan.py’ to ‘main.py.’

Additionally, here are a few points for users to be aware of:

① Besides this download method, you can also rename the file you want to download to ‘main.py’ before proceeding with the download.

② The ‘Download and Run’ function first resets the device (restarts) and then downloads and runs the program, which helps enhance the stability of the program execution.

③ If you do not need to execute the program immediately, you can click or right-click the target file and select ‘Download.’ Before executing the program later, we can first click to reset the device and then run the program.

• Terminal Usage (Debugging)

The terminal is a functional area that combines an information window and a debugging interface. However, it is important to note that if no device is connected, the terminal area is only for viewing information and cannot be used for editing or debugging.

Regarding information viewing, you have already experienced this in the previous steps, so I won’t elaborate further. Here, I will mainly explain the debugging functions.

(1) The terminal supports code input. When you input the code `print(123)` in the terminal and press Enter, the output will be as follows:

(2) Additionally, the terminal supports automatic indentation. When you type a Python statement that ends with a colon (such as `if`, `for`, or `while`) and press Enter, the next line will continue at the same indentation level as a regular statement or, when appropriate, at a different indentation level. If you press the Backspace key, it will undo one level of indentation.

(3) To copy and paste code, select the target code and right-click in the terminal interface to perform the operation.

It is important to note that because the terminal has automatic indentation, you must press `Ctrl+E` to enter edit mode before pasting code; otherwise, you may encounter indentation errors during debugging. The following image shows the correct method for copying and pasting, with the indentation format appearing correctly.

The following image shows an incorrect indentation format:

To exit edit mode, you can press Ctrl+C. Additionally, if you write an infinite loop, you can also press `Ctrl+C` to exit.

Friendly Reminder: In the terminal, the Ctrl+C shortcut key can only be used to interrupt a running program; it does not have a copy function, and Ctrl+V has no paste function.

(4) When entering commands in the terminal, you can use the Tab key for code completion. For example, after typing os in the terminal, pressing the Tab key will yield the following result:

If there are two or more options available for the current code completion, the terminal will list all options; if there is only one option, the terminal will automatically complete it; if there are none, it will have no effect.

(5) In the terminal, you can use the “and” keys on your keyboard to view the command history, saving you input time.

For more commands and command descriptions, you can visit http://docs.micropython.org/en/latest/library/uos.html.

5.2 Secondary Development Course

5.2.1 Breathing Light Project

• Project Introduction

In this lesson, we will make the RGB LED on the ultrasonic sensor blink.

• Program Implementation Process

• Introduction to Glowy Ultrasonic Sensor

This is a glowy ultrasonic distance sensor module that communicates via I2C. It reads the distance measured by the ultrasonic sensor using the I2C protocol.

When measuring distance, the module sends out 8 pulses of 40kHz square waves and checks for any returning signals. If a signal is detected, the module outputs a high-level signal, and the duration of this high signal corresponds to the time taken for the ultrasonic waves to travel to the target and back.

The ultrasonic sensor is pre-installed on the robot when it leaves the factory.

• Program Download

Breathing Light Program

(1) Launch the “Hiwonder Python Editor” software .

(2) Drag the “breathing_light.py” file (located in the same folder as this document) into the editor window.

(3) Click the connection button in the menu bar. Once connected, the icon will turn green, indicating a successful connection.

(4) After the connection is established, click the download button in the menu bar to upload the program to Tonybot. Wait for the completion message to appear in the info box below.

• Achieved Effect

Note

For the best experience, try this activity in a dimly lit room.

You’ll see the RGB light on the ultrasonic module glowing blue, with a blink every 2 seconds.

• Program Analysis

Breathing Light Program

(1) Load the necessary libraries: The Hiwonder library handles robot control and low voltage alarms, the time library manages time-related functions, and the Hiwonder_IIC library controls sensors via I2C communication.

import Hiwonder
import Hiwonder_IIC
import time

(2) First, initialize the I2C communication object, then set up the Tonybot robot object, and finally, initialize the ultrasonic sensor to communicate over I2C.

iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar  = Hiwonder_IIC.I2CSonar(iic)

(3) Run the first action from the action group, then connect the head. After a 1-second pause, disconnect the head.

tony.runActionGroup(0 , 1)
tony.attachHead()
time.sleep(1)
tony.detachHead()

(4) Adjust the RGB light color and brightness on the ultrasonic sensor. The parameters control the starting and target values for red, green, and blue, as well as the speed of the breathing effect.

sonar.setRGBBreathingValue(0, 0, 20, 0, 0, 20)

• Function Extension

You can modify the color of the breathing light by following these steps:

(1) In the main function, locate the section of code responsible for the color. When the distance is less than 250, the color will transition from red to blue. Specifically, the red component is set to 250 - color_diff, the green component is set to color_diff, and the blue component is set to 0.

while True:
  distance = sonar.getDistance() * 10
  distance = distance if distance < 1000 else 1000
  color_diff = distance if distance < 250 else 250
  sonar.setRGB(0 , 250-color_diff , color_diff , 0)
  ledmatrix.showNum(distance)
  time.sleep_ms(200)

(2) Adjust the ratio between the red and green components to achieve different colors. For example, if color_diff is 0, the color will be red (250, 0, 0). If color_diff is 250, the color will be green (0, 250, 0). You can change the value of color_diff to create a variety of colors.

(3) After making the changes, refer to the Program Download section to run the updated program.

5.2.2 Distance Ranging & Display Project

• Assembly

• Project Introduction

This section utilizes the ultrasonic module to measure the distance to an obstacle. The measured distance is displayed on the LED dot matrix, and the RGB light color of the ultrasonic module is adjusted accordingly.

• Program Implementation Process

• Sensor Introduction

(1) Glowy Ultrasonic Sensor

This is a light-emitting ultrasonic distance sensor module that communicates via I2C. It reads the distance measured by the ultrasonic sensor using the I2C protocol.

When measuring distance, the module sends out 8 pulses of 40kHz square waves and checks for any returning signals. If a signal is detected, the module outputs a high-level signal, and the duration of this high signal corresponds to the time taken for the ultrasonic waves to travel to the target and back.

(2) LED Dot Matrix Display

The LED dot matrix module utilizes an LED display screen that offers high brightness, flicker-free performance, and easy wiring. It is capable of displaying numbers, text, patterns, and other content. The module consists of two red 8x8 LED matrices and is controlled by the TM640B driver chip, which enables efficient control of the dot matrix screen.

For a detailed introduction to the installation and assembly process, please refer to the tutorial video.

• Program Download

Distance Ranging & Display Program

(1) Launch the “Hiwonder Python Editor” software .

(2) Drag the “distance_display.py” file (located in the same folder as this document) into the editor window.

(3) Click the connection button in the menu bar. Once connected, the icon will turn green, indicating a successful connection.

(4) After the connection is established, click the download button in the menu bar to upload the program to Tonybot. Wait for the completion message to appear in the info box below.

• Achieved Effect

Slowly bring the object closer to the glowy ultrasonic sensor. As the distance decreases, the ultrasonic light will shift towards red, and as the distance increases, it will become greener. The LED dot matrix display will update in real-time to show the measured distance.

• Function Extension

You can customize the color of the breathing light by following these steps:

(1) Find the section of the main function that controls the color. When the distance is less than 250, the color will gradually shift from red to blue. Specifically, the red component is calculated as 250 - color_diff, the green component is color_diff, and the blue component remains 0.

    while True:
        distance = sonar.getDistance() * 10
        distance = distance if distance < 1000 else 1000
        color_diff = distance if distance < 250 else 250
        sonar.setRGB(0 ,250-color_diff , color_diff ,0)
        ledmatrix.showNum(distance)
        time.sleep_ms(200)

(2) You can adjust the balance between the red and green components to create different colors. For example, if color_diff is 0, the color will be red (250, 0, 0), and if color_diff is 250, the color will be green (0, 250, 0). Modify color_diff to get the color you prefer.

(3) Once you’ve made the changes, refer to section “Program Download” to run the updated program.

• Program Analysis

Distance Ranging & Display Program

(1) Load the required libraries:

The Hiwonder library contains functions for robot control and low-voltage alarms.

The time library is used for time-related functions.

The Hiwonder_IIC library is used to control sensors via I2C communication.

import Hiwonder
import Hiwonder_IIC
import time

(2) Initialize the I2C communication object, followed by the initialization of the Tonybot robot object. Next, initialize the ultrasonic sensor object for I2C communication. The ledmatrix is the dot matrix module object, connected to GPIO pins 2 and 5.

iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar  = Hiwonder_IIC.I2CSonar(iic)
ledmatrix = Hiwonder.Digitaltube(2,5)

(3) Execute the first action in the action group. Then, connect the head, and after a one-second pause, disconnect the head.

tony.runActionGroup(0 , 1)
tony.attachHead()
time.sleep(1)
tony.detachHead()

(4) Use the sonar.getDistance() function to retrieve the distance measured by the ultrasonic sensor, and multiply it by 10 to convert the value to millimeters. If the distance exceeds 1000 mm, set it to 1000 mm. Calculate the color_diff value based on the distance. If the distance is greater than 250 mm, limit color_diff to 250 mm. Use the sonar.setRGB(0, 250 - color_diff, color_diff, 0) function to set the head LED color, transitioning from green (for closer distances) to red (for farther distances). Use ledmatrix.showNum(distance) to display the measured distance on the dot matrix module. Finally, call time.sleep_ms(200) to pause the program for 200 milliseconds.

while True:
  distance = sonar.getDistance() * 10
  distance = distance if distance < 1000 else 1000
  color_diff = distance if distance < 250 else 250
  sonar.setRGB(0 , 250-color_diff , color_diff , 0)
  ledmatrix.showNum(distance)
  time.sleep_ms(200)

5.2.3 Distance-Controlled Walking Project

• Project Introduction

In this lesson, the ultrasonic sensor is used to measure obstacles in front of the robot. When the detected distance falls within the predefined range set by the program, the robot will move forward or backward accordingly.

• Program Implementation Process

• Sensor Introduction

This is a light-emitting ultrasonic distance sensor module that communicates via I2C. It reads the distance measured by the ultrasonic sensor using the I2C protocol.

When measuring distance, the module sends out 8 pulses of 40kHz square waves and checks for any returning signals. If a signal is detected, the module outputs a high-level signal, and the duration of this high signal corresponds to the time taken for the ultrasonic waves to travel to the target and back.

• Program Download

Distance-Controlled Walking Program

(1) Launch the “Hiwonder Python Editor” software

(2) Drag the “ultrasonic_follow.py” file (located in the same folder as this document) into the editor window.

(3) Click the connection button in the menu bar. Once connected, the icon will turn green, indicating a successful connection.

(4) After the connection is established, click the download button in the menu bar to upload the program to Tonybot. Wait for the completion message to appear in the info box below.

• Achieved Effect

Note

Please avoid using the ultrasonic sensor to continuously detect objects that are too close.

Place a cardboard box in front of the sensor. If the distance is between 28-40 cm, the robot will move forward. If the distance is between 3-18 cm, the robot will move backward.

• Program Analysis

Distance-Controlled Walking Program

(1) Load the necessary libraries:

The Hiwonder library includes functions for controlling the robot and managing low-voltage alarms.

The time library is used for time-related functions.

The Hiwonder_IIC library controls sensors via I2C communication.

import Hiwonder
import Hiwonder_IIC
import time

(2) Create the following objects:

iic: The I2C communication object.

tony: The Tonybot object.

sonar: The ultrasonic sensor object.

iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar  = Hiwonder_IIC.I2CSonar(iic)

(3) Define the following variables:

Distance: Stores the distance measured by the ultrasonic sensor.

have_move: A flag indicating whether the robot has moved.

step: The current step of the state machine.

last_tick: Holds the timestamp of the last executed action.

# Global variables
Distance = 0
have_move = False
step = 0
last_tick = 

(4) The ticks_ms() function retrieves the current time. If the current time is less than or equal to last_tick, the function returns immediately. If the time has passed, the program proceeds to call sonar.getDistance() to obtain the distance value. When the step variable is set to 0, the program should decide the next action based on the Distance value. For instance, the program might turn on the red light and trigger action 18, depending on the distance.

def Distancewalking():
    """Distance-Controlled Walking Task"""
    global step, last_tick, have_move
    
    if ticks_ms() <= last_tick:
        return
    Distance = sonar.getDistance() * 10
    if step == 0:
        # 若距离过近 (If the distance is too close)
        if 30 < Distance < 180:
            # 亮红灯，执行过渡动作 (Turn on the red light, execute transition action)
            sonar.setRGB(0, 250, 0, 0)
            tony.runActionGroup(18, 1)
            last_tick = ticks_ms() + 350
            have_move = True
            step = 1

• Function Extension

(1) You can modify the ultrasonic distance range in the program by editing the code in the designated section, as shown in the image below.

(2) The program checks if the distance is between 30 mm and 180 mm, in which case the robot will move backward. If the distance is between 300 mm and 400 mm, the robot will move forward. To adjust the ultrasonic distance range, simply update the comparison values in the code, for example:

(3) After making the modifications, refer to section “Program Download” to run the updated program.

5.2.4 Treasure Hunt Project

• Project Introduction

In this lesson, the robot’s servo controlling the ultrasonic sensor will rotate back and forth. When the set conditions are met, the robot will turn in the corresponding direction.

• Program Implementation Process

• Sensor Introduction

(1) Ultrasonic Sensor

This is a light-emitting ultrasonic distance sensor module that communicates via I2C. It reads the distance measured by the ultrasonic sensor using the I2C protocol.

When measuring distance, the module sends out 8 pulses of 40kHz square waves and checks for any returning signals. If a signal is detected, the module outputs a high-level signal, and the duration of this high signal corresponds to the time taken for the ultrasonic waves to travel to the target and back.

(2) LED Dot Matrix Display

The LED dot matrix display is a high-brightness display unit that offers flicker-free visuals and easy wiring. It is capable of displaying numbers, text, images, and more. The module is made up of two 8x8 red LED panels and is controlled by the TM640B driver chip, which manages the dot matrix screen’s operation.

For installation and wiring details, please refer to the section 1.

• Program Download

Treasure Hunt Program

(1) Launch the “Hiwonder Python Editor” software .

(2) Drag the “find_target.py” file (located in the same folder as this document) into the editor window.

(3) Click the connection button in the menu bar. Once connected, the icon will turn green, indicating a successful connection.

(4) After the connection is established, click the download button in the menu bar to upload the program to Tonybot. Wait for the completion message to appear in the info box below.

• Achieved Effect

The servo below the ultrasonic sensor will rotate back and forth while measuring the distance between the robot and nearby obstacles. The robot compares the distances on both sides.

If the distance on one side is shorter, the robot will turn in that direction. For example, if the left side is closer, the robot will turn left; if the right side is closer, it will turn right.

• Program Analysis

Treasure Hunt Program

(1) Library Imports and Variable Initialization:

① First, load the necessary libraries:

• The Hiwonder library, which provides functions for robot control, low-voltage alarms, and other system operations.

• The time library, which is used for time-related functions.

• The Hiwonder_IIC library, which enables control of sensors via I2C communication.

import Hiwonder
import Hiwonder_IIC
import time
from time import ticks_ms, sleep_ms

② Next, initialize the I2C communication object, followed by the Tonybot robot object. Finally, initialize the ultrasonic sensor object to communicate using the I2C protocol.

iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)

③ The action group IDs for left and right turns are defined here. Using macro definitions for these actions enhances code readability and clarity.

④ The servo offset for the pan-tilt is also specified. Users can measure the deviation between the expected and actual servo angles, and declare the offset here. This ensures that subsequent control algorithms can compensate for the discrepancy, improving accuracy during operation.

# 初始化 (Initialization)
TURN_RIGHT = 35  # 右转动作组 (Right turn action group)
TURN_LEFT = 34   # 左转动作组 (Left turn action group)
B = 0            # 舵机偏差 (Servo deviation)

(2) Loop

Within the loop function, the ESP32 continuously calls the find_ function to measure distances, calculate the optimal direction, and adjust the robot’s orientation accordingly.

while True:
    find_()
    sleep_ms(100)

(3) Optimal Navigation Decision Function — find_

① Upon entering the find_ function, the robot first calls the getDistance function to measure the distance from the front (with the servo rotated to 90°) to the target box.

First execution (step = 0):

• The program executes the logic under case 0, using the measured distance to control the color of the ultrasonic RGB module.

• If the front-facing distance to the box is within a moderate range (30–350), the program proceeds to evaluate case 1 logic but retains step = 0. If the distance is too large, step is set to 1, and the next iteration will enter case 1 to search for the nearest direction to the box.

    if step == 0:
        if 30 < distance <= 350:
            tony.detachHead()
            sleep_ms(100)
            sonar.setRGB(0, 50, 50, 50)  # 白色(White)
            step = 0
        else:
            tony.attachHead()
            sonar.setRGB(0, 0, 50, 0)    # 绿色(Green)
            step = 1

Second execution (case 1):

• The servo rotates to scan four angles: 45° (front-left), 0° (left), 145° (front-right), and 180° (right), measuring the distance to the box at each angle.

• Step is then updated to 2, and the next iteration will enter case 2 for direction analysis and decision-making.

    elif step == 1:
        tony.moveHeadAngle(45 + B)    # 转到右边 (Turn to the right)
        sleep_ms(600)
        distanceR = sonar.getDistance() * 10     # 测量右边距离 (Measure the right distance)
        
        tony.moveHeadAngle(0 + B)     # 转到右上 (Turn to the right upper)
        sleep_ms(600)
        distanceRC = sonar.getDistance() * 10    # 测量右上距离 (Measure the right upper distance)
        
        tony.moveHeadAngle(145 + B)   # 转到左边 (Turn to the left)
        sleep_ms(800)
        distanceL = sonar.getDistance() * 10     # 测量左边距离 (Measure the left distance)
        
        tony.moveHeadAngle(180 + B)   # 转到左上 (Turn to the left upper)
        sleep_ms(600)
        distanceLC = sonar.getDistance() * 10    # 测量左上距离 (Measure the left upper distance)
        
        tony.moveHeadAngle(90 + B)    # 转回中间 (Turn back to the center)
        sleep_ms(600)
        step = 2

Third execution (case 2):

• The program first standardizes the distance values. If the box is not detected within the valid range (sensor returns 0), the corresponding direction is treated as out of range, and the distance is set to a default high value of 9999.

• It then performs logic evaluation across all directions to identify the one with the shortest valid distance. If the closest direction is on the left, the ultrasonic RGB module lights up blue; if on the right, it lights up red. The robot then turns left or right accordingly to align with the selected direction.

Note

If the front-facing distance is still shorter than other directions, even if it exceeds the “moderate” range, the robot will not perform a turn. In this case, the ultrasonic RGB module remains green as in case 0.

② After three cycles of executing the find_ function, the robot completes a full decision-making process for optimal navigation.

• Function Extension

(1) Adjusting the Detection Range

① To modify the detection range, update the code within the designated section of the program, as shown in the example below.

② The system identifies a “treasure box” when the detected distance between the object and the robot is greater than 30 mm but less than 350 mm. To change the detection range, simply adjust the conditions in the distance judgment. For example:

(2) Modifying the Turning Conditions

① To modify the turning conditions, edit the appropriate code section within the program, as illustrated in the example below.

② The program evaluates five directional measurements, identifies the closest direction, and checks if the distance falls within the specified detection range. If it does, the robot executes a turn in that direction.

③ To adjust the distance thresholds for turning, modify the distance condition for each direction. For instance, if the left direction is the closest and its distance is less than 400 mm, the robot will turn left, as shown in the example below:

5.2.5 Low-Altitude Navigation Project

• Project Introduction

When the ultrasonic sensor detects an obstacle ahead, the robot will move forward a short distance in a crouching posture. Afterward, it will return to an upright position and continue moving forward in its normal stance.

• Program Implementation Process

• Sensor Introduction

This is a light-emitting ultrasonic distance sensor module that communicates via I2C. It reads the distance measured by the ultrasonic sensor using the I2C protocol.

When measuring distance, the module sends out 8 pulses of 40kHz square waves and checks for any returning signals. If a signal is detected, the module outputs a high-level signal, and the duration of this high signal corresponds to the time taken for the ultrasonic waves to travel to the target and back.

• Program Download

Low Posture Crossing Program

(1) Launch the “Hiwonder Python Editor” software .

(2) Drag the “low_posture_crossing.py” file (located in the same folder as this document) into the editor window.

(3) Click the connection button in the menu bar. Once connected, the icon will turn green, indicating a successful connection.

(4) After the connection is established, click the download button in the menu bar to upload the program to Tonybot. Wait for the completion message to appear in the info box below.

• Achieved Effect

Note

Avoid using the ultrasonic sensor to continuously detect very close objects.

You can create a “small tunnel” using several cardboard boxes. As the robot moves forward, if the ultrasonic sensor detects an obstacle ahead, it will trigger a beeping sound as a warning. At the same time, the RGB light on the ultrasonic sensor will change from green to red.

The robot will then adopt a crouching position and move forward for a short distance. After passing through the obstacle, it will return to a standing position and continue moving forward as usual.

• Program Analysis

(1) Load Libraries:

The Hiwonder library includes functions for robot control, low-voltage alarms, and more.

The time library is used for time-related functions. Import the ticks_ms and sleep_ms functions from it.

The Hiwonder_IIC library is used to control sensors via the I2C interface.

import Hiwonder
import Hiwonder_IIC
import time
from time import ticks_ms, sleep_ms

(2) Initialize Objects:

IIC: Initialize the I2C interface object.

tony: Initialize the Tonybot robot object.

sonar: Initialize the ultrasonic sensor object, which uses the I2C interface.

buzzer: Initialize the buzzer object.

iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)
buzzer = Hiwonder.Buzzer()

(3) Action Group IDs:

H_STAND: Normal standing position, action group ID = 0.

L_STAND: Crouching position, action group ID = 30.

H_GO_FORWARD: Normal forward movement, action group ID = 21.

L_GO_FORWARD: Forward movement in a crouching position, action group ID = 31.

MIN_DISTANCE: Minimum safe distance = 150 mm.

# 动作组定义(Action group definitions)
H_STAND = 0
L_STAND = 30
H_GO_FORWARD = 21
L_GO_FORWARD = 31

# 常量定义 (Constant definitions)
MIN_DISTANCE = 150  # 150mm

(4) Execute Initial Action:

Run the first action in the action group, then connect the robot’s head.

After a 1-second pause, disconnect the head.

# 初始化机器人 (Initialize robot)
tony.runActionGroup(0, 1)
tony.attachHead()
time.sleep(1)
tony.detachHead()

(5) Global Variables:

step: Tracks the current step in the process, initially set to 0.

last_tick: Stores the last timestamp, initially set to 0.

# 全局变量 (Global variables)
step = 0
last_tick = 0

(6) Upon entering the function, the robot follows different motion logic based on the current

First Execution (step = 0):

①　Execute the code under case 0. If the measured distance is greater than the threshold (i.e., no obstacle ahead and distance = 0), the robot will perform a predefined action group and light up a green LED. On the next call to this function, the logic for the second step will be executed.

②　If the measured distance is less than the threshold, the next function call will still follow the logic for the first execution.

    if step == 0:
        if Distance > MIN_DISTANCE or Distance == 0:
            sonar.setRGB(0, 0, 50, 0)  # 绿色 (Green light)
            tony.runActionGroup(18, 1)
            sleep_ms(400)
            step = 1

Second Execution (step = 1):

①　Execute the code under case 1. If a new measurement taken since the last function call indicates the distance is now below the threshold, the robot will execute two action groups, light up a red LED, and activate the buzzer as an alert. It will then switch to a low-stance standing posture. On the next function call, it will proceed to the third step of the logic.

②　If the new distance remains above the threshold, the robot continues with a high-stance walking motion. The next function call will still follow the logic of the second execution.

    elif step == 1:
        if 0 < Distance < MIN_DISTANCE:
            tony.runActionGroup(18, 1)
            sleep_ms(2000)
            tony.runActionGroup(19, 1)
            sleep_ms(2000)
            sonar.setRGB(0, 50, 0, 0)  # 红色 (Red light)
            buzzer.playTone(1500, 100 , False)  # 蜂鸣器响 (Play tone on the buzzer)
            tony.runActionGroup(L_STAND, 1)  # 运行下蹲 (Run squat action)
            sleep_ms(1300)
            step = 2
        else:
            tony.runActionGroup(H_GO_FORWARD, 1)  # 正常前进 (Move forward normally)
            sleep_ms(1150)

Third Execution (step = 2):

①　Execute the code under case 2. If the latest measurement still shows the distance below the threshold, the robot will perform two action groups, light up a blue LED, and transition from a low-stance standing position into low-stance forward movement.

②　If the new distance is greater than the threshold, the robot maintains its low-stance standing posture. The next function call will still follow the logic of the third execution.

    elif step == 2:
        if Distance > MIN_DISTANCE or Distance == 0:
            sleep_ms(500)
            sonar.setRGB(0, 0, 0, 50)  # 蓝色 (Blue light)
            tony.runActionGroup(L_GO_FORWARD, 14)  # 以下蹲姿态前进 (Move forward in squat position)
            sleep_ms(28500)
            step = 3

Fourth Execution (step = 3):

①　The robot first switches back to a low-stance standing posture, ending its previous low-stance movement, then transitions into a stationary high-stance posture.

②　On the next function call, the logic loop returns to the first step (step = 0).

    elif step == 3:
        tony.runActionGroup(L_STAND, 1)  # 运行下蹲 (Run squat action)
        sleep_ms(2000)
        sleep_ms(200)
        tony.runActionGroup(H_STAND, 1)  # 正常立正 (Run standing action)
        sleep_ms(2000)
        sleep_ms(200)
        step = 0

• Function Extension

You can modify the color of the breathing light. Follow these steps:

(1) Find the section in the main function code that controls the color. When the distance is less than 250, the color will gradually transition from red to blue. Specifically:

The red component is calculated as 250 - color_diff.

The green component is set to color_diff.

The blue component remains at 0.

(2) Adjust the ratio between the red and green components to achieve different colors. For instance:

If color_diff is 0, the color will be red (250, 0, 0).

If color_diff is 250, the color will be green (0, 250, 0).

You can modify color_diff to create a range of colors.

(3) Once you’ve made the changes, refer to “Program Download” to run the updated program.

5.2.6 Head-Shaking Obstacle Avoidance Project

• Project Introduction

When the ultrasonic sensor detects an obstacle ahead, the robot will move to avoid the obstacle, implementing intelligent obstacle avoidance.

• Program Implementation Process

• Sensor Introduction

This is a light-emitting ultrasonic distance sensor module that communicates via I2C. It reads the distance measured by the ultrasonic sensor using the I2C protocol.

When measuring distance, the module sends out 8 pulses of 40kHz square waves and checks for any returning signals. If a signal is detected, the module outputs a high-level signal, and the duration of this high signal corresponds to the time taken for the ultrasonic waves to travel to the target and back.

• Program Download

Head-Shaking Obstacle Avoidance Program

(1) Launch the “Hiwonder Python Editor” software .

(2) Drag the “ultrasonic_obstacle_avoidance.py” file (located in the same folder as this document) into the editor window.

(3) Click the connection button in the menu bar. Once connected, the icon will turn green, indicating a successful connection.

(4) After the connection is established, click the download button in the menu bar to upload the program to Tonybot. Wait for the completion message to appear in the info box below.

• Achieved Effect

Note

Avoid using the ultrasonic sensor to continuously detect objects that are very close.

When an obstacle is detected in front of the robot, it will stop and perform a head-shaking motion to check for obstacles on the left and right sides. Based on the results of this check, the robot will turn towards the direction with no obstacles and continue moving forward.

• Program Analysis

Head-Shaking Obstacle Avoidance Program

(1) Load Required Libraries:

Hiwonder library: Contains functions for robot control, low voltage alerts, etc.

time library: Provides time-related functions.

Hiwonder_IIC library: Used to control the sensor via IIC communication.

import Hiwonder
import Hiwonder_IIC
import time
from time import ticks_ms, sleep_ms

(1) Initialize Objects

Initialize the IIC interface object (iic).

Initialize the Tonybot robot object (tony).

Initialize the ultrasonic sensor object (sonar), which depends on the IIC interface.

iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)

(2) Define Action Groups:

Action group for moving forward, backward, left turn, and right turn with predefined action group numbers.

# 动作组定义 (Action group definitions)
GO_FORWARD = 21  # 前进动作组 (Forward action group)
GO_BACK = 22    # 后退动作组 (Backward action group)
TURN_LEFT = 23  # 左转动作组 (Left turn action group)
TURN_RIGHT = 24 # 右转动作组 (Right turn action group)

(3) Set Parameters:

MIN_DISTANCE_TURN: Set to 200 mm. When the detected distance is less than 200 mm, the robot will perform an obstacle avoidance operation.

BIAS: Set to 0, meaning there is no servo bias or deviation.

Initialize the robot, run action group 0, attach the head, wait for 1 second, then detach the head. (This can be adjusted to change the robot’s startup position by modifying the action group numbers.)

# 常量定义 (Constant definitions)
MIN_DISTANCE_TURN = 200  # 避障距离 (Obstacle avoidance distance)
BIAS = 0                 # 舵机偏差 (Servo bias)

# 初始化机器人 (Initialize robot)
tony.runActionGroup(0, 1)
tony.attachHead()
time.sleep(1)
tony.detachHead()

(4) Define Global Variables:

Track the current step, the last timestamp, distances in the front, left, and right directions, movement status, and whether the last action was a backward movement.

# 全局变量 (Global variables)
step = 0
last_tick = 0
Distance = 0     # 中间位置距离 (Distance at the center position)
LDistance = 0    # 左侧距离 (Distance on the left side)
RDistance = 0    # 右侧距离 (Distance on the right side)
have_move = False
lastActionIsGoBack = False  # 添加新变量 (New variable to track if last action was to go back)

(5) getAllDistance() Function:

This function measures the distances in the front, left, and right directions. Here’s the process:

Set the ultrasonic sensor to blue-green light.

Front Distance: Adjust the head to the middle (90° + BIAS), wait for 200 ms, then read the distance and convert it to mm by multiplying by 10.

Left Distance: Adjust the head to 40° (145° + BIAS) and 80° (180° + BIAS), wait 400 ms between each adjustment, then read the distance and multiply by 10. Take the smaller value as the final left distance.

Right Distance: Adjust the head to 40° (45° + BIAS) and 80° (0° + BIAS), wait 600 ms and 400 ms for each, read the distance, and multiply by 10. Take the smaller value as the final right distance.

Reset the Head: Return the head to the middle (90° + BIAS), and wait 400 ms.

def getAllDistance():
    """获取三个方向的距离(Get distances from three directions)"""
    global Distance, LDistance, RDistance
    
    sonar.setRGB(0, 0, 50, 50)  # 蓝绿混合 (Blue-green mixed color)
    tony.moveHeadAngle(90 + BIAS)  # 中间位置 (Move to center position)
    sleep_ms(200)
    Distance = sonar.getDistance() * 10
    
    tony.moveHeadAngle(145 + BIAS)  # 左侧40度 (Left side 40 degrees)
    sleep_ms(400)
    temp_distance = sonar.getDistance() * 10
    
    tony.moveHeadAngle(180 + BIAS)  # 左侧80度 (Left side 80 degrees)
    sleep_ms(400)
    LDistance = sonar.getDistance() * 10
    
    if temp_distance < LDistance:  # 取较小值 (Take the smaller value)
        LDistance = temp_distance
    
    tony.moveHeadAngle(45 + BIAS)  # 右侧40度 (Right side 40 degrees)
    sleep_ms(600)
    temp_distance = sonar.getDistance() * 10
    
    tony.moveHeadAngle(0 + BIAS)   # 右侧80度 (Right side 80 degrees)
    sleep_ms(400)
    RDistance = sonar.getDistance() * 10
    
    if temp_distance < RDistance:   # 取较小值 (Take the smaller value)
        RDistance = temp_distance
    
    tony.moveHeadAngle(90 + BIAS)  # 回到中间 (Move back to center)
    sleep_ms(400)

(6) This function handles obstacle avoidance movements. The steps are as follows:

Check if the current time is less than the last recorded timestamp. If so, exit the function.

If the current step is 0, check the front distance.

If the distance is greater than or equal to MIN_DISTANCE_TURN or is 0, change the RGB light to green, run action group 18, wait 400 ms, and then continue moving forward. Update the last_tick timestamp after the operation.

def obstacleAvoidance():
    """避障行走任务(Obstacle avoidance walking task)"""
    global step, last_tick, Distance, have_move, lastActionIsGoBack
    
    if ticks_ms() <= last_tick:
        return
        
    if step == 0:
        Distance = sonar.getDistance() * 10
        if Distance >= MIN_DISTANCE_TURN or Distance == 0:
            sonar.setRGB(0, 0, 50, 0)  # 绿色 (Green light)
            tony.runActionGroup(18, 1)
            sleep_ms(400)
            tony.runActionGroup(GO_FORWARD, 0)  # 持续前进 (Keep moving forward)
            last_tick = ticks_ms() + 1300
            have_move = True
            lastActionIsGoBack = False  # 记录不是后退动作 (Record that the last action was not to go back)

• Function Extension

(1) To modify the obstacle avoidance actions, update the corresponding section of the code as shown in the image below.

GO_FORWARD = 21  # 前进动作组 (Forward action group)
GO_BACK = 22    # 后退动作组 (Backward action group)
TURN_LEFT = 23  # 左转动作组 (Left turn action group)
TURN_RIGHT = 24 # 右转动作组 (Right turn action group)

(2) We have defined action group numbers using macros. These action group numbers represent the actions downloaded to the robot through the control software. To modify the obstacle avoidance behavior, simply adjust the relevant macro definitions, as shown in the image below.

GO_FORWARD = 25  # 前进动作组 (Forward action group)
GO_BACK = 22    # 后退动作组 (Backward action group)
TURN_LEFT = 23  # 左转动作组 (Left turn action group)
TURN_RIGHT = 24 # 右转动作组 (Right turn action group)

(3) After making the changes, refer to section “Program Download” to run the updated program.

5.2.7 Fall and Recover Project

• Project Introduction

This section utilizes an accelerometer to monitor the current state. Upon detecting a fall, the robot will initiate the action to stand up.

• Program Implementation Process

• Sensor Introduction

This section makes use of the onboard QMI8658 motion sensor, which has a wide range of applications, including handheld gaming devices, 3D controllers, and portable navigation systems.

It integrates a 3-axis MEMS gyroscope, a 3-axis MEMS accelerometer, and an expandable Digital Motion Processor (DMP).

• Program Download

Fall and Recover Program

(1) Launch the “Hiwonder Python Editor” software .

(2) Drag the “rise_after_fall.py” file (located in the same folder as this document) into the editor window.

(3) Click the connection button in the menu bar. Once connected, the icon will turn green, indicating a successful connection.

(4) After the connection is established, click the download button in the menu bar to upload the program to Tonybot. Wait for the completion message to appear in the info box below.

• Achieved Effect

Note

In the “Fall and Stand Up” mode, when the robot is powered on or restarted, the IMU will initialize. Wait for a “beep” sound, which indicates that the initialization is complete.

After turning on the robot, place it flat on either its front or back. After a brief moment, the robot will automatically perform the standing-up action. (Please note that the standing-up action differs depending on whether the robot is placed face-up or face-down.)

• Program Analysis

Fall and Recover Program

(1) Import Libraries:

The program begins by importing the Hiwonder library and its relevant modules, including IIC communication, time control, buzzer, ultrasonic sensor, and IMU (Inertial Measurement Unit).

(2) Initialize Objects:

Objects for IIC communication, Tonybot, ultrasonic sensor, buzzer, and IMU are created.

(3) Robot Initialization:

The robot is initialized using the runActionGroup(0, 1) method. After waiting for 6 seconds to ensure the IMU is fully initialized, the buzzer emits a “beep” sound to confirm the initialization is complete.

import Hiwonder
import Hiwonder_IIC
import time
from time import ticks_ms, sleep_ms

iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)
buzzer = Hiwonder.Buzzer()
imu = Hiwonder_IIC.MPU()

# 初始化机器人(Initialize the robot)
tony.runActionGroup(0, 1)
time.sleep(6)  # 等待IMU初始化完成(Wait for IMU initialization to complete)
buzzer.playTone(1500, 100, False)

(4) Variables:

step: Keeps track of the current execution step.

last_tick: Records the timestamp of the last execution, used to control the execution frequency.

count1 and count2: Track the number of forward and backward tumbles, respectively.

# 全局变量(Global variables)
step = 0
last_tick = 0
count1 = 0
count2 = 0

(5) Infinite Loop:

The program enters an infinite loop, continuously calling the tumble() function and then pausing for 50 milliseconds after each execution.

# 主循环(Main loop)
while True:
    tumble()
    sleep_ms(50)

5.2.8 Smart Fan Project

• Assembly

• Project Introduction

This section utilizes a glowy ultrasonic sensor to measure the distance to obstacles. Based on the detected distance, the arm is raised, the fan is activated, and the color of the ultrasonic RGB LED changes accordingly.

• Program Implementation Process

• Sensor Introduction

(1) Glowy Ultrasonic Sensor

This is a light-emitting ultrasonic distance sensor module that communicates via I2C. It reads the distance measured by the ultrasonic sensor using the I2C protocol.

When measuring distance, the module sends out 8 pulses of 40kHz square waves and checks for any returning signals. If a signal is detected, the module outputs a high-level signal, and the duration of this high signal corresponds to the time taken for the ultrasonic waves to travel to the target and back.

(2) Fan Module

The fan module is a sensor that operates without the need for an external motor driver board, utilizing PWM (pulse-width modulation) to control its movement. The direction of the fan’s rotation is managed by PWM pulse waveforms in both directions, while the fan speed is adjustable by modifying the duty cycle of the PWM signal.

For module installation and wiring, please refer to the section 1 in the same folder.

• Program Download

Smart Fan Program

(1) Launch the “Hiwonder Python Editor” software .

(2) Drag the “smart_fan.py” file (located in the same folder as this document) into the editor window.

(3) Click the connection button in the menu bar. Once connected, the icon will turn green, indicating a successful connection.

(4) After the connection is established, click the download button in the menu bar to upload the program to Tonybot. Wait for the completion message to appear in the info box below.

• Achieved Effect

Slowly move the object towards the glowy ultrasonic sensor. If the distance is less than 300mm, the robot will extend its right hand and turn on the fan. If the distance exceeds 300mm, the robot will lower its right hand and turn off the fan.

• Function Extension

You can customize the color of the breathing light. Follow the steps below for detailed instructions:

(1) Locate the section of the main function code that controls the color. When the distance is less than 250, the color transitions from red to blue. Specifically, the red component is calculated as 250 - color_diff, the green component is color_diff, and the blue component is always 0.

(2) You can adjust the ratio of the red and green channels to create different colors. For example, when color_diff is 0, the color will be red (250, 0, 0). When color_diff is 250, the color will be green (0, 250, 0). Adjust the color_diff value to achieve various color combinations.

(3) After making the necessary changes, follow the steps in the “Program Download” section to run the program.

• Program Overview

Smart Fan Program

(1) The Hiwonder, Hiwonder_IIC, and time modules are imported.

(2) The ticks_ms and sleep_ms functions are imported from the time module. ticks_ms retrieves the current time in milliseconds, while sleep_ms pauses the program for a specified duration (in milliseconds).

import Hiwonder
import Hiwonder_IIC
from time import ticks_ms, sleep_ms

(3) The iic object is used to establish an IIC communication channel, enabling communication with other IIC devices later.

(4) The tony object represents the robot, providing control over its various movements and functions.

(5) The sonar object is a sonar device based on IIC communication, used for distance detection.

(6) The fan object is created with parameters 33 and 32, which correspond to the control pins for the fan.

(7) A global variable, last_tick, is defined to store the timestamp.

# 初始化硬件(Initialize hardware)
iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)
fan = Hiwonder.Fan_IO(33,32)

# 全局变量(Global variables)
last_tick = 0

(8) The runActionGroup method of the tony object is called to execute a movement from action group 0.

(9) The attachHead method is called on the tony object to enable the robot’s head servo, allowing for control over the head. After initialization, the head servo is released.

(10) The program prints “start.” to indicate that initialization is complete and the program is ready to run.

# 初始化(Initialization)
tony.runActionGroup(0, 1)  # 立正动作(Perform the standing action (tony stands up))
tony.attachHead()  # 使能舵机(Enable the servo for head movement)
sleep_ms(1000)
tony.detachHead()  # 释放舵机(Release the servo (disables the head movement))
print("start.")

(11) The getDistance function is called to measure the distance. If the distance is less than 300mm, the RGB light is set to green using sonar.setRGB(0, 0, 250, 0).

(12) If the distance exceeds 300mm, the RGB light is set to blue using sonar.setRGB(0, 0, 0, 250), indicating a change in state.

(13) The moveServo method of the tony object is used to move the servo (servo number 16) to position 670 within 500 milliseconds if the distance is less than 300mm. If the distance is greater than 300mm, the servo is moved to position 275 within the same 500-millisecond duration.

(14) The break statement is used to exit the inner while True loop.

# 主循环(Main loop)
while True:
    if getDistance() < 300:  # 若距离<300mm(If the distance is less than 300mm)
        sonar.setRGB(0, 0, 250, 0)  # 绿色(Set sonar LED to green)
        tony.moveServo(16, 670, 500)  # 控制舵机(Control the servo to move to position 670)
        sleep_ms(500)
        
        # 风扇开启(Turn on the fan)
        fan.open()
        
        while True:
            sleep_ms(50)
            if getDistance() > 300:
                sonar.setRGB(0, 0, 0, 250)  # 蓝色(Set sonar LED to blue)
                # 风扇关闭(Turn off the fan)
                fan.close()
                sleep_ms(500)
                tony.moveServo(16, 275, 500)
                sleep_ms(500)
                break
                
    sleep_ms(100)  # 适当延时(Add a small delay before the next loop iteration)

5.2.9 Touch Control Project

• Sensor Installation & Wiring

• Project Introduction

Touch the capacitive pad on the touch sensor, and Tonybot will perform a bowing motion.

• Program Implementation Process

• Sensor Introduction

The touch sensor operates based on capacitive sensing technology, detecting the presence of human skin or metal through its gold-plated contact surface.

When no human or metal is in contact with the surface, the signal output is high. Upon contact with a human or metal, the signal output becomes low. Additionally, the sensor can detect contact through materials such as plastic or paper. The sensitivity of detection is influenced by the size of the contact surface and the thickness of the covering material.

For module installation and wiring, please refer to the section 1 in the same directory.

• Program Download

Touch Control Program

(1) Launch the “Hiwonder Python Editor” software .

(2) Drag the “touch_control.py” file (located in the same folder as this document) into the editor window.

(3) Click the connection button in the menu bar. Once connected, the icon will turn green, indicating a successful connection.

(4) After the connection is established, click the download button in the menu bar to upload the program to Tonybot. Wait for the completion message to appear in the info box below.

• Achieved Effect

Upon powering on Tonybot, touching the capacitive plate on the touch sensor will trigger the robot to perform a bowing motion.

• Program Analysis

Touch Control Program

(1) The Hiwonder, Hiwonder_IIC, and time modules are imported.

(2) The ticks_ms and sleep_ms functions are imported from the time module. The ticks_ms function retrieves the current time in milliseconds, while sleep_ms pauses the program for a specified duration (in milliseconds).

import Hiwonder
import Hiwonder_IIC
from time import ticks_ms, sleep_ms

(3) The iic object is used to establish an IIC communication channel, preparing for communication with other IIC devices.

(4) The tony object represents the robot and allows control over its various actions and functions.

(5) The sonar is an IIC-based sonar device used for distance detection.

(6) The touch instance, initialized with 13 as a parameter, controls the touch sensor.

# 初始化硬件(Initialize hardware)
iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)
touch = Hiwonder.Button(13)  # 触摸传感器,使用IO13引脚(Touch sensor, using IO13 pin)

(7) tony.runActionGroup(0, 1): Calls the runActionGroup method of the tony object to execute action group 0 once.

(8) tony.attachHead(): Enables the robot’s head servo, preparing it for head-related actions.

(9) sleep_ms(1000): Pauses the program for 1000 milliseconds (1 second), ensuring the robot has enough time to complete its standing action.

(10) tony.detachHead(): Releases the robot’s head servo after the initialization process is complete.

(11) print(“start.”): Outputs the string “start.” to indicate that the robot has been initialized and the program is about to enter the main loop.

# 初始化机器人(Initialize robot)
tony.runActionGroup(0, 1)  # 立正动作(Perform the standing action (tony stands up))
tony.attachHead()  # 使能舵机(Enable the servo for head movement)
sleep_ms(1000)
tony.detachHead()  # 释放舵机(Release the servo (disables the head movement))
print("start.")

(12) while True: The program enters an infinite loop and continuously executes the following actions.

(13) if touch.read():: Calls the read method of the touch object to check the status of the touch sensor. if not enter_flag:: Checks whether enter_flag is False, which indicates no action is being executed, allowing a new action to be triggered.

(14) enter_flag = True: Sets enter_flag to True, marking that an action is currently in progress to prevent multiple triggers of the same action.

(15) tony.runActionGroup(10, 1): Calls the runActionGroup method of the tony object to execute action group 10 once, causing the robot to perform the corresponding action.

(16) else:: If no touch event is detected, sets enter_flag to False, allowing the next touch event to trigger a new action.

(17) sleep_ms(50): Pauses the program for 50 milliseconds after each loop iteration to prevent frequent checks of the touch sensor, allowing the robot sufficient time to complete its actions and reducing CPU usage.

# 主循环(Main loop)
while True:
    if touch.read():  # 检测到触摸(Detect touch)
        if not enter_flag:  # 避免重复触发(Prevent repeated triggers)
            enter_flag = True
            tony.runActionGroup(10, 1)  # 执行动作组10(Execute action group 10)
    else:
        enter_flag = False
        
    sleep_ms(50)

• Function Extension

Here’s an example of how to modify the original action group from ‘bow’ to ‘twist’: Locate the code for the action sequence and replace action group 10 (bow) with action group 50 (twist). This will enable the twist action to be triggered by the touch sensor.

            tony.runActionGroup(50, 1)  # 执行动作组10(Execute action group 10)

5.2.10 Temperature and Humidity Detection Project

• Sensor Installation & Wiring

• Project Introduction

Once the mode is activated, Tonybot will monitor the temperature and humidity, and display the data on the matrix module.

• Program Implementation Process

• Sensor Introduction

(1) Temperature and Humidity Sensor

This sensor detects the temperature and humidity of the environment and provides accurate readings.

The AHT10 module converts the air’s temperature and humidity into digital signals, which are then sent to the control board via I2C communication.

(2) LED Dot Matrix Module

The LED dot matrix module uses an 8x8 LED display, known for its high brightness, flicker-free display, and easy wiring. It can display numbers, text, patterns, and more. The module consists of two red 8x8 LED arrays and is controlled by the driver chip TM640B, enabling full control of the dot matrix screen.

For installation and wiring details, refer to the section “01 Sensor Assembly and Wiring” in this directory.

• Program Download

Temperature and Humidity Detection Program

(1) Launch the “Hiwonder Python Editor” software .

(2) Drag the “main.py” file (located in the same folder as this document) into the editor window.

(3) Click the connection button in the menu bar. Once connected, the icon will turn green, indicating a successful connection.

(4) After the connection is established, click the download button in the menu bar to upload the program to Tonybot. Wait for the completion message to appear in the info box below.

• Achieved Effect

Once the temperature and humidity are detected, the dot matrix module will display the numerical values.

• Program Analysis

Temperature and Humidity Detection Program

(1) Import the custom hardware control library to provide the necessary interfaces and functions for hardware operations.

① Import the ticks_ms and sleep_ms functions from the time module. ticks_ms retrieves the current millisecond counter, and sleep_ms is used for millisecond-level delays.

② The nums list stores dot matrix data for numbers 0-9, as well as for a blank character, enabling the display of numbers on the dot matrix module.

③ t_str and b_str store dot matrix data for the Celsius symbol and percentage sign, which are used to display the temperature and humidity units.

import Hiwonder
import Hiwonder_IIC
from time import ticks_ms, sleep_ms

# 数字0-9的点阵数据(以及空白字符)(Digit 0-9 dot matrix data (and blank character))
nums = [
    [0x7E, 0x42, 0x42, 0x7E],  # 0
    [0x00, 0x00, 0x7E, 0x00],  # 1
    [0x7A,0x4A,0x4A,0x4E],  # 2
    [0x4A,0x4A,0x4A,0x7E],  # 3
    [0x0E,0x08,0x08,0x7E],  # 4
    [0x4E,0x4A,0x4A,0x7A],  # 5
    [0x7E,0x4A,0x4A,0x7A],  # 6
    [0x02,0x02,0x02,0x7E],  # 7
    [0x7E,0x4A,0x4A,0x7E],  # 8
    [0x4E,0x4A,0x4A,0x7E],  # 9
    [0x00, 0x00, 0x00, 0x00]   # null
]

t_str = [0x02, 0x38, 0x44, 0x44] # 摄氏度符号(Celsius symbol)
b_str = [0x26, 0x16, 0x68, 0x64] # 百分号(Percentage symbol)

(2) Create the I2C communication object to establish communication with hardware via the I2C protocol.

① Create the robot object tony to control the robot’s movements.

② Create the temperature and humidity sensor object tmp, using I2C communication to read the environmental temperature and humidity data.

③ Create the dot matrix display module object mx, where parameters 2 and 5 are likely configuration settings specific to the module for displaying information.

# 初始化硬件(Initialize hardware)
iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
tmp = Hiwonder_IIC.AHTxx(iic)  # 温湿度传感器(Temperature and humidity sensor)
mx = Hiwonder.Digitaltube(2 , 5)  # 点阵显示模块(Dot matrix display module)

(3) last_time stores a timestamp, which may be used for time-based control and operation scheduling.

① step serves as a state flag, used in the main loop to switch between displaying the temperature and humidity values.

② temperature and humidity store the respective temperature and humidity values retrieved from the sensor.

# 全局变量(Global variables)
last_time = 0
step = 0
temperature = 0
humidity = 0

(4) tony.runActionGroup(0, 1): Executes action group 0 once, likely corresponding to a standing action.

① tony.attachHead(): Activates the robot’s head servo.

② sleep_ms(200): Pauses for 200 milliseconds to allow the head servo to initialize.

③ tony.detachHead(): Deactivates the head servo to prevent unnecessary power consumption or misoperations.

# 初始化机器人(Initialize robot)
tony.runActionGroup(0, 1)  # 立正动作(Standing action)
tony.attachHead()  # 使能舵机(Enable servo)
sleep_ms(200)  # 等待初始化(Wait for initialization)
tony.detachHead()  # 释放舵机(Release servo)

(5) An infinite loop is created to ensure the program runs continuously. Every 2000 milliseconds, the following actions are performed based on the comparison between the current time and last_time:

① mx_data: Create and initialize a 14-byte list to store the dot matrix display data.

② When step == 0: Display the temperature:

③ Call tmp.read_temperature() to read the temperature and store it in temperature.

④ Calculate the tens and ones digits of the temperature, then retrieve the corresponding dot matrix data from the nums list to store in mx_data.

⑤ Store the dot matrix data for the temperature symbol.

⑥ Display the temperature value and set step to 1.

# 主循环(Main loop)
while True:
    if ticks_ms() > last_time:
        last_time = ticks_ms() + 2000
        mx_data = [0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00]
        if step == 0:  # 温度显示(Temperature display)
            temperature = tmp.read_temperature()
            # 显示十位(Display tens place)
            index = temperature // 10
            if index == 0:
                mx_data[1:5] = nums[10]  # 空白显示(Blank display)
            else:
                mx_data[1:5] = nums[index]
            
            # 显示个位(Display ones place)
            index = temperature % 10
            mx_data[6:10] = nums[index]
            
            # 显示温度符号(Display temperature symbol)
            mx_data[11:15] = t_str
            print("Temperature: {}".format(temperature))
            step = 1
            
        else:  # 湿度显示(Humidity display)
            humidity = tmp.read_humidity()
            # 显示十位(Display tens place)
            index = humidity // 10
            if index == 0:
                mx_data[1:5] = nums[10]  # 空白显示(Blank display)
            else:
                mx_data[1:5] = nums[index]
            
            # 显示个位(Display ones place)
            index = humidity % 10
            mx_data[6:10] = nums[index]
            
            # 显示百分比符号(Display percentage symbol)
            mx_data[11:15] = b_str
            print("Humidity: {}".format(humidity))
            step = 0

5.3 Voice Interaction Course

5.3.1 WonderEcho Introduction & Assembly

• Assembly

• Introduction and Working Principle

(1) Introduction

The voice interaction module is built on embedded voice recognition technology. It consists of a voice recognition chip and supporting circuits, designed for seamless communication with the main control chip. This module enables both voice recognition and voice playback functionalities. Developers can easily integrate it into their products to enable voice interaction capabilities.

(2) Working Principle

The module communicates via I2C and functions as a slave device, allowing communication with a master device. When the microphone on the module detects sound, it processes the audio and sends the result back to the master device.

The module operates in a keyword wake-up mode, requiring a specific trigger word to initiate recognition. By default, the wake-up keyword is “Hello Hiwonder.” Once recognition is complete, if further recognition is required, the module must be re-activated by the wake-up keyword after a 15-second delay.

• Interface Description

For detailed specifications and chip diagrams, please refer to the “3. Chip Manual.”

Type-C Interface: Used for powering the module and for firmware programming.

I2C Interface: Used for communication with host devices.

PIN	Function Description
5V	Power input
GND	Power ground
SDA	I2C data cable
SCL	I2C clock cable

• Important Notes

(1) Ensure the voltage does not exceed the rated range during use.

(2) The environment should be quiet, as noise may impact recognition performance.

(3) When speaking the trigger phrase, make sure your voice is clear and loud, and avoid speaking too quickly. It is recommended to maintain a distance of no more than 5 meters from the module.

5.3.2 Distance Ranging & Announcement Project

• Project Introduction

This section utilizes the glowy ultrasonic module to measure the distance to an obstacle. Based on the detected distance, it controls the voice interaction module to announce messages and triggers the robot to perform specific action groups.

• Program Implementation Process

• Sensor Introduction

(1) Glowy Ultrasonic Sensor

This is a light-emitting ultrasonic distance sensor module that communicates via I2C. It reads the distance measured by the ultrasonic sensor using the I2C protocol.

When measuring distance, the module sends out 8 pulses of 40kHz square waves and checks for any returning signals. If a signal is detected, the module outputs a high-level signal, and the duration of this high signal corresponds to the time taken for the ultrasonic waves to travel to the target and back.

(2) Voice Interaction Module

The voice interaction module is built on embedded voice recognition technology. It consists of a voice recognition chip and supporting circuits, designed for seamless communication with the main control chip. This module enables both voice recognition and voice playback functionalities. Developers can easily integrate it into their products to enable voice interaction capabilities.

For instructions on module installation and wiring, please refer to assembly guide saved in the same directory.

• Program Download

Distance Ranging & Announcement Program

(1) Launch the “Hiwonder Python Editor” software .

(2) Drag the “Range_broadcasting.py” file (located in the same folder as this document) into the editor window.

(3) Click the connection button in the menu bar. Once connected, the icon will turn green, indicating a successful connection.

(4) After the connection is established, click the download button in the menu bar to upload the program to Tonybot. Wait for the completion message to appear in the info box below.

• Achieved Effect

After powering on the Tonybot, slowly bring an object close to the glowy ultrasonic sensor. When the distance is less than 200mm, action group 9 will be executed, and the message ‘Hello, welcome’ will be announced.

• Program Analysis

Distance Ranging & Announcement Program

(1) The Hiwonder and Hiwonder_IIC modules are imported, providing the necessary interfaces and functionality for controlling specific hardware components.

The ticks_ms function retrieves the current millisecond timestamp, while sleep_ms pauses the program for a specified duration (in milliseconds). These functions are essential for precise time control and introducing time delays when needed.

import Hiwonder
import Hiwonder_IIC
from time import ticks_ms, sleep_ms

(2) An IIC object is created for I2C communication.

① The tony object is instantiated from the Tonybot class to control the robot’s movements.

② The sonar object is an instance of the I2CSonar class, used for ultrasonic distance measurement and receiving the iic parameter.

③ The asr object is an instance of the asr_module class, used for voice functionality, also communicating via I2C.

④ The command tony.runActionGroup(0, 1) is used to execute action group 0 on the robot.

⑤ The tony.attachHead() function enables the robot’s head, and after waiting 1000 milliseconds, tony.detachHead() is called to release the head.

iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar  = Hiwonder_IIC.I2CSonar(iic)
asr = Hiwonder_IIC.asr_module(iic)

tony.runActionGroup(0 , 1)
tony.attachHead()
sleep_ms(1000)
tony.detachHead()

(3) A global variable have_move is defined and initialized to True to track whether the robot is moving or engaged in some activity.

have_move = True

(4) The distance is measured using sonar.getDistance() * 10 and stored in the distance variable.

① If the distance is less than MIN_DISTANCE but greater than 0, the have_move variable is set to True, the ultrasonic sensor’s RGB light is set to blue, the robot executes action group 9, and the voice module announces “Hello, welcome”.

② If the have_move variable is True in other cases, it is set to False, the ultrasonic sensor’s RGB light transitions to a green gradient, and the robot performs action group 0.

def sonar_task():
    """用户函数(User functionv)"""
    global have_move
    distance = sonar.getDistance() * 10
    if distance < MIN_DISTANCE and distance > 0:  # 如果测得距离小于指定距离(If the measured distance is less than the specified distance)
        global have_move
        have_move = True
        sonar.setRGB(0 , 0, 0, 250)
        tony.runActionGroup(9, 1)  # 运行9号动作组(Run action group 9)
        sleep_ms(500)
        asr.speak(asr.ASR_ANNOUNCER, 0x0F)  # 你好，欢迎光临(Say "Hello, welcome")
        sleep_ms(2000)
    else:
        if have_move:
            have_move = False
            sonar.setRGB(0 , 20, 0, 0)  # 设置发光超声波颜d色为绿色渐变(Set ultrasonic RGB color to a gradient green)
            tony.runActionGroup(0, 1)  # 运行0号动作组(Run action group 0)

(5) The program runs in an infinite loop, continuously executing the following operations:

① The program repeatedly calls the sonar_task function to detect distances and perform corresponding actions.

② After each call to sonar_task, the program pauses for 50 milliseconds to reduce resource usage and allow time for hardware operations.

# 主循环(Main loop)
while True:
    sonar_task()  # 用户函数(User function)
    sleep_ms(50)  # 注意需要给相应的延时(Remember to add a delay)

• Function Extension

(1) To modify the voice announcement from “Hello, welcome” to “There is an obstacle ahead” when the ultrasonic distance is less than 200mm, locate the voice announcement code in the program.

        asr.speak(asr.ASR_ANNOUNCER, 0x0F)  # 你好，欢迎光临(Say "Hello, welcome")

(2) Update the ID of the voice recognition module’s announcement data to “0x05”.

 asr.speak(asr.ASR_ANNOUNCER, 0x05)  # 你好，欢迎光临(Say "Hello, welcome")

(3) Finally, follow the steps in “Program Download” to upload the modified code to the Tonybot.

5.3.3 Human-Robot Interaction Project

• Project Introduction

The voice interaction module recognizes trigger phrases, which then control the Tonybot to execute the corresponding actions.

• Program Implementation Process

• Sensor Introduction

The voice interaction module is built on embedded voice recognition technology. It consists of a voice recognition chip and supporting circuits, designed for seamless communication with the main control chip. This module enables both voice recognition and voice playback functionalities. Developers can easily integrate it into their products to enable voice interaction capabilities.

For instructions on module installation and wiring, please refer to assembly guide saved in the same directory.

• Program Download

Human-Robot Interaction Program

(1) Launch the “Hiwonder Python Editor” software .

(2) Drag the “Communicate.py” file (located in the same folder as this document) into the editor window.

(3) Click the connection button in the menu bar. Once connected, the icon will turn green, indicating a successful connection.

(4) After the connection is established, click the download button in the menu bar to upload the program to Tonybot. Wait for the completion message to appear in the info box below.

• Achieved Effect

Say “Hello Hiwonder” to activate the voice recognition module. Then, use commands like “Hello,” “Introduce yourself,” “Show off,” “Take two steps,” or “Shake your head” to control Tonybot and trigger various actions.

• Program Analysis

Human-Robot Interaction Program

(1) The Hiwonder and Hiwonder_IIC modules are imported, providing the necessary interfaces and functionality for controlling specific hardware components.

ticks_ms retrieves the current millisecond timestamp, while sleep_ms pauses the program for a specified duration (in milliseconds).

import Hiwonder
import Hiwonder_IIC
from time import sleep_ms

(2) tony.runActionGroup(0, 1): Executes action group 0 once to initialize the robot.

① tony.attachHead(): Enables the robot’s head servo.

② tony.moveHeadAngle(90): Rotates the robot’s head to a 90-degree position.

③ tony.detachHead(): Disables the head servo after initialization.

tony.runActionGroup(0, 1)  # 初始化机器人姿态(Initialize robot posture)
tony.attachHead()          # 设定舵机控制io口(Set servo control I/O port)
tony.moveHeadAngle(90)
sleep_ms(200)             # 等待底板初始化完毕(Wait for the base to initialize)
tony.detachHead()

(3) The function calls the asr.getResult() method to retrieve the recognition result from the voice recognition module and stores it in the result variable. If the result is valid (not None or an invalid value), it is printed for debugging or monitoring.

def communicate():
    """语音识别函数(Speech recognition function)"""
    result = asr.getResult()  # 获取语音识别模块数据(Get speech recognition module data)
    if result:
        print("ASR result is:", result)

(4) If the voice recognition result is 0x1A, indicating the “Hello” command, the robot executes action group 10 using tony.runActionGroup(10, 1) once.

① If the result is 0x1D (representing the “Take two steps” command), the robot pauses for 500 milliseconds and then performs the following actions:

② tony.runActionGroup(18, 1): Executes action group 18 once.

③ tony.runActionGroup(21, 3): Executes action group 21 three times for forward movement.

④ tony.waitForStop(5000): Pauses for 5000 milliseconds to allow the action to complete.

⑤ The robot then executes transitional actions by calling tony.runActionGroup(18, 1) and tony.waitForStop(2000).

⑥ Finally, tony.runActionGroup(19, 1) completes the movement.

        if result == 0x1A:  # 你好(Hello)
            tony.runActionGroup(10, 1)
            sleep_ms(1000)
            
        elif result == 0x1B:  # 介绍自己(Introduce yourself)
            tony.runActionGroup(48, 1)
            sleep_ms(4000)
            
        elif result == 0x1C:  # 露一手(Show a trick)
            sleep_ms(500)
            tony.runActionGroup(17, 1)
            sleep_ms(10000)
            
        elif result == 0x1D:  # 走两步(Walk two steps)
            sleep_ms(500)
            tony.runActionGroup(18, 1)
            tony.waitForStop(2000)
            tony.runActionGroup(21, 3)
            tony.waitForStop(5000)
            tony.runActionGroup(18, 1)
            tony.waitForStop(2000)
            tony.runActionGroup(19, 1)

(5) The program enters an infinite loop with while True, continuously calling the communicate() function.

sleep_ms(50): After each communicate() call, the program pauses for 50 milliseconds. This allows hardware to process previous operations, reduces CPU load, and ensures stable and smooth system performance.

# 主循环(Main loop)
while True:
    communicate()  # 语音识别函数(Speech recognition function)
    sleep_ms(50)  # 注意需要给相应的延时(Remember to add a delay)

5.3.4 Voice Control Project

• Project Introduction

This section demonstrates how to control Tonybot’s movements, including moving forward, backward, turning left, and turning right, using the voice recognition module.

• Program Implementation Process

• Sensor Introduction

The voice interaction module is built on embedded voice recognition technology. It consists of a voice recognition chip and supporting circuits, designed for seamless communication with the main control chip. This module enables both voice recognition and voice playback functionalities. Developers can easily integrate it into their products to enable voice interaction capabilities.

For instructions on module installation and wiring, please refer to assembly guide saved in the same directory.

• Program Download

Voice Control Program

(1) Launch the “Hiwonder Python Editor” software .

(2) Drag the “main.py” file (located in the same folder as this document) into the editor window.

(3) Click the connection button in the menu bar. Once connected, the icon will turn green, indicating a successful connection.

(4) After the connection is established, click the download button in the menu bar to upload the program to Tonybot. Wait for the completion message to appear in the info box below.

• Achieved Effect

Activate the voice recognition module by saying “Hello Hiwonder” Then, use commands like “Forward,” “Backward,” “Turn left,” “Turn right,” and others to control Tonybot and execute different actions. The specific control commands are listed in the table below:

Command Phrase	Returned Value	Command Phrase	Returned Value
GO-STRAIGHT	0x01	GO-BACKWARD	0x02
TURN LEFT	0x03	TURN RIGHT	0x04
LEFT-FOOT-SHOOT	0x6E	TWIST	0x71
SIT-UP	0x72	BOW	0x73
SPREAD ARMS	0x74	WAVE HAND	0x75
MARCH-IN-PLACE	0x76	RIGHT-HOOK	0x7A

• Program Analysis

Voice Control Program

(1) The Hiwonder and Hiwonder_IIC modules are imported, providing the necessary interfaces and functions for controlling specific hardware.

The time module imports the ticks_ms and sleep_ms functions. ticks_ms retrieves the current millisecond timestamp, and sleep_ms pauses the program for a specified duration (in milliseconds).

import Hiwonder
import Hiwonder_IIC
from time import sleep_ms

(2) tony.runActionGroup(0, 1): Executes action group 0 once to initialize the robot.

① tony.attachHead(): Connects the robot’s head servo.

② tony.moveHeadAngle(90): Rotates the robot’s head to a 90-degree position.

③ sleep_ms(200): Pauses for 200 milliseconds to allow sufficient time for the robot’s initialization.

④ tony.detachHead(): Disconnects or disables the head servo to prevent misoperation and save energy after the initialization.

tony.runActionGroup(0, 1)  # 初始化机器人姿态(Initialize robot posture)
tony.attachHead()          # 设定舵机控制io口(Set servo control I/O port)
tony.moveHeadAngle(90)
sleep_ms(200)             # 等待底板初始化完毕(Wait for base initialization to complete)
tony.detachHead()

(3) The function calls asr.getResult() to retrieve the recognition result from the voice recognition module, storing it in the result variable. If the result is valid (not None or other invalid values), it is printed for debugging or monitoring purposes.

def ASR_task():
    """语音识别函数(Speech recognition function)"""
    result = asr.getResult()  # 获取语音识别模块数据(Get speech recognition module data)
    if result:
        print("ASR result is:", result)

(4) If the recognition result is 0x1A (indicating the “Hello” voice command), the robot executes action group 10 once using tony.runActionGroup(10, 1). This action typically corresponds to a greeting gesture such as waving or nodding. The program then pauses for 1000 milliseconds to give the robot time to complete the action.

① If the recognition result is 0x1D (indicating the “Take two steps” command), the following sequence of actions is performed:

② tony.runActionGroup(18, 1): Executes action group 18 once.

③ tony.runActionGroup(21, 3): Executes action group 21 three times, corresponding to the “move forward” action.

④ tony.waitForStop(5000): Waits for the action to complete by pausing for 5000 milliseconds.

⑤ Additional transitional actions are executed by calling tony.runActionGroup(18, 1) and tony.waitForStop(2000).

⑥ Finally, tony.runActionGroup(19, 1) completes the movement.

def ASR_task():
    """语音识别函数(Speech recognition function)"""
    result = asr.getResult()  # 获取语音识别模块数据(Get speech recognition module data)
    if result:
        print("ASR result is:", result)
        
        if result == 0x71:  # 扭腰(Twist waist)
            tony.runActionGroup(50, 1)
            
        elif result == 0x72:  # 仰卧起坐(Sit-up)
            tony.runActionGroup(8, 1)
            
        elif result == 0x73:  # 鞠躬(Bow)
            tony.runActionGroup(10, 1)
            
        elif result == 0x74:  # 大鹏展翅(Spread wings)
            tony.runActionGroup(17, 1)
            
        elif result == 0x75:  # 招招手(Wave hand)
            tony.runActionGroup(9, 1)
            
        elif result == 0x76:  # 原地踏步(Marching in place)
            tony.runActionGroup(49, 1)
            
        elif result == 0x01:  # 前进(Move forward)
            tony.runActionGroup(18, 1)
            tony.waitForStop(2000)
            tony.runActionGroup(21, 3)
            tony.waitForStop(5000)
            tony.runActionGroup(18, 1)
            tony.waitForStop(2000)
            tony.runActionGroup(19, 1)
            
        elif result == 0x02:  # 后退(Move backward)
            tony.runActionGroup(18, 1)
            tony.waitForStop(2000)
            tony.runActionGroup(22, 3)
            tony.waitForStop(5000)
            tony.runActionGroup(18, 1)
            tony.waitForStop(2000)
            tony.runActionGroup(19, 1)
            
        elif result == 0x03:  # 左转(Turn left)
            tony.runActionGroup(34, 5)
            
        elif result == 0x04:  # 右转(Turn right)
            tony.runActionGroup(35, 5)
            
        elif result == 0x6E:  # 左脚射门(Left foot kick)
            tony.runActionGroup(55, 1)
            
        elif result == 0x7A:  # 右勾拳(Right hook punch)
            tony.runActionGroup(58, 1)

(5) The program enters an infinite loop with while True, continuously calling the communicate() function to listen for voice commands and respond accordingly.

sleep_ms(50): After each communicate() call, the program pauses for 50 milliseconds. This allows the hardware enough time to process prior operations, reduces CPU usage, and ensures system stability.

# 主循环(Main loop)
while True:
    ASR_task()  # 语音识别函数(Speech recognition function)
    sleep_ms(50)  # 注意需要给相应的延时(Remember to add a delay)

5.3.5 Fall Recovery Project

• Project Introduction

After the robot falls, say the command ‘fall recovery’ to make the robot stand up.

• Program Implementation Process

• Sensor Introduction

The voice interaction module is built on embedded voice recognition technology. It consists of a voice recognition chip and supporting circuits, designed for seamless communication with the main control chip. This module enables both voice recognition and voice playback functionalities. Developers can easily integrate it into their products to enable voice interaction capabilities.

For instructions on module installation and wiring, please refer to assembly guide saved in the same directory.

• Program Download

Fall Recovery Program

(1) Launch the “Hiwonder Python Editor” software .

(2) Drag the “wake_up.py” file into the editor window.

(3) Click the connection button in the menu bar. Once connected, the icon will turn green, indicating a successful connection.

(4) After the connection is established, click the download button in the menu bar to upload the program to Tonybot. Wait for the completion message to appear in the info box below.

• Achieved Effect

If the robot is in a fallen state, activate the voice recognition module by saying ‘Hello Hiwonder,’ and then use the command ‘fall and stand up’ to instruct the robot to stand up.

• Program Overview

Fall Recovery Program

(1) Module Imports

The Hiwonder and Hiwonder_IIC modules are imported to provide necessary interfaces and functions for hardware control. The time module imports the ticks_ms and sleep_ms functions. ticks_ms retrieves the current millisecond count, while sleep_ms pauses the program for a specified duration in milliseconds.

import Hiwonder
import Hiwonder_IIC
from time import sleep_ms

(2) Robot Initialization

The robot’s posture is initialized by adjusting the head angle and waiting for the base and IMU to complete initialization. Initially, action group 0 is executed to configure the head servos and rotate the head to a 90-degree position. A pause is introduced to allow time for initialization. Afterward, the head servos are detached, and the program waits for the IMU to finish initializing. A “beep” sound signifies the successful completion of the initialization process.

tony.runActionGroup(0, 1)  # 初始化机器人姿态(Initialize robot posture)
tony.attachHead()          # 设定舵机控制io口(Set servo control I/O port)
tony.moveHeadAngle(90)
sleep_ms(200)             # 等待底板初始化完毕(Wait for base initialization to complete)
tony.detachHead()
sleep_ms(6000)           # 等待IMU初始化完成(Wait for IMU initialization to complete)

(3) State Management and Recovery Logic

① Global Variable step: A global variable step is used to track the robot’s recovery stages when it falls.

② tumble() Function: The tumble() function is the core of the program, responsible for detecting if the robot has fallen and performing recovery actions based on the current state.

③ global step: This ensures that step can be modified within the function to reflect the robot’s state changes.

④ result = asr.getResult(): Retrieves the current result from the voice recognition module.

⑤ if not tony.isRunning(): This check ensures that the fall detection logic is only executed when the robot is not engaged in other actions, avoiding interference with ongoing movements.

def tumble():
    """跌倒检测函数(Fall detection function)"""
    global step
    result = asr.getResult()  # 获取语音识别模块数据(Get speech recognition module data)
    
    if not tony.isRunning():  # 若无动作进行(If no action is running)
        if step == 0:
            if result == 0x77:  # "跌倒起立(Fall and stand up)"
                angle = imu.read_angle()
                radianX = angle[0]  # X轴角度(X-axis angle)
                # 判断是倒正面还是背面(Determine whether it is face down or back down)
                if radianX < 60 and radianX > -30:
                    step = 1
                elif radianX > 120 or radianX < -140:
                    step = 2
                    
        elif step == 1:
            tony.runActionGroup(102, 1)  # 恢复立正状态(Recover to standing posture)
            step = 0
            
        elif step == 2:
            tony.runActionGroup(101, 1)  # 恢复立正状态(Recover to standing posture)
            step = 0
            
        else:
            step = 0

(5) Continuous Monitoring and Controlwhile

① Infinite Loop (while True): An infinite loop is implemented to continually monitor and control the robot. The tumble() function is called in each iteration to check for falls and initiate recovery actions as needed.

② Delay (sleep_ms(50)): After each call to tumble(), a 50-millisecond delay is introduced. This prevents excessive function calls, reducing CPU usage and ensuring that hardware operations have sufficient time to complete, maintaining system stability and smoothness.

# 主循环(Main loop)
while True:
    tumble()  # 跌倒检测函数(Fall detection function)
    sleep_ms(50)  # 注意需要给相应的延时(Remember to add a delay)

5.4 AI Vision Course

5.4.1 Overview and Installation of ESP32S3 Vision Module

• Assembly

• Overview of the ESP32-S3

The ESP32-S3 Vision Module is a compact camera module that can function as a standalone system. It captures images with its built-in camera, processes the data using the ESP32 microcontroller, and wirelessly transmits it via Wi-Fi. With support for various communication protocols and low-power features, this module is widely used in a range of IoT applications.

(1) Specification

Parameter	Specification
Size	59*43*17.5
Power Supply Range	4.75~5.25V
Operating Temperature	Ambient Temperature
Storage Environment	Ambient Temperature and Pressure
SPI Flash	Supports up to 16MB
RAM	512KB Internal + 16MB External PSRAM
Bluetooth	Suppport Bluetooth 5 and Bluetooth Mesh
WiFi	802.11 b/g/n
Supported Interfaces	UART、I2C
Serial Baud Rate	Default 115200
Image Output Format	JPEG (only supported by OV2640), BMP, Grayscale
Frequency Range	2412~2484MHz
Antenna Type	Onboard PCB Antenna
Transmit Power	802.11b:20.5 dBm(@11Mbps) 802.11g:18 dBm(@54Mbps) 802.11n:17 dBm(@MCS7)
Receive Sensitivity	802.11b:-88.5 dBm(@11Mbps) 802.11g:-76.2 dBm(@54Mbps) 802.11n:-74.2dBm(@MCS7 HT20) -71.2dBm(@MCS7 HT40)
Power Consumption (assuming peripherals are on)	Active (RF On): TX: 285-355mA @ 3.3V RX:95-97mA@3.3V Deep-sleep:8uA@5V Modem-sleep:47.6-107.6mA@5V 240MHZ Light-sleep:240uA@5V
Security	RSA/AES/SHA/HMAC/XTS-AES

Kind Reminder: The product specifications are theoretical values and are for reference only. Please rely on actual performance for precise details.

(2) Interface Description

Interface Name	Interface Description
USB Serial Port	Serial communication, firmware programming
Custom Button	Allows customization of the button event in code
IIC Interface	Secondary development interface, connects with the main controller

• ESP32-S3 Image Coordinate Framework

Here are a few important points to note about the module’s image coordinate system:

(1) The origin is positioned at the center of the screen, not at the top-left corner.

(2) The Y-axis direction is opposite to that of a typical Cartesian coordinate system.

(3) Important Notes:

① Ensure that the module is supplied with at least 5V 2A to avoid potential image distortion.

② The module comes with the default firmware pre-installed, which supports image transmission. No additional downloads are provided. If you need to reflash with different firmware, please proceed carefully.

5.4.2 Image Transmission

Note

The image transmission firmware is pre-installed at the factory. If you flash other firmware for different functionalities, you will need to reflash the image transmission firmware to enable this feature.

In this section, connect to the hotspot created by the ESP32-S3 vision module and visit the provided URL to view the real-time video feed from the camera.

• Program Download

Image Transmission Python Example

(1) ESP32-S3 Vision Module Firmware Download

① Connect the ESP32-S3 vision module to your computer using a Type-C data cable.

② Extract the ZIP file “flash_download_tool_3.9.7” located in the same folder, then double-click “flash_download_tool_3.9.7.exe” to open the program.

③ In the opened window, configure it as shown below.

④ In the flashing interface, click to select the firmware file (”02 Image Transmission Firmware & Flash Tool\image_transmit.bin”), then configure the settings according to the image below.

⑤ Choose the correct port number (do not select COM1, as this will cause the flashing to fail).

⑥ Click “ERASE” to erase the existing firmware (this step is essential), then click “START” to begin the flashing process.

⑦ Wait for a completion message to appear.

⑧ Once flashing is complete, disconnect the Type-C data cable and restart the device.

(2) Python Program Download

① Open the “main.py” file located in the “03 Image Transmission Python Example” folder in your Python editor.

② Power on the device and connect it to your computer using a Type-C data cable.

③ In the Python editor, click to establish the connection. Once successful, you will see a confirmation prompt.

④ Click to transfer the program to the Tonybot robot.

⑤ After the download is complete, restart the device to begin running the program.

• Device Connection

(1) Power on the device (before powering on, ensure that the vision module has been installed on the robot, as described in section 5.4.1 Overview and Installation of ESP32S3 Vision Module.

(2) Locate the hotspot generated by the module: HW_ESP32S3CAM and connect to it.

• Image Transmission Implementation

Open a browser (this works on both mobile and PC browsers; we’ll use a PC as an example) and enter “192.168.5.1” in the address bar, then press Enter. On the resulting page, click to access the camera’s live transmission interface.

5.4.3 Color Recognition Project

• Assembly

• Project Introduction

Tonybot, equipped with the ESP32-S3 vision module, can recognize red, green, and blue objects, while also controlling the RGB lights and displaying information on the dot matrix display.

• Program Implementation Process

• Module Introduction

(1) ESP32-S3 Vision Module

The ESP32-S3 Vision Module is a compact camera module that can function as a standalone system. It captures images with its built-in camera, processes the data using the ESP32 microcontroller, and wirelessly transmits it via Wi-Fi. With support for various communication protocols and low-power features, this module is widely used in a range of IoT applications.

To get the detailed installation guide, please refer to ‘5.4.1 Overview and Installation of ESP32S3 Vision Module’.

(2) LED Dot Matrix Module

The LED dot matrix module features a high-brightness display, flicker-free operation, and easy wiring. It is capable of displaying numbers, text, graphics, and other content. The module consists of two 8x8 red LED matrices, controlled by the TM640B driver chip, enabling full control of the display.

For installation and wiring details, please refer to “01 Dot Matrix Display Installation” in this directory.

• Program Download

(1) ESP32-S3 Vision Module Program Download

ESP32S3 Color Recognition Example

① Connect the ESP32-S3 to your computer using a Type-C data cable.

② Open the program file “ESP32S3 Color Recognition Example/ ESP32S3_ColorDetection/ ESP32S3_ColorDetection.ino” located in the same directory as this document.

③ Select the “ESP32S3 Dev Module” as the target board.

④ In the “Tools” menu, configure the ESP32-S3 development board settings as shown in the image below. (This configuration is only required for the first use. It does not need to be repeated for subsequent uses.)

⑤ Click to transfer the code to the ESP32-S3 and wait for the flashing process to complete.

(2) Python Program Download

Color Recognition Python Program

① Open the “04 Color Recognition Python Program\main.py” file from the same directory in your Python editor.

② Power on the device and connect it to your computer using a Type-C data cable.

③ Click in the Python editor to establish the connection. Once the connection is successful, a confirmation prompt will appear.

④ Click to upload the program to the Tonybot robot.

⑤ After the upload is complete, a success prompt will be displayed.

⑥ In the “Devices” list, rename the file “color_recognition.py” to “main.py”. Then, restart the device to execute the program.

• Achieved Effect

(1) When the vision module detects red, the RGB on the ultrasonic module will light up red, and the dot matrix module will display the word “Red” scrolling.

(2) When the vision module detects blue, the RGB on the ultrasonic module will light up blue, and the dot matrix module will display the word “Blue” scrolling.

(3) When the vision module detects green, the RGB on the ultrasonic module will light up green, and the dot matrix module will display the word “Green” scrolling.

(4) If the vision module does not detect any color, both the RGB on the ultrasonic module and the dot matrix module will turn off.

• Program Analysis

(1) Initialization Program

① Initialize the I2C interface, sensors, and vision module hardware interfaces.

# 初始化硬件(Initialize hardware)
iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)
cam = Hiwonder_IIC.ESP32S3Cam(iic)  # ESP32S3Cam通讯对象(ESP32S3Cam communication object)
mx = Hiwonder.Digitaltube(5, 2)   # 点阵显示模块(Dot matrix display module)
sonar.setRGB(0, 0, 0, 0)

② Initialize the robot’s initial posture and set the servo rotation angles.

# 初始化机器人(Initialize robot)
tony.runActionGroup(0, 1)  # 初始化机器人姿态(Initialize robot posture)
tony.attachHead()          # 设定舵机控制io口(Set up servo control IO port)
tony.moveHeadAngle(90)
sleep_ms(200)             # 等待底板初始化完毕(Wait for the base board initialization to complete)
tony.detachHead()         # 失能舵机接口(Disable the servo interface)

sleep_ms(2000)

print("start.")

(2) Main Program

① In the main loop, retrieve the color detection data and determine the recognized color based on the results. If no color is detected, clear the display on the dot matrix module. When red, green, or blue is detected, display the corresponding color name in English on the dot matrix module.

# 主循环(Main loop)
while True:
    # 获取颜色检测数据(Get color detection data)
    res = colorDetect()
    
    if res == 0:  # 无检测到(No color detected)
        mx.clear()
        
    elif res == 1:  # 检测到红色(Red detected)
        display_color("Red")
        
    elif res == 2:  # 检测到绿色(Green detected)
        display_color("Green")
        
    elif res == 3:  # 检测到蓝色(Blue detected)
        display_color("Blue")
        
    sleep_ms(100)  # 注意需要给相应的延时(Note: Proper delay is required)

② The display_color() function updates the content on the dot matrix module. It uses the drawStr() function to display the passed text on the module.

def display_color(text):
    """显示颜色文字(Display color text)"""
    str_len = len(text)
    for i in range(16, -str_len*6, -1):  # 滚动显示(Scrolling display)
        mx.drawStr(0 , i, text)
        sleep_ms(40)      # 滚动速度(Scrolling speed)
    sleep_ms(200)

③ The colorDetect() function handles color detection. It retrieves the detection results for red, green, and blue. The value in res[2] indicates the width of the detected color. If the width is greater than 0, the color is considered detected, and the corresponding color ID is returned.

def colorDetect():
    """颜色检测函数(Color detection function)"""
    res = cam.read_color(1)
    if res is not None:
        if res[2] > 0:
            return 1
    res = cam.read_color(2)
    if res is not None:
        if res[2] > 0:
            return 2
    res = cam.read_color(3)
    if res is not None:
        if res[2] > 0:
            return 3
    return 0

• FAQ

Q: The color detected by the camera is inaccurate or incorrectly recognized.

A: To improve detection accuracy, minimize background noise and complexity. It is recommended to use a solid-colored or simple background.

5.4.4 Color Tracking Project

• Project Introduction

Tonybot, equipped with the ESP32-S3 vision module, tracks a red ball while the robot’s head follows the ball’s movement direction.

• Program Implementation Process

• Module Introduction

The ESP32-S3 Vision Module is a compact camera module that can function as a standalone system. It captures images with its built-in camera, processes the data using the ESP32 microcontroller, and wirelessly transmits it via Wi-Fi. With support for various communication protocols and low-power features, this module is widely used in a range of IoT applications.

To get the detailed installation guide, please refer to 5.4.1 Overview and Installation of ESP32S3 Vision Module.

• Program Download

ESP32S3 Color Recognition Example

(1) ESP32-S3 Vision Module Program Download

① Connect the ESP32-S3 to your computer using a Type-C data cable.

② Open the program file “03 ESP32S3 Color Recognition Example/ ESP32S3_ColorDetection/ ESP32S3_ColorDetection.ino” located in the same directory as this document.

③ Select the “ESP32S3 Dev Module” as the target board.

④ In the “Tools” menu, configure the ESP32-S3 development board settings as shown in the image below. (This configuration is only required for the first use. It does not need to be repeated for subsequent uses.)

⑤ Click to transfer the code to the ESP32-S3 and wait for the flashing process to complete.

(2) Python Program Download

Color Follow Python Program

① Open the “03 Color Tracking Python Example/main.py” file from the same directory in your Python editor.

② Power on the device and connect it to your computer using a Type-C data cable.

③ Click in the Python editor to establish the connection. Once the connection is successful, a confirmation prompt will appear.

④ Click to upload the program to the Tonybot robot.

⑤ After the upload is complete, a success prompt will be displayed.

⑥ In the “Devices” list, rename the file “color_follow.py” to “main.py”. Then, restart the device to execute the program.

• Achieved Effect

When a red ball is moved in front of the ESP32-S3 vision module camera, the robot’s head will move in response to the ball’s movement.

• Program Analysis

(1) Initialization Program

① Initialize the I2C interface, sensors, and vision module hardware interfaces.

# 初始化硬件(Initialize hardware)
iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)
cam = Hiwonder_IIC.ESP32S3Cam(iic)  # ESP32S3Cam通讯对象(ESP32S3Cam communication object)
sonar.setRGB(0, 0, 0, 0)

② Initialize the robot’s initial posture and servo angles, then turn off the RGB lights on the ultrasonic module.

# 初始化机器人(Initialize robot)
tony.runActionGroup(0, 1)  # 初始化机器人姿态(Initialize robot posture)
tony.attachHead()          # 设定舵机控制io口(Set up servo control IO port)
tony.moveHeadAngle(90)
sleep_ms(200)             # 等待底板初始化完毕(Wait for base board initialization to complete)
sonar.setRGB(0, 0, 0, 0)  # 关闭LED(Turn off the LED)
sleep_ms(2000)

(2) Main Program

① First, define global variables to set the left and right boundaries for detection and the servo’s rotation step size.

# 全局变量(Global variables)
left = 90        # 左边界阈值(Left boundary threshold)
right = 150      # 右边界阈值(Right boundary threshold)
dev = 4         # 转动角度步长(Rotation angle step)
angle = 90      # 舵机角度(Servo angle)

② In the main program, retrieve the recognition results from the vision module. Use the x-axis coordinate of the top-left corner of the detected face (result[0]) and the width of the face area (result[2]) to calculate the x-coordinate of the face’s center. Based on the center point’s x-coordinate, adjust the servo’s rotation angle. If the center point’s x-coordinate is greater than the “right” boundary, the face is shifted to the right, and the servo should rotate right. If the center point’s x-coordinate is less than the “left” boundary, the face is shifted to the left, and the servo should rotate left.

# 主循环(Main loop)
while True:
    # 获取识别结果(Get recognition result)
    result = cam.read_color(1)
    if result:
        # 计算x中心(Calculate x-center)
        x = result[0] + result[2]//2
        print(x)
        
        # 若偏右(If biased to the right)
        if x > right:
            dev = int((x - right)*0.04)
            angle = max(0, angle - dev)
            
        # 若偏左(If biased to the left)
        elif x < left:
            dev = int((left - x)*0.04)
            angle = min(180, angle + dev)
            
        # 设置舵机角度(Set servo angle)
        tony.moveHeadAngle(angle)
        
    sleep_ms(100)  # 注意需要给相应的延时(Note: Proper delay is required)

• FAQ

Q: The color detected by the camera is inaccurate or incorrectly recognized.

A: To enhance accuracy, reduce background noise and complexity. It is recommended to use a solid-colored or simple background.

5.4.5 Face Recognition Project

• Project Introduction

This section demonstrates how the ESP32-S3 vision module detects faces. When a face is recognized, the robot performs a waving gesture to greet the person.

• Program Implementation Process

• Module Introduction

The ESP32-S3 Vision Module is a compact camera module that can function as a standalone system. It captures images with its built-in camera, processes the data using the ESP32 microcontroller, and wirelessly transmits it via Wi-Fi. With support for various communication protocols and low-power features, this module is widely used in a range of IoT applications.

To get the detailed installation guide, please refer to 5.4.1 Overview and Installation of ESP32S3 Vision Module.

• Program Download

ESP32S3 Face Detection Example

(1) ESP32-S3 Vision Module Program Download

① Connect the ESP32-S3 to your computer using a Type-C data cable.

② Open the program file “ESP32S3 Face Detection Example/ ESP32S3_FaceDetection/ ESP32S3_FaceDetection.ino” located in the same directory as this document.

③ Select the “ESP32S3 Dev Module” as the target board.

④ In the “Tools” menu, configure the ESP32-S3 development board settings as shown in the image below. (This configuration is only required for the first use. It does not need to be repeated for subsequent uses.)

⑤ Click to transfer the code to the ESP32-S3 and wait for the flashing process to complete.

⑥ After the flashing is complete, disconnect the Type-C data cable.

(2) Python Program Download

Face Recognition Python Example

① Open the “03 Face Recognition Python Example/facial_recognition.py” file from the same directory in your Python editor.

② Power on the device and connect it to your computer using a Type-C data cable.

③ Click in the Python editor to establish the connection. Once the connection is successful, a confirmation prompt will appear.

④ Click to upload the program to the Tonybot robot.

⑤ After the upload is complete, a success prompt will be displayed.

⑥ In the “Devices” list, rename the file “facial_recognition.py” to “main.py”. Then, restart the device to execute the program.

• Achieved Effect

Upon detecting a face, the robot will perform a waving gesture sequence to greet.

• Program Analysis

Face Recognition Python Example

(1) Initialization Program

① Initialize the I2C interface, sensors, and vision module hardware interfaces.

# 初始化硬件(Initialize hardware)
iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)
cam = Hiwonder_IIC.ESP32S3Cam(iic)  # ESP32S3Cam通讯对象(ESP32S3Cam communication object)

② Initialize the robot’s initial posture and servo angles, then turn off the RGB lights on the ultrasonic module.

# 初始化机器人(Initialize robot)
tony.runActionGroup(0, 1)  # 初始化机器人姿态(Initialize robot posture)
tony.attachHead()          # 设定舵机控制io口(Set up servo control IO port)
tony.moveHeadAngle(90)
sleep_ms(200)             # 等待底板初始化完毕(Wait for base board initialization to complete)
tony.detachHead()         # 失能舵机接口(Disable servo interface)
sonar.setRGB(0, 0, 0, 0)  # 关闭LED(Turn off the LED)
sleep_ms(2000)

(2) Main Program

In the main program, retrieve the recognition results from the vision module. If the recognition result is valid (not empty), execute the waving gesture.

# 主循环(Main loop)
while True:
    # 获取人脸检测结果(Get face detection result)
    result = cam.read_face()
    
    # 若识别到人脸(If a face is detected)
    if result is not None:
        tony.runActionGroup(9, 1)  # 执行招手动作(Perform a waving action)
        sleep_ms(3000)
        cam.read_face()  # 清除缓存数据(Clear the cached data)
        
    sleep_ms(100)  # 注意需要给相应的延时(Note: Proper delay is required)

5.4.6 Vision Line Following Project

• Project Introduction

Tonybot, equipped with the ESP32-S3 vision module, detects red lines and follows their path. The robot moves accordingly, adjusting its position to stay aligned with the detected lines in its field of view.

• Program Implementation Process

• Module Introduction

The ESP32-S3 Vision Module is a compact camera module that can function as a standalone system. It captures images with its built-in camera, processes the data using the ESP32 microcontroller, and wirelessly transmits it via Wi-Fi. With support for various communication protocols and low-power features, this module is widely used in a range of IoT applications.

To get the detailed installation guide, please refer to ‘5.4.1 Overview and Installation of ESP32S3 Vision Module’.

• Program Download

ESP32S3 Color Recognition Example

(1) ESP32-S3 Vision Module Program Download

① Connect the ESP32-S3 to your computer using a Type-C data cable.

② Open the program file “ESP32S3 Color Recognition Example/ ESP32S3_ColorDetection/ ESP32S3_ColorDetection.ino” located in the same directory as this document.

(2) Select the “ESP32S3 Dev Module” as the target board.

(3) In the “Tools” menu, configure the ESP32-S3 development board settings as shown in the image below. (This configuration is only required for the first use. It does not need to be repeated for subsequent uses.)

④ Click to transfer the code to the ESP32-S3 and wait for the flashing process to complete.

⑤ After the flashing is complete, disconnect the Type-C data cable.

(2) Python Program Download

Vision Line Following Python Example

① Open the “Vision Line Following Python Example/facial_recognition.py” file from the same directory in your Python editor.

② Power on the device and connect it to your computer using a Type-C data cable.

③ Click in the Python editor to establish the connection. Once the connection is successful, a confirmation prompt will appear.

④ Click to upload the program to the Tonybot robot.

⑤ After the upload is complete, a success prompt will be displayed.

⑥ In the “Devices” list, rename the file “visual_patrol.py” to “main.py”. Then, restart the device to execute the program.

• Achieved Effect

The robot’s vision module will conduct line-following movements along red lines.

• Program Analysis

Vision Line Following Python Example

(1) Initialization Program

① Initialize the I2C interface, sensors, and vision module hardware interfaces.

# 初始化硬件(Initialize hardware)
iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)
cam = Hiwonder_IIC.ESP32S3Cam(iic)  # ESP32S3Cam通讯对象(ESP32S3Cam communication object)
sonar.setRGB(0, 0, 0, 0)

② Initialize the robot’s initial posture and servo angles, then turn off the RGB lights on the ultrasonic module.

# 初始化机器人(Initialize robot)
tony.runActionGroup(0, 1)  # 初始化机器人姿态(Initialize robot posture)
tony.attachHead()          # 设定舵机控制io口(Set up servo control IO port)
tony.moveHeadAngle(90)
sleep_ms(200)             # 等待底板初始化完毕(Wait for base board initialization to complete)
sonar.setRGB(0, 0, 0, 0)  # 关闭LED(Turn off the LED)
sleep_ms(2000)

(2) Main Program

① Define the action group ID and set the detection threshold.

# 动作组定义(Action group definitions)
TURN_RIGHT = 66    # 右转动作组(Right turn action group)
TURN_LEFT = 65     # 左转动作组(Left turn action group)
GO_STRAIGHT = 63   # 直走动作(Go straight action)
RETREAT = 22       # 后退动作(Retreat action)
TRANSITION = 18    # 过渡动作（右倾）(Transition action (right tilt))

# 阈值定义(Threshold definitions)
LEFT_MIN = 300//2 - 40   # 左临界值(Left threshold)
RIGHT_MAX = 300//2 + 40  # 右临界值(Right threshold)

② In the main loop, obtain the recognition result of color ID1. If the result is not empty, calculate the x-coordinate of the center point and pass it to the body_follow function for further processing.

# 主循环 (Main loop)
while True:
    # 获取线条位置 (Get line position)
    result = cam.read_color(3)
    if result is not None:
        # 计算x中心 (Calculate the x center)
        x = result[0] + result[2]/2
        body_follow(x)
    sleep_ms(50)  # 注意需要给相应的延时 (Note: Proper delay is required)

③ In the movement function, first check if there is any ongoing action. If no action is being executed, determine the position of the line based on the x-coordinate. If the x-coordinate is within the left and right boundary thresholds, execute the straight movement action group.

def body_follow(x):
    """机体运动函数(Body movement function)"""
    global step
    print(x)
    # 
    if not tony.isRunning():  # 若无动作进行(If no action is running)
        if step == 0:
            # 在中间，直走(In the middle, go straight)
            if LEFT_MIN < x < RIGHT_MAX:
                tony.runActionGroup(GO_STRAIGHT, 1)
                sleep_ms(850)

④ If the x-coordinate of the center point is smaller than the left boundary (or greater than the right boundary), execute the left turn (or right turn) action group. At the same time, set the step variable to 1 (or 2). The step variable will be used for subsequent position checking after the robot completes the turn.

            # 左偏，左转 (Deviating to the left, Turn left)
            elif 0 < x < LEFT_MIN:
                tony.runActionGroup(TURN_LEFT, 2)
                sleep_ms(2700)
                step = 1
            # 右偏(Deviating to the right)
            else:
                tony.runActionGroup(TURN_RIGHT, 2)
                sleep_ms(2850)
                step = 2

⑤ When step=1, recheck the robot’s position. If the line is still to the left, continue executing the left turn action group. Otherwise, execute the straight movement action group to make the robot move forward. The same process applies when step=2.

            # 右偏(Deviating to the right)
            else:
                tony.runActionGroup(TURN_RIGHT, 2)
                sleep_ms(2850)
                step = 2
        elif step == 1:  # 左偏 (Deviating to the left)
            if 0 < x < LEFT_MIN:  # 未调整(Not adjusted)
                tony.runActionGroup(TURN_LEFT, 1)
                sleep_ms(1350)
            else:
                tony.runActionGroup(GO_STRAIGHT, 1)
                sleep_ms(900)
                step = 0
        elif step == 2:  # 右偏 （Deviating to the right）
            if x > RIGHT_MAX:  # 未调整 (Not adjusted)
                tony.runActionGroup(TURN_RIGHT, 1)
                sleep_ms(1450)
            else:
                tony.runActionGroup(GO_STRAIGHT, 1)
                sleep_ms(900)
                step = 0
        else:
            step = 0

• FAQ

Q: The color recognized by the camera is inaccurate or misidentified.

A: To improve accuracy, minimize background clutter by using a monochromatic background or a simple environment.

5.5 IoT (Internet of Things) Course

Tonybot IOT Example

5.5.1 Module Installation & Wiring

5.5.2 Program Download

• Vision Module Program Download

This section provides a step-by-step guide for downloading the ESP32-S3 vision module program, using the face detection program as an example. The process for downloading the color recognition program is identical to that of the face detection program.

(1) Program Download

① Connect the ESP32-S3 to your computer using a Type-C data cable.

② Open the program file “02 Face Detection Porgram/FaceDetection/FaceDetection.ino” located in the same directory as this document.

③ Select the “ESP32S3 Dev Module” as the target board.

④ In the “Tools” menu, configure the ESP32-S3 development board settings as shown in the image below. (This configuration is only required for the first use. It does not need to be repeated for subsequent uses.)

⑤ Click to transfer the code to the ESP32-S3 and wait for the flashing process to complete.

⑥ After the flashing is complete, disconnect the Type-C data cable.

• Tonybot IOT Program Download Instructions

(1) Preparation and Precautions

Before downloading the program, ensure that Tonybot is powered on and connected via a Type-C data cable. Failure to do so may result in the port not being detected.

(2) Program Download

① Open the “03 Tonybot IOT Example/Tonybot_IoT.py” file from the same directory in your Python editor.

② Power on the device and connect it to your computer using a Type-C data cable.

③ Click in the Python editor to establish the connection. Once the connection is successful, a confirmation prompt will appear.

④ Click to upload the program to the Tonybot robot.

⑤ After the upload is complete, a success prompt will be displayed.

⑥ In the “Devices” list, rename the file “Tonybot_IoT.py” to “main.py”. Then, restart the device to execute the program.

• Achieved Effect

After opening the app, you can control Tonybot to perform functions such as face detection alerts, unknown object alerts, and collision alerts for unidentified items. For detailed operational instructions, please refer to section “5.5.3 App Usage Instruction”.

5.5.3 App Usage Instruction

• App Installation

For iOS users: Simply search for “IoT Control” in the App Store and download it.

For Android users: You can find the “IoT Control”, or refer to the appendix to download and install it on your phone.

• App Connection

Note

Before using the app, make sure that Wi-Fi and location services are enabled in your phone’s settings. This section demonstrates using the Android version of the app, but the steps are the same for the iOS version.

(1) Power on the Tonybot, then open the “IoT Control” app on your phone.

(2) Tap in the top-right corner, select “IOT_Tonybot” from the Wi-Fi list, and enter the password “12345678” to connect.

(3) Once connected, the Wi-Fi icon in the top-right corner will become solid.

• Control Overview

The app provides 8 different modes: Face Detection Alert, Unknown Object Alert, Unknown Impact Alert, Remote Warning, Color Detection, Distance Detection, RGB Light Control, and Robot Control.

A detailed description of each function is provided in the table below:

Icon	Function
	When the Tonybot vision module detects a face, the phone will vibrate. (Note: This feature requires the face detection program to be pre-installed on the vision module.)
	When the Tonybot ultrasonic module detects an object, the buzzer will sound.
	When an unknown object causes Tonybot to fall, the buzzer will sound.
	Once activated, the buzzer will continue to sound until Tonybot is upright again.
	When the Tonybot vision module detects a specific color, the corresponding color will be displayed in the circle below. (Note: This feature requires the color detection program to be pre-installed on the vision module.)
	With the Remote Distance Measurement feature enabled, the detected distance will be shown below.
	Controls the color of the RGB light on the Tonybot ultrasonic module.
	Controls Tonybot to execute various action groups.

5.5.4 Face Detection Alert Project

• Project Introduction

In this section, Tonybot utilizes the ESP32-S3 vision module to detect faces and triggers a vibration alert on the phone upon face detection.

• Program Implementation Process

• Module Introduction

(1) ESP32-S3 Vision Module

The ESP32-S3 Vision Module is a compact camera module that can function as a standalone system. It captures images with its built-in camera, processes the data using the ESP32 microcontroller, and wirelessly transmits it via Wi-Fi. With support for various communication protocols and low-power features, this module is widely used in a range of IoT applications.

To get the detailed installation guide, please refer to ‘5.5 IoT (Internet of Things) Course/ 5.5.1 WiFi Module Installation & Wiring’.

• Program Download

(1) Connect the ESP32-S3 to your computer using a Type-C data cable.

(2) Open the program file “03 ESP32S3 Face Detection Example/FaceDetection/FaceDetection.ino” located in the same directory as this document.

(3) Select the “ESP32S3 Dev Module” as the target board.

(4) In the “Tools” menu, configure the ESP32-S3 development board settings as shown in the image below.

(5) Click to transfer the code to the ESP32-S3 and wait for the flashing process to complete.

• Achieved Effect

When Tonybot detects a face, the phone will vibrate as an alert.

• Program Analysis

(1) Importing Library Files

① Import the necessary libraries: The Hiwonder library includes sensor libraries, low voltage alarm functionality, and more. The time library is used for time-related functions, while the Hiwonder_IIC library facilitates I2C communication with the modules.

import Hiwonder
import Hiwonder_IIC
import time
import struct

(2) Hardware Initialization

① Instantiate the hardware components, including enabling I2C communication, creating the Tonybot object, and activating the buzzer.

 
iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)
buzzer = Hiwonder.Buzzer()
imu = Hiwonder_IIC.MPU()
cam = Hiwonder_IIC.ESP32S3Cam(iic)

② Configure the Wi-Fi settings with the name “IOT_Tonybot” and the password “12345678”.

time.sleep_ms(100)
 
iic.writeto(0x69 , "NIOT_Tonybot|||12345678$$$")
time.sleep_ms(1000)

③ Initialize the robot’s posture, enable the head servo to rotate to 90°, and then disable the servo. Finally, initialize the IMU module.

 
tony.runActionGroup(0, 1)   
tony.attachHead()          
tony.moveHeadAngle(90)
time.sleep_ms(200)             
tony.detachHead()         
time.sleep_ms(1000)           
buzzer.playTone(1500, 100, False)

④ Define functions for reading and writing data between the ESP32 and the vision module.

def wifi_send(buf):
  iic.writeto(0x69 , buf)

def wifi_read():
  return iic.readfrom(0x69 , 20)

⑤ Define flags and data structures to control function switches and trigger events.

 
onoff_face = False
onoff_undef_obj = False
onoff_hit = False
onoff_color_detec = False
onoff_distance = False

 
warn_face = False
warn_undef_obj = False
warn_hit = False
color_detec_num = 0
sensor_distance = 0

(3) Face Detection Function Analysis

① First, define and call the function to check the current mode of the vision module, determining whether it is in color recognition or face recognition mode.

def ESP32S3Cam_type():
    
    result = cam.read_color(3)
    time.sleep_ms(100)
    result = cam.read_color(3)
    print(result)
    if result is not None:
      if result[0] != 0x00 and result[0] == result[2]:
        print("type:face")
        cam.read_face()
        return 1
    print("type:color")
    return 2

esp32s3_type = ESP32S3Cam_type()

② In the sensor task, when the vision module is set to face recognition, use the onoff_face flag to determine if the face detection alert function is enabled. When enabled, if a face is detected (i.e., if the face width is greater than 0, result[2] > 0), set the warn_face flag to true. Otherwise, set it to false.

    if esp32s3_type == 1:   
        if onoff_face:
            result = cam.read_face()
            print(result)
            if result[2] > 0:
              warn_face = True
            else:
              warn_face = False
        else:
            warn_face = False

③ Finally, send the warn_face flag to the mobile app.

    if send_step == 0:
      send_step = 1
       
      wifi_send("CMD|1|{}|{}|{}|$".format(
          1 if warn_face else 0,
          1 if warn_undef_obj else 0,
          1 if warn_hit else 0
      ))

5.5.5 Unknown Impact Alert

• Project Introduction

In this section, Tonybot will trigger a vibration alert on the phone when an object is detected by the IMU sensor.

• Program Implementation Process

• IMU Sensor

Tonybot is equipped with an on-board IMU sensor, widely used in applications such as handheld gaming devices, 3D controllers, and portable navigation systems. This sensor integrates a 3-axis MEMS gyroscope, a 3-axis MEMS accelerometer, and a Digital Motion Processor (DMP) for advanced motion detection and processing.

• Program Download

For instructions on downloading the vision module and Tonybot program, please refer to the “5.5.2 Program Download” section.

• Achieved Effect

When Tonybot is knocked over by an unknown object, the phone will vibrate.

• Program Analysis

(1) Importing Library Files

Import the required libraries: The Hiwonder library includes sensor libraries, low voltage alarms, and other functions. The time library is used for time-related operations, while the Hiwonder_IIC library enables I2C communication with the modules.

import Hiwonder
import Hiwonder_IIC
import time
import struct

(2) Hardware Initialization

① Instantiate the hardware components, including enabling I2C communication, creating the Tonybot object, and activating the buzzer.

 
iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)
buzzer = Hiwonder.Buzzer()
imu = Hiwonder_IIC.MPU()
cam = Hiwonder_IIC.ESP32S3Cam(iic)

② Set up the Wi-Fi network with the name “IOT_Tonybot” and the password “12345678”.

time.sleep_ms(100)
 
iic.writeto(0x69 , "NIOT_Tonybot|||12345678$$$")
time.sleep_ms(1000)

③ Initialize the robot’s posture, enable the head servo to rotate to 90°, then disable the servo. Finally, initialize the IMU module.

 
tony.runActionGroup(0, 1)   
tony.attachHead()          
tony.moveHeadAngle(90)
time.sleep_ms(200)             
tony.detachHead()         
time.sleep_ms(1000)           
buzzer.playTone(1500, 100, False)

④ Define functions for reading and writing data between the ESP32 and the vision module.

def wifi_send(buf):
  iic.writeto(0x69 , buf)

def wifi_read():
  return iic.readfrom(0x69 , 20)

⑤ Create flags and data structures to control function switches and trigger specific actions.

 
onoff_face = False
onoff_undef_obj = False
onoff_hit = False
onoff_color_detec = False
onoff_distance = False

 
warn_face = False
warn_undef_obj = False
warn_hit = False
color_detec_num = 0
sensor_distance = 0

• Unknown Object Collision Alert Function Analysis

(1) In the sensor task, use the onoff_hit flag to check if the unknown object collision alert function is enabled. When enabled, retrieve the IMU data. If the roll angle is less than 70° or greater than 110°, it indicates that a collision has occurred. In this case, set the warn_hit flag to true; otherwise, set it to false.

     
    if onoff_hit:
        angle = imu.read_angle()
        roll = angle[0]
        warn_hit = True if roll < 70 or roll > 110 else False
    else:
        warn_hit = False

(2) Finally, transmit the warn_hit flag to the mobile app.

    if send_step == 0:
      send_step = 1
     
      wifi_send("CMD|1|{}|{}|{}|$".format(
          1 if warn_face else 0,
          1 if warn_undef_obj else 0,
          1 if warn_hit else 0
      ))

5.5.6 Unknown Object Alert

• Project Introduction

This section shows how Tonybot uses the ultrasonic sensor to detect objects and sends a vibration alert to the mobile phone.

• Program Implementation Process

• Module Introduction

This is a light-emitting ultrasonic distance sensor module that communicates via I2C. It reads the distance measured by the ultrasonic sensor using the I2C protocol.

When measuring distance, the module sends out 8 pulses of 40kHz square waves and checks for any returning signals. If a signal is detected, the module outputs a high-level signal, and the duration of this high signal corresponds to the time taken for the ultrasonic waves to travel to the target and back.

• Program Download

For instructions on downloading the vision module and Tonybot program, please refer to the “5.5.2 Program Download” section.

• Achieved Effect

When Tonybot detects an object at a distance of less than 15 cm, the phone will vibrate.

• Program Analysis

(1) Importing Library Files

Import the required libraries: The Hiwonder library includes sensor libraries, low voltage alarms, and other functions. The time library is used for time-related operations, while the Hiwonder_IIC library enables I2C communication with the modules.

import Hiwonder
import Hiwonder_IIC
import time
import struct

(2) Hardware Initialization

① Instantiate the hardware components, including enabling I2C communication, creating the Tonybot object, and activating the buzzer.

 
iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)
buzzer = Hiwonder.Buzzer()
imu = Hiwonder_IIC.MPU()
cam = Hiwonder_IIC.ESP32S3Cam(iic)

② Set up the Wi-Fi network with the name “IOT_Tonybot” and the password “12345678”.

time.sleep_ms(100)
 
iic.writeto(0x69 , "NIOT_Tonybot|||12345678$$$")
time.sleep_ms(1000)

③ Initialize the robot’s posture, enable the head servo to rotate to 90°, then disable the servo. Finally, initialize the IMU module.

 
tony.runActionGroup(0, 1)   
tony.attachHead()          
tony.moveHeadAngle(90)
time.sleep_ms(200)             
tony.detachHead()         
time.sleep_ms(1000)           
buzzer.playTone(1500, 100, False)

④ Define functions for reading and writing data between the ESP32 and the vision module.

def wifi_send(buf):
  iic.writeto(0x69 , buf)

def wifi_read():
  return iic.readfrom(0x69 , 20)

⑤ Create flags and data structures to control function switches and trigger specific actions.

 
onoff_face = False
onoff_undef_obj = False
onoff_hit = False
onoff_color_detec = False
onoff_distance = False

 
warn_face = False
warn_undef_obj = False
warn_hit = False
color_detec_num = 0
sensor_distance = 0

• Unknown Object Alert Function Analysis

(1) In the sensor task, retrieve the ultrasonic distance measurement. Use the onoff_undef_obj flag to check if the unknown object detection function is enabled. When enabled, if the obstacle distance is less than 200mm, set the warn_undef_obj flag to true; otherwise, set it to false.

    
    sensor_distance = sonar.getDistance() * 10
    
    
    if onoff_undef_obj:
        warn_undef_obj = True if sensor_distance < 200 else False
    else:
        warn_undef_obj = False

(2) Finally, send the warn_undef_obj flag to the app.

    if send_step == 0:
      send_step = 1
      
      wifi_send("CMD|1|{}|{}|{}|$".format(
          1 if warn_face else 0,
          1 if warn_undef_obj else 0,
          1 if warn_hit else 0
      ))

5.5.7 Color Detection Project

• Project Introduction

This section allows Tonybot, equipped with the ESP32-S3 vision module, to detect red, green, and blue colors and display the corresponding colors on the mobile phone.

• Program Implementation Process

• Module Introduction

(1) ESP32-S3 Vision Module

The ESP32-S3 Vision Module is a compact camera module that can function as a standalone system. It captures images with its built-in camera, processes the data using the ESP32 microcontroller, and wirelessly transmits it via Wi-Fi. With support for various communication protocols and low-power features, this module is widely used in a range of IoT applications.

To get the detailed installation guide, please refer to ‘5.5 IoT (Internet of Things) Course/ 5.5.1 WiFi Module Installation & Wiring’

• Vision Module Program Download

(1) Connect the ESP32-S3 to your computer using a Type-C data cable.

(2) Open the program file “02 ESP32S3 Color Recognition Example/ColorDetection/ColorDetection.ino” located in the same directory as this document.

(3) Select the “ESP32S3 Dev Module” as the target board.

(4) In the “Tools” menu, configure the ESP32-S3 development board settings as shown in the image below.

(5) Click to transfer the code to the ESP32-S3 and wait for the flashing process to complete.

• Achieved Effect

When Tonybot detects red, green, or blue, it sends the data to the app, which then displays the recognized color.

• Program Analysis

(1) Importing Library Files

Import the required libraries: The Hiwonder library includes sensor libraries, low voltage alarms, and other functions. The time library is used for time-related operations, while the Hiwonder_IIC library enables I2C communication with the modules.

import Hiwonder
import Hiwonder_IIC
import time
import struct

(2) Hardware Initialization

① Instantiate the hardware components, including enabling I2C communication, creating the Tonybot object, and activating the buzzer.

 
iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)
buzzer = Hiwonder.Buzzer()
imu = Hiwonder_IIC.MPU()
cam = Hiwonder_IIC.ESP32S3Cam(iic)

② Configure the Wi-Fi network with the name “IOT_Tonybot” and the password “12345678”.

time.sleep_ms(100)
 
iic.writeto(0x69 , "NIOT_Tonybot|||12345678$$$")
time.sleep_ms(1000)

③ Initialize the robot’s posture, enable the head servo to rotate to 90°, then disable the servo. Finally, initialize the IMU module.

 
tony.runActionGroup(0, 1)   
tony.attachHead()          
tony.moveHeadAngle(90)
time.sleep_ms(200)             
tony.detachHead()         
time.sleep_ms(1000)            
buzzer.playTone(1500, 100, False)

④ Define functions for reading and writing data between the ESP32 and the vision module.

def wifi_send(buf):
  iic.writeto(0x69 , buf)

def wifi_read():
  return iic.readfrom(0x69 , 20)

⑤ Create flags and data structures to control function switches and trigger specific actions.

 
onoff_face = False
onoff_undef_obj = False
onoff_hit = False
onoff_color_detec = False
onoff_distance = False

 
warn_face = False
warn_undef_obj = False
warn_hit = False
color_detec_num = 0
sensor_distance = 0

(3) Remote Color Detection Function Analysis

① First, define and call the function to determine the current mode of the vision module, checking whether it is in color recognition or face recognition mode.

def ESP32S3Cam_type():
   
    result = cam.read_color(3)
    time.sleep_ms(100)
    result = cam.read_color(3)
    print(result)
    if result is not None:
      if result[0] != 0x00 and result[0] == result[2]:
        print("type:face")
        cam.read_face()
        return 1
    print("type:color")
    return 2

esp32s3_type = ESP32S3Cam_type()

② In the sensor task, when the vision module is in color recognition mode, use the onoff_color_detec flag to check if the color detection function is enabled. If enabled, retrieve the detected color ID and store it in the color_detec_num variable.

    elif esp32s3_type == 2:  
        if onoff_color_detec:
            color_detec_num = colorDetect()
            print(color_detec_num)
        else:
            color_detec_num = 0

The color detection function calls the read_color function to obtain data for red, green, and blue colors, storing the results in the res variable. If valid color data is obtained, return the corresponding color ID.

def colorDetect():
    
    res = cam.read_color(1)
    if res is not None:
        if res[2] > 0:
            return 3
    res = cam.read_color(2)
    if res is not None:
        if res[2] > 0:
            return 1
    res = cam.read_color(3)
    if res is not None:
        if res[2] > 0:
            return 2
    return 0

③ In the Wi-Fi reception task, receive the data sent by the mobile app and parse the command and data.

def wifi_receive_Task():
    
    global onoff_face, onoff_undef_obj, onoff_hit, onoff_color_detec, onoff_distance,last_receive,receive_data,buzzer_flag
    
    receive_data = wifi_read()
    if last_receive != receive_data:
        last_receive = receive_data
        rec = receive_data.decode('utf-8')
        print(rec)
        if rec.find("CMD") != -1 and rec.find("$"):
          cmd = rec.split('|')[1:]
          print(cmd)

④ When the command is 2, send the detected color ID to the app.

          elif int(cmd[0]) == 2:  
              onoff_color_detec = bool(int(cmd[1]))

5.5.8 Distance Detection

• Project Introduction

This section enables Tonybot to measure the obstacle distance using an ultrasonic sensor and display the distance on the app.

• Program Implementation Process

• Module Introduction

This is a light-emitting ultrasonic distance sensor module that communicates via I2C. It reads the distance measured by the ultrasonic sensor using the I2C protocol.

When measuring distance, the module sends out 8 pulses of 40kHz square waves and checks for any returning signals. If a signal is detected, the module outputs a high-level signal, and the duration of this high signal corresponds to the time taken for the ultrasonic waves to travel to the target and back.

• Program Download

For instructions on downloading the vision module and Tonybot program, please refer to the “5.5.2 Program Download” section.

• Achieved Effect

Tonybot will continuously monitor the distance to objects and display the measurement on the app.

• Program Analysis

(1) Import Library Files

Import the required libraries: The Hiwonder library includes sensor functions, low voltage alarm, and more. The time library is used for time-related operations, and the Hiwonder_IIC library facilitates I2C communication with modules.

import Hiwonder
import Hiwonder_IIC
import time
import struct

(2) Hardware Initialization

① Instantiate the hardware components, including enabling I2C communication, creating the Tonybot object, and activating the buzzer.

# 初始化硬件
iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)
buzzer = Hiwonder.Buzzer()
imu = Hiwonder_IIC.MPU()
cam = Hiwonder_IIC.ESP32S3Cam(iic)

② Set up the Wi-Fi network with the name “IOT_Tonybot” and the password “12345678”.

time.sleep_ms(100)
 
iic.writeto(0x69 , "NIOT_Tonybot|||12345678$$$")
time.sleep_ms(1000)

③ Initialize the robot’s posture, enable the head servo to rotate to 90°, then disable the servo. Finally, initialize the IMU module.

 
tony.runActionGroup(0, 1)  
tony.attachHead()          
tony.moveHeadAngle(90)
time.sleep_ms(200)             
tony.detachHead()         
time.sleep_ms(1000)           
buzzer.playTone(1500, 100, False)

④ Define functions for reading and writing data between the ESP32 and the vision module.

def wifi_send(buf):
  iic.writeto(0x69 , buf)

def wifi_read():
  return iic.readfrom(0x69 , 20)

⑤ Create flags and data structures to control function switches and trigger specific actions.

 
onoff_face = False
onoff_undef_obj = False
onoff_hit = False
onoff_color_detec = False
onoff_distance = False

 
warn_face = False
warn_undef_obj = False
warn_hit = False
color_detec_num = 0
sensor_distance = 0

• Distance Detection Function Analysis

(1) First, define and call the function to determine the current mode of the vision module, whether it is in color recognition or face recognition mode.

def ESP32S3Cam_type():
     
    result = cam.read_color(3)
    time.sleep_ms(100)
    result = cam.read_color(3)
    print(result)
    if result is not None:
      if result[0] != 0x00 and result[0] == result[2]:
        print("type:face")
        cam.read_face()
        return 1
    print("type:color")
    return 2

esp32s3_type = ESP32S3Cam_type()

(2) First, define and call the function to determine the current mode of the vision module, whether it is in color recognition or face recognition mode.

     
    sensor_distance = sonar.getDistance() * 10

(3) When the remote distance switch is enabled, transmit the obstacle distance data to the mobile app.

    elif send_step == 1:
      send_step = 2
      
      if onoff_distance:
          wifi_send("CMD|3|{}|$".format(int(sensor_distance//10)))
    elif send_step == 2:
      send_step = 0

5.5.9 RGB Light Control

• Project Introduction

This section allows you to control the RGB light color of Tonybot’s ultrasonic sensor through the app.

• Program Implementation Process

• Module Introduction

This is a light-emitting ultrasonic distance sensor module that communicates via I2C. It reads the distance measured by the ultrasonic sensor using the I2C protocol.

When measuring distance, the module sends out 8 pulses of 40kHz square waves and checks for any returning signals. If a signal is detected, the module outputs a high-level signal, and the duration of this high signal corresponds to the time taken for the ultrasonic waves to travel to the target and back.

• Program Download

For instructions on downloading the vision module and Tonybot program, please refer to the “5.5.2 Program Download” section.

• Achieved Effect

Tonybot will adjust the RGB light color of the ultrasonic sensor according to the commands sent from the app.

• Program Analysis

(1) Import Library Files

Import the required libraries: The Hiwonder library includes sensor functions, low voltage alarm, and more. The time library is used for time-related operations, and the Hiwonder_IIC library facilitates I2C communication with modules.

import Hiwonder
import Hiwonder_IIC
import time
import struct

(2) Hardware Initialization

① Instantiate the hardware components, including enabling I2C communication, creating the Tonybot object, and activating the buzzer.

# 初始化硬件
iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)
buzzer = Hiwonder.Buzzer()
imu = Hiwonder_IIC.MPU()
cam = Hiwonder_IIC.ESP32S3Cam(iic)

② Set up the Wi-Fi network with the name “IOT_Tonybot” and the password “12345678”.

time.sleep_ms(100)
 
iic.writeto(0x69 , "NIOT_Tonybot|||12345678$$$")
time.sleep_ms(1000)

③ Initialize the robot’s posture, enable the head servo to rotate to 90°, then disable the servo. Finally, initialize the IMU module.

 
tony.runActionGroup(0, 1)   
tony.attachHead()          
tony.moveHeadAngle(90)
time.sleep_ms(200)             
tony.detachHead()         
time.sleep_ms(1000)            
buzzer.playTone(1500, 100, False)

④ Define functions for reading and writing data between the ESP32 and the vision module.

def wifi_send(buf):
  iic.writeto(0x69 , buf)

def wifi_read():
  return iic.readfrom(0x69 , 20)

⑤ Create flags and data structures to control function switches and trigger specific actions.

 
onoff_face = False
onoff_undef_obj = False
onoff_hit = False
onoff_color_detec = False
onoff_distance = False

 
warn_face = False
warn_undef_obj = False
warn_hit = False
color_detec_num = 0
sensor_distance = 0

• Remote RGB Light Control Function Analysis

(1) In the Wi-Fi reception task, receive the data sent from the mobile app and extract the command and data.

def wifi_receive_Task():
     
    global onoff_face, onoff_undef_obj, onoff_hit, onoff_color_detec, onoff_distance,last_receive,receive_data,buzzer_flag
    
    receive_data = wifi_read()
    if last_receive != receive_data:
        last_receive = receive_data
        rec = receive_data.decode('utf-8')
        print(rec)
        if rec.find("CMD") != -1 and rec.find("$"):
          cmd = rec.split('|')[1:]
          print(cmd)

(2) When the command value is 4, use the setRGB function to control the color of the ultrasonic RGB light based on the received color data.

          elif int(cmd[0]) == 4:   
              sonar.setRGB(0, int(cmd[1]), int(cmd[2]), int(cmd[3]))

5.5.10 Remote Warning

• Project Introduction

In this section, you will learn how to use the app to control Tonybot’s buzzer and trigger sound.

• Program Implementation Process

• Module Introduction

Tonybot is equipped with a 5V buzzer. By adjusting the frequency of the PWM signal output, the buzzer can produce sounds of different pitches.

• Program Download

For instructions on downloading the vision module and Tonybot program, please refer to the “5.5.2 Program Download” section.

• Achieved Effect

After enabling the remote warning feature, Tonybot’s buzzer will sound.

• Program Analysis

(1) Import Library Files

Import the required libraries: The Hiwonder library includes sensor functions, low voltage alarm, and more. The time library is used for time-related operations, and the Hiwonder_IIC library facilitates I2C communication with modules.

import Hiwonder
import Hiwonder_IIC
import time
import struct

(2) Hardware Initialization

① Instantiate the hardware components, including enabling I2C communication, creating the Tonybot object, and activating the buzzer.

 
iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)
buzzer = Hiwonder.Buzzer()
imu = Hiwonder_IIC.MPU()
cam = Hiwonder_IIC.ESP32S3Cam(iic)

② Set up the Wi-Fi network with the name “IOT_Tonybot” and the password “12345678”.

time.sleep_ms(100)
 
iic.writeto(0x69 , "NIOT_Tonybot|||12345678$$$")
time.sleep_ms(1000)

③ Initialize the robot’s posture, enable the head servo to rotate to 90°, then disable the servo. Finally, initialize the IMU module.

 
tony.runActionGroup(0, 1)    
tony.attachHead()          
tony.moveHeadAngle(90)
time.sleep_ms(200)             
tony.detachHead()         
time.sleep_ms(1000)           
buzzer.playTone(1500, 100, False)

④ Define functions for reading and writing data between the ESP32 and the vision module.

def wifi_send(buf):
  iic.writeto(0x69 , buf)

def wifi_read():
  return iic.readfrom(0x69 , 20)

⑤ Create flags and data structures to control function switches and trigger specific actions.

 
onoff_face = False
onoff_undef_obj = False
onoff_hit = False
onoff_color_detec = False
onoff_distance = False

 
warn_face = False
warn_undef_obj = False
warn_hit = False
color_detec_num = 0
sensor_distance = 0

• Remote Warning Function Analysis

(1) In the Wi-Fi reception task, receive the data sent from the mobile app and parse the command and data.

def wifi_receive_Task():
    
    global onoff_face, onoff_undef_obj, onoff_hit, onoff_color_detec, onoff_distance,last_receive,receive_data,buzzer_flag
    
    receive_data = wifi_read()
    if last_receive != receive_data:
        last_receive = receive_data
        rec = receive_data.decode('utf-8')
        print(rec)
        if rec.find("CMD") != -1 and rec.find("$"):
          cmd = rec.split('|')[1:]
          print(cmd)

(2) When the command value is 5, activate the buzzer to emit an alert.

          elif int(cmd[0]) == 5:  
              print("55")
              if int(cmd[1]):
                  print("55-1")
                  buzzer_flag = True
              else:
                  print("55-2")
                  buzzer_flag = False

5.5.11 Robot Control

• Project Introduction

In this section, you’ll use the mobile app to control Tonybot and make it perform a series of actions.

• Program Implementation Process

• Program Download

For instructions on downloading the vision module and Tonybot program, please refer to the “5.5.2 Program Download” section.

• Achieved Effect

Tonybot includes 16 built-in action groups, and we can call these action groups to make Tonybot perform the corresponding actions. The specific action groups are shown in the table below:

Action Group Name	Content
left_foot_kick	Left Foot Kick
right_foot_kick	Right Foot Kick
stand_four_legs	Four-Legged Stand
sit_dowm	Sit Down
go_prone	Lie Down
stand_two_legs	Two-Legged Stand
handshake	Handshake
scrape_a_bow	Bow
nodding_motion	Nod
boxing	Boxing
stretch_oneself	Stretch
pee	Urinate
press_up	Push-up
rotation_pitch	Rotate Pitch
rotation_roll	Rotate Roll
normal_attitude	Attention (Stand at Attention)

Program Analysis

(1) Import Library Files

Import the required libraries: The Hiwonder library includes sensor functions, low voltage alarm, and more. The time library is used for time-related operations, and the Hiwonder_IIC library facilitates I2C communication with modules.

import Hiwonder
import Hiwonder_IIC
import time
import struct

(2) Hardware Initialization

① Instantiate the hardware components, including enabling I2C communication, creating the Tonybot object, and activating the buzzer.

 
iic = Hiwonder_IIC.IIC()
tony = Hiwonder.Tonybot()
sonar = Hiwonder_IIC.I2CSonar(iic)
buzzer = Hiwonder.Buzzer()
imu = Hiwonder_IIC.MPU()
cam = Hiwonder_IIC.ESP32S3Cam(iic)

② Set up the Wi-Fi network with the name “IOT_Tonybot” and the password “12345678”.

time.sleep_ms(100)
 
iic.writeto(0x69 , "NIOT_Tonybot|||12345678$$$")
time.sleep_ms(1000)

③ Initialize the robot’s posture, enable the head servo to rotate to 90°, then disable the servo. Finally, initialize the IMU module.

 
tony.runActionGroup(0, 1)   
tony.attachHead()          
tony.moveHeadAngle(90)
time.sleep_ms(200)             
tony.detachHead()         
time.sleep_ms(1000)           
buzzer.playTone(1500, 100, False)

④ Define functions for reading and writing data between the ESP32 and the vision module.

def wifi_send(buf):
  iic.writeto(0x69 , buf)

def wifi_read():
  return iic.readfrom(0x69 , 20)

⑤ Create flags and data structures to control function switches and trigger specific actions.

 
onoff_face = False
onoff_undef_obj = False
onoff_hit = False
onoff_color_detec = False
onoff_distance = False

 
warn_face = False
warn_undef_obj = False
warn_hit = False
color_detec_num = 0
sensor_distance = 0

• Remote Robot Control Function Analysis

(1) In the Wi-Fi reception task, receive the data sent from the mobile app and parse the command and data.

def wifi_receive_Task():
     
    global onoff_face, onoff_undef_obj, onoff_hit, onoff_color_detec, onoff_distance,last_receive,receive_data,buzzer_flag
    
    receive_data = wifi_read()
    if last_receive != receive_data:
        last_receive = receive_data
        rec = receive_data.decode('utf-8')
        print(rec)
        if rec.find("CMD") != -1 and rec.find("$"):
          cmd = rec.split('|')[1:]
          print(cmd)

(2) When the command value is 6, control the robot to execute the corresponding action group. Note that a flag is used to check whether the current action group has finished. A new action group will only be executed if no action group is currently running. Furthermore, if the previous action group involved a crouching motion, the robot must first stand up before executing the new action group.

           elif int(cmd[0]) == 6:  
              action_num = int(cmd[2])
              if not tony.isRunning():
                  if action_num in [16, 30, 77]:   
                      tony.runActionGroup(action_num, 1)
                  else:
                      tony.runActionGroup(19, 1)   
                      time.sleep_ms(600)
                      tony.runActionGroup(action_num, 1)

5.6 Serial Communication Course

5.6.1 Master-Slave Communication Principles

In this section, we’ll explore how Tonybot communicates with various devices (like STM32, 51 microcontrollers, Arduino, Raspberry Pi, etc.) in a master-slave relationship. You’ll learn how Tonybot functions as a slave device for data exchange, and how other devices act as masters to control Tonybot.

Throughout this chapter, Tonybot operates as a slave device, using the UART serial port to exchange information with other devices.

• Master-Slave Relationship

In the master-slave control system, Tonybot functions as a slave, working alongside other devices (like microcontrollers) that act as the master to control the system.

(1) Tonybot’s Functions as a Slave Device

① Receive and Parse Signals from the Master

Tonybot waits for incoming serial signals. Once data is received, it parses the information according to the communication protocol and calls the corresponding function based on the data.

② Execute Actions Triggered by the Received Data

After parsing the signal, Tonybot executes the required action, such as triggering a pre-set action group or reading data from the ultrasonic sensor.

③ Data Packaging and Feedback

When a read command is received, Tonybot calls the appropriate read function, retrieves the data, packages it according to the communication protocol, and sends it back to the master device.

(2) Other Devices as the Master

① Command Packaging and Sending

The master device packages control commands and relevant data into data packets according to the communication protocol and sends them to the device.

② Control Coordination

The master device is responsible for managing the system’s operations, ensuring smooth communication between Tonybot and other devices, avoiding conflicts, and keeping the entire system stable.

③ Data Reception

When the master needs to read Tonybot’s status, it sends the read command and then receives the status data from Tonybot. The master must ensure the data’s integrity and accuracy, parse the data packets, and extract the useful information.

• Hardware Connection

For example, connecting Tonybot to a PC:

Connect the TXD, RXD, and GND pins of the USB-to-serial adapter to the D6, D7, and GND pins on Tonybot using DuPont wires, and then connect the USB adapter to the computer. (Note: The 5V connection is optional, but the GND connection is required.)

Note

• The power supplies for the master device and Tonybot can be different, but they must share a common ground to ensure stable communication levels.

• When wiring, ensure that the TX and RX pins of the UART serial port are crossed (TX to RX, RX to TX); otherwise, communication will not be possible.

• Data Transmission Format

The default data transmission format for the UART serial port on Tonybot is as follows:

Baud Rate	9600
Data Bits	8
Parity Bit	None
Stop Bit	1

• Communication Protocol

The command format sent from the master to Tonybot is as follows: it begins with “CMD”, followed by command parameters separated by “|”, and ends with “$”.

5.6.2 PC Serial Port Control

This document demonstrates how to sequentially control Tonybot’s action groups, head servo, read battery levels, capture ultrasonic data, and retrieve IMU data through the PC serial port.

• Program Logic

Note

To proceed with this section, make sure the Tonybot slave-side program is downloaded first. You can find the “Tonybot Slave-side Program” in the same directory as this document.

(1) After connecting Tonybot to TTL for serial communication, it can be connected to a PC for UART communication. This enables serial control of Tonybot. The default UART data transmission format is as follows:

Baud rate	9600
Data bit	8
Parity bit	None
Stop bit	1

• Communication protocol

Next, we will explain the format of the protocol command packet. The format is as follows: it begins with “CMD”, parameters are separated by “|”, and it ends with “$”.

• Preparation

(1) Hardware Preparation

Connect the TXD, RXD, and GND pins of the USB-to-serial adapter to the IO27, IO19, and GND pins on Tonybot using DuPont wires. Then, plug the USB adapter into the computer. (Note: The 5V connection is optional, but the GND connection is mandatory.)

(2) Software Preparation

① First, navigate to the Serial Port Debugging Tool in the same folder as this section’s content.

② Once you open the tool, make sure to set the baud rate to 9600, choose NONE for the parity bit, set data bits to 8, and stop bits to 1. Don’t forget to check the “Send as ASCII” option. The configuration should look like the one shown below:

• Function Implementation

Control Tonybot by sending protocol commands:

(1) Command Name: Action Group Call

Function Code: 1

Command Data: Action group number, ranging from 0 to 255, corresponding to action groups 0-255.

Description: This command is used to instruct Tonybot to execute a specific action group.

Example: Sending the command “CMD|1|0|$” will make Tonybot perform the “Stand at Attention” action group.

(2) Command Name: Head Servo Control

Function Code: 2

Command Data: Servo angle, ranging from 0 to 180, corresponding to angles from 0° to 180°.

Description: This command controls the movement of Tonybot’s head servo.

Example: Sending the command “CMD|2|90|$” will rotate the head servo to a 90° position.

(3) Command Name: Battery Voltage

Function Code: 3

Command Data: This is a read operation, so no command data is needed.

Description: This command retrieves Tonybot’s battery voltage, in millivolts (mV).

Example: Sending the command “CMD|3|$” will request the battery voltage.

After sending the data, Tonybot will return the requested information in the format:

“CMD|Function Code|data|$”, where “data” represents the retrieved value.

(4) Command Name: Ultrasonic Data

Function Code: 4

Command Data: This is a read operation, so no command data is required.

Description: This command retrieves the distance measured by Tonybot’s ultrasonic sensor.

Example: Sending the command “CMD|4|$” will return the distance detected by the ultrasonic sensor, in millimeters (mm).

After sending the data, Tonybot will return the requested information in the format:

“CMD|Function Code|data|$”, where “data” represents the retrieved distance.

(5) Command Name: IMU Data

Function Code: 5

Command Data: This is a read operation, so no command data is needed.

Description: This command retrieves Tonybot’s tilt angles along the X and Y axes, measured in degrees.

Example: Sending the command “CMD|5|$” will return the IMU data.

After sending the data, Tonybot will return the requested information in the format:

“CMD|Function Code|x|y|$”, where “x” and “y” represent the tilt angles on the X and Y axes.