Exploring the DFRobot ESP32-P4: Creating a Custom ‘Xiao Zhi’ with MicroPython

The previous article “Exploring the DFRobot ESP32-P4 Development Board, Mastering MicroPython” introduced the process of flashing MicroPython firmware onto the ESP32-P4 and achieving most functionalities.This article continues to share how to create a custom ‘Xiao Zhi’ based on MicroPython.

1. Development Board Introduction

The latest release from DFRobot, the FireBeetle 2 ESP32-P4 development board is currently the smallest ESP32-P4 board, yet it integrates a rich set of peripherals:

2. Hardware

1. Core Controller

   • DFRobot ESP32-P4 Development Board (with onboard PDM digital microphone)

   • Power Supply: USB-C 5 V (also serves as debugging serial port)

2. Human-Machine Interaction

   • ST7789 1.69 inch IPS Display (240×280, SPI)

   • Boot Button: Long press for voice input

3. Audio Link

   • Input: Onboard PDM Microphone → ESP32-P4 I²S Interface

   • Output: MAX98357A Class D Amplifier Module (I²S Output) → 1 W 8 Ω Speaker

4. Connectivity

   • Wi-Fi 2.4 GHz (802.11 b/g/n)

   • Reserved UART/I²C header for easy sensor expansion

3. Software

Principle: [Long press Boot button] → Recording → ASR API → LLM API (Text displayed on LCD) → TTS API → Download wav → Decode and Play → Sleep

ASR and LLM use the free model from <span>siliconflow</span>, while TTS uses Baidu’s speech synthesis. The reference code is as follows:

# main.py  ——  Complete loop for recording + speech recognition + SiliconFlow conversation
import network, urequests, ujson, gc, time, st7789_spi, baidu_tts
from easydisplay import EasyDisplay
from machine import Pin, I2S, SPI

# ---------- Global Configuration ----------
WIFI_SSID = ""
WIFI_PASS = ""
API_KEY   = ""
TTS_API_KEY = ""
TTS_SEC_KEY = ""

ASR_URL   = "https://api.siliconflow.cn/v1/audio/transcriptions"
CHAT_URL  = "https://api.siliconflow.cn/v1/chat/completions"
ASR_MODEL = "FunAudioLLM/SenseVoiceSmall"
LLM_MODEL = "Qwen/Qwen3-8B"

# I2S Pins
SCK_PIN = 12
SD_PIN  = 9
boot = Pin(35, Pin.IN, Pin.PULL_UP)   # BOOT button, pressed is 0
led = Pin(3,Pin.OUT)
spi = SPI(2, baudrate=20000000, polarity=0, phase=0, sck=Pin(28), mosi=Pin(29))
dp = st7789_spi.ST7789(width=240, height=280, spi=spi, cs=20, dc=4, res=30, rotate=1,invert=False, rgb=False)
ed = EasyDisplay(dp, "RGB565", font="/text_lite_16px_2312.v3.bmf", show=True, color=0xFFFF, clear=True,auto_wrap=True)

# ---------- Connect to Wi-Fi ----------
def connect_wifi():
    sta = network.WLAN(network.STA_IF)
    sta.active(True)
    sta.connect(WIFI_SSID, WIFI_PASS)
    while not sta.isconnected():
        time.sleep(0.5)
    print("Wi-Fi OK:", sta.ifconfig()[0])
    return sta


def record_audio(sr=8000):
    # Wait for button press
    print("Long press BOOT to start recording...")
    while boot.value() == 1:
        time.sleep_ms(10)

    # Start recording
    pcm = bytearray()
    audio = I2S(0,
                sck=Pin(SCK_PIN),
                sd=Pin(SD_PIN),
                mode=I2S.PDM_RX,
                bits=16,
                format=I2S.MONO,
                rate=sr * 4,
                ibuf=10240)

    # Check button while recording
    chunk = bytearray(1024)
    print("Recording, release BOOT to stop...")
    while boot.value() == 0:           # 0 means still pressed
        n = audio.readinto(chunk)
        pcm.extend(chunk[:n])
    audio.deinit()
    return pcm, sr

# ---------- Construct WAV ----------
def wav_header(data_len, sample_rate):
    hdr = bytearray(44)
    hdr[0:4]   = b'RIFF'
    hdr[4:8]   = (data_len + 36).to_bytes(4, 'little')
    hdr[8:12]  = b'WAVE'
    hdr[12:16] = b'fmt '
    hdr[16:20] = (16).to_bytes(4, 'little')
    hdr[20:22] = (1).to_bytes(2, 'little')         # PCM
    hdr[22:24] = (1).to_bytes(2, 'little')         # mono
    hdr[24:28] = sample_rate.to_bytes(4, 'little')
    hdr[28:32] = (sample_rate * 2).to_bytes(4, 'little')
    hdr[32:34] = (2).to_bytes(2, 'little')         # block align
    hdr[34:36] = (16).to_bytes(2, 'little')        # bits per sample
    hdr[36:40] = b'data'
    hdr[40:44] = data_len.to_bytes(4, 'little')
    return hdr

# ---------- Speech Recognition ----------
def speech_to_text(pcm, sr):
    wav = wav_header(len(pcm), sr) + pcm
    boundary = "----VoiceBoundary"
    body  = b"--" + boundary.encode() + b"\r\n"
    body += b'Content-Disposition: form-data; name="file"; filename="mic.wav"\r\n'
    body += b"Content-Type: audio/wav\r\n\r\n"
    body += wav
    body += b"\r\n--" + boundary.encode() + b"\r\n"
    body += b'Content-Disposition: form-data; name="model"\r\n\r\n'
    body += ASR_MODEL.encode()
    body += b"\r\n--" + boundary.encode() + b"--\r\n"

    headers = {
        "Authorization": "Bearer " + API_KEY,
        "Content-Type": "multipart/form-data; boundary=" + boundary
    }
    print("Recognizing...")
    res = urequests.post(ASR_URL, data=body, headers=headers)
    text = res.json().get("text", "").strip()
    res.close()
    gc.collect()
    return text

# ---------- Chat with AI ----------
def chat_with_ai(text):
    headers = {
        "Authorization": "Bearer " + API_KEY,
        "Content-Type": "application/json"
    }
    payload = {
        "model": LLM_MODEL,
        "messages": [
            {"role": "system", "content": "You are my AI assistant Xiao Zhi, you must respond in Chinese and not exceed 100 words, and you are not allowed to use MD format"},
            {"role": "user", "content": text}
        ],
        "enable_thinking":False,
    }
    print("AI is thinking...")
    start = time.time()
    res = urequests.post(CHAT_URL, data=ujson.dumps(payload).encode(), headers=headers)
    delta = time.time() - start
    if res.status_code == 200:
        reply = res.json()['choices'][0]['message']['content'].replace("\n", "")
        print(f"({delta:.1f}s) AI:", reply)
        ed.text(f"({delta:.1f}s) AI:"+reply, 0, 50)
        baidu_tts.run(
            access=TTS_API_KEY,
            secret=TTS_SEC_KEY,
            text=reply,
        )
    else:
        print("Error:", res.status_code, res.text)
        reply = ""
    res.close()
    gc.collect()
    return reply

# ---------- Main Loop ----------
def main():
    connect_wifi()
    while True:
        pcm, sr = record_audio()
        text = speech_to_text(pcm, sr)
        if not text:
            print("I didn't catch that, please say it again")
            baidu_tts.run(
                access=TTS_API_KEY,
                secret=TTS_SEC_KEY,
                text="I didn't catch that, please say it again",
                out_path='welcome.wav'
            )
            continue
        elif "turn on the light" in text:
            print("You:", text)
            led.on()
            print("AI: The LED light has been turned on")
            ed.text("AI: The LED light has been turned on", 0, 50)
            baidu_tts.run(
                access=TTS_API_KEY,
                secret=TTS_SEC_KEY,
                text="The LED light has been turned on",
            )
        elif "turn off the light" in text:
            print("You:", text)
            led.off()
            print("AI: The LED light has been turned off")
            ed.text("AI: The LED light has been turned off", 0, 50)
            baidu_tts.run(
                access=TTS_API_KEY,
                secret=TTS_SEC_KEY,
                text="The LED light has been turned off",
            )
        else:
            print("You:", text)
            chat_with_ai(text)

if __name__ == "__main__":
    main()

4. Effectiveness

Since MicroPython does not support offline speech recognition and synthesis, this “Xiao Zhi” operates entirely in the cloud: speech recognition, AI inference, and speech synthesis are all completed online, so the response may be slower than local solutions.PS: The speaker volume is relatively low, and children may have shaky hands.

Recommended Reading:

• Practical ESP32S3: Experience AI Chatbot
• ESP32-P4 Development Board MicroPython Firmware Compilation Method
• ESP32-P4 Driving LCD Screen and Touch Based on MicroPython
• ESP32-P4 Displaying Images Based on MicroPython
• ESP32-P4 Drawing Chinese Characters Point by Point Based on MicroPython
• ESP32-P4 Mastering ESP-DL: Implementing Face, Human, and Cat Detection
• ESP32-P4 Mastering ESP-DL: Implementing YOLO v11 Object Detection
• Exploring the DFRobot ESP32-P4 Development Board, Mastering MicroPython
• Exploring the DFRobot ESP32-P4 Development Board, Driving DSI Touch Screen