Building a Local Voice Chatbot with Raspberry Pi and Large Models

In the previous article “Building a Local Voice Chatbot with Raspberry Pi and Large Models”, we introduced a voice chatbot project based on large language models (LLM) running on Raspberry Pi 4, which has garnered a lot of attention.

Running LLM on low-power clients like Raspberry Pi presents unique challenges, and here we discuss some of the issues encountered during development and the corresponding solutions.

Ubuntu Server for Raspberry Pi

The simplest way is to download the pre-installed image, where the default username and password are both ubuntu.

7-inch Small Screen and Its Configuration for Raspberry Pi

First, there’s the UI issue; Raspberry Pi doesn’t have a screen. Even if we use a pre-installed Linux (like Ubuntu Server for Raspberry Pi), we still need a UI for further configuration. The easiest method is to connect it to an HDMI display with a USB keyboard and mouse, but this turns the whole system into something resembling a desktop, losing the original compactness of the Raspberry Pi.

Fortunately, there are many small screens suitable for Raspberry Pi on the market, some are 3.5-inch screens made into boxes, and there are slightly larger 7-inch touch screens. Since some command-line configuration is needed, we chose a 7-inch HDMI touch screen with built-in speakers.

The image below shows that this screen has external HDMI input and a Mini HDMI adapter, connected directly to the screen via a ribbon cable, with two speakers on the upper left and right of the back. The screen itself has screws for mounting the Raspberry Pi motherboard, allowing it to be fixed to the back of the screen.

With this screen, the pre-installed Linux can directly display the output upon startup, but there’s a problem: the default console output font in Linux is set for computer screens, making it appear very small on a 7-inch screen, which is quite hard to read.

We need to adjust the font settings for the Linux console by editing the file /etc/default/console-setup:

sudo vi /etc/default/console-setup

Change FONTSIZE to 16×32:

# CONFIGURATION FILE FOR SETUPCON
# Consult the console-setup(5) manual page.
ACTIVE_CONSOLES="/dev/tty[1-6]"
CHARMAP="UTF-8"
CODESET="Uni2"
FONTFACE="Fixed"
#FONTSIZE="8x16"
FONTSIZE="16x32"
VIDEOMODE=
# The following is an example how to use a braille font
# FONT='lat9w-08.psf.gz brl-8x8.psf'

After saving and rebooting, the screen output looks much more comfortable.

Command Line Configuration of Wifi

Ubuntu Server does not have a graphical interface by default, only a text interface. We need to configure Wifi from the command line interface.

Check if there is a wlan device in the system (wlan0):

$ ls /sys/class/net
eth0  lo  wlan0

Edit the file /etc/netplan/50-cloud-init.yaml and add wlan0 configuration under wifis:

network:    version: 2    wifis:        wlan0:            optional: true            access-points:                "<SSID>":                    password: "<Wifi_Password>"            dhcp4: true

After saving and rebooting, the Raspberry Pi connects to Wifi, allowing it to download software packages from the internet and be accessed via ssh from other computers.

If the Raspberry Pi is also connected to Ethernet, using a static IP can be added in this configuration file with eth0:

network:    ethernets:        eth0:            dhcp4: false            optional: true            addresses: [<ip_addr>/<mask>]    version: 2    wifis:        wlan0:            optional: true            access-points:                "<SSID>":                    password: "<SSID>"            dhcp4: true

Audio Output of Raspberry Pi

The default audio output is the built-in 3.5mm audio jack on the board. If you want to use this, you need a speaker that can connect to a 3.5mm audio input, which also needs to be powered, as the 3.5mm cable does not provide power, making it quite troublesome.

Since the 7-inch small screen has built-in speakers, why not use HDMI as the audio output for Raspberry Pi?

You can switch the default audio output to HDMI with this command:

pactl set-default-sink 0

To adjust the volume, you can use amixer:

amixer sset 'HDMI' 90%

USB microphones do not seem to require special settings; they become the default audio input once connected.

The next article will discuss how to use llamafile packaging and replacing the LLM of the chatbot, as well as some methods for streaming audio output.