MAKER：Zardam/ Translated by：Endless Fun
Working Principle
1. Add an application to the calculator that will display the output from the Raspberry Pi and the key data sent from the keyboard to it.2. Drive the display on the Raspberry Pi through SPI using fbtft, and some exposed pads of NumWorks contain the SPI bus.
The video effect is as follows:
Displaying Data from SPI
The calculator firmware has completed all the hard work of the lower level (initializing the display) and provides an API for controlling the display.In fact, it is driven by FSMC (Flexible Static Memory Controller), so from the CPU’s perspective, the display can be accessed through two static addresses, one for commands and one for data, with a width of 16 bits.
For this application, the only command that needs to be set is the command for the pixel display area, which has already been implemented in the firmware.Pushing pixels is as simple as writing each pixel sequentially to the data address, and they will display on the standard monitor from left to right and top to bottom.
Therefore, it is only necessary to copy the pixels from the SPI controller to the display address.For the DMA engine, copying pixels is very simple.
The window setup is done before each frame, using the unused MISO pin in the SPI bus as the software chip select.Thus, when MISO goes low, it triggers an interrupt, sets the software chip select on the SPI controller to accept incoming data, and configures the window in the display controller to cover the entire screen.
Setting the window takes about 3µs, so the first word of DMA will be delayed.On the Raspberry Pi side, there is about a 10 µs delay between chip select and the first input byte.
Setting GPIO
The SPI pins must be set to alternate function mode to connect to the SPI controller internally.The MISO pin is kept as a normal GPIO because it is used to trigger interrupts.
GPIOA.MODER()->setMode(5, GPIO::MODER::Mode::AlternateFunction);
GPIOA.AFR()->setAlternateFunction(5, GPIO::AFR::AlternateFunction::AF5);
GPIOA.MODER()->setMode(6, GPIO::MODER::Mode::Input);
GPIOA.MODER()->setMode(7, GPIO::MODER::Mode::AlternateFunction);
GPIOA.AFR()->setAlternateFunction(7, GPIO::AFR::AlternateFunction::AF5);Setting the SPI Controller
The configuration of the SPI controller is very simple.It is set to 16-bit mode, RXONLY, and software chip select.The MISO pin is repurposed for chip select.
SPI1.CR1()->setRXONLY(true);
SPI1.CR1()->setSSI(true); // Software chip select
SPI1.CR1()->setSSM(true); // Software chip select mode
SPI1.CR1()->setDFF(true); // 16 bits
SPI1.CR1()->setSPE(true); // enableSetting the DMA Controller
To make the DMA work, you need to select the correct DMA controller, stream, and channel.Then configure the source address (here it is the SPI1 data register), the destination address (display controller data address), source and destination data width (16 bits), mode (circular), and the number of items to transfer (here it is 1, as we are in circular mode).No need to increment the source/destination address, it is always the same.
DMAEngine.SPAR(DMAStream)->set((uint32_t)SPI1.DR()); // Source
DMAEngine.SM0AR(DMAStream)->set((uint32_t)Ion::Display::Device::DataAddress); // Destination
DMAEngine.SNDTR(DMAStream)->set(1); // Number of items
DMAEngine.SCR(DMAStream)->setCHSEL(3); // SPI Channel
DMAEngine.SCR(DMAStream)->setDIR(DMA::SCR::Direction::PeripheralToMemory);
DMAEngine.SCR(DMAStream)->setMSIZE(DMA::SCR::DataSize::HalfWord);
DMAEngine.SCR(DMAStream)->setPSIZE(DMA::SCR::DataSize::HalfWord);
DMAEngine.SCR(DMAStream)->setCIRC(true); // Circular
DMAEngine.SCR(DMAStream)->setEN(true); // EnableSetting the SPI Controller to Issue DMA Requests
Enabling the RXDMAEN bit in the SPI control register completes this.
SPI1.CR2()->setRXDMAEN(true); // enable DMA requestsSetting Interrupts via the MISO Pin
This part is relatively complex to set up.There are multiple levels of abstraction between the pin and the actual interrupt handler.1. Configure the EXTI (External Interrupt/Event Controller) to trigger an interrupt line in the NVIC (Nested Vector Interrupt Controller).2. Enable the NVIC line and define the corresponding interrupt handler.Don’t forget to acknowledge the interrupt in the handler.
SYSCFG.EXTICR2()->setEXTI(Ion::Rpi::Device::ChipSelectPin, Ion::Rpi::Device::ChipSelectGPIO);
EXTI.RTSR()->set(Ion::Rpi::Device::ChipSelectPin, true);
EXTI.FTSR()->set(Ion::Rpi::Device::ChipSelectPin, true);
NVIC.NVIC_ISER0()->set(23, true);Interrupt HandlerThere are two cases here:When CS goes low, it activates the software chip select for the SPI controller and triggers the configuration of the window in the display controller.
When CS goes high, it disables the software chip select for the SPI controller.Any data received on SPI will be discarded.
void rpi_isr() {
EXTI.PR()->set(Ion::Rpi::Device::ChipSelectPin, true);
if(GPIOA.IDR()->get(6)) {
SPI1.CR1()->setSSI(true);
} else {
Ion::Display::Device::setDrawingArea(KDRect(0,0,320,240), Ion::Display::Device::Orientation::Landscape);
*Ion::Display::Device::CommandAddress = Ion::Display::Device::Command::MemoryWrite;
SPI1.CR1()->setSSI(false);
}
}Note:If there are issues and no error handling is done, the whole chain can get blocked.This mainly happens on the SPI controller.If the DMA reads data too slowly, it will get stuck waiting for an error acknowledgment.
Sending Raspberry Pi Display via SPI Bus
Initially, I planned to use the original fbtft, but after studying the code, I found it couldn’t be used directly because it accesses the display controller directly to optimize pixel pushing (by limiting to the areas of the screen that have changed).But I didn’t want to implement this functionality on the calculator, so I decided to write the code myself.https://github.com/notro/fbtft
Using the concepts and code from fbtft, which is another driver written by Sprite_tm and is also the vfb driver in the kernel, I assembled a “quick and dirty” Linux module that meets my needs:Push the entire framebuffer to the SPI bus.http://spritesmods.com/?art=spitfthttps://github.com/torvalds/linux/blob/ffefb181728f7b97df49ceba18cacfb6c5ee19f2/drivers/video/fbdev/vfb.c
The display is 320×240 pixels, 16bpp, so each frame is 1228800 bits.The maximum SPI frequency for the STM32F412 is 50MHz, but the Raspberry Pi cannot produce it accurately.After testing with 62.5 MHz, it works well, so theoretically the maximum frame rate is (1228800/(62.5×106))−1 ≈ 50 fps.https://www.raspberrypi.org/documentation/hardware/raspberrypi/spi/README.md
Setting Up the Keyboard
This is the most “flexible” part of the application on the calculator (SPI and DMA run in the background).The calculator simply sends the results of the firmware’s keyboard scanning routine (64-bit bitfield) to the Raspberry Pi via UART.On the Raspberry Pi side, the daemon listens on the UART and generates kernel key codes using uinput.https://www.kernel.org/doc/html/v4.12/input/uinput.html
Using custom key mapping on the Linux side.I didn’t take this route as external Bluetooth keyboards can still be used (and I don’t know if different key mappings can be used across multiple keyboards).You can try it out.
The calculator keyboard has only 46 keys, so to map enough keys, the buttons “x, n, t” and “var” are used to switch between standard keys and numbers.Not all keys of a standard keyboard are mapped.This is worth emphasizing.
The mouse relies solely on X.Org’s mouse emulation.Pressing the power button triggers it.https://wiki.archlinux.org/index.php/Xorg/Keyboard_configuration#Enabling_mouse_keys
Here is a large chunk of source codeTo avoid affecting the reading experience, please click the original text at the end for viewing
Powering the Raspberry Pi
The board I initially used was the Raspberry Pi Zero, non-WiFi version.When powered at 2.8 V (the internal regulated voltage on the calculator), it worked well at first, and the board was equipped with an SD card reader and a transistor to control its power.I decided to reuse the SD power board to control the power of the Raspberry Pi.
But later I found that not having WiFi was inconvenient, so I ultimately opted for the WiFi version.Note that the 2.8V power supply cannot run stably, the WiFi version requires a 3V power supply.Disabling the WiFi chip (“dtoverlay=pi3-disable-wifi” in config.txt) allows the Raspberry Pi to work normally at 2.8 V.https://www.cypress.com/file/298756/download
Finally, I chose to power the Raspberry Pi directly with a battery.Since I could no longer use the space occupied by the SD card, I soldered a transistor to the unconnected pins of the SD cart slot and pulled up the resistor in a “free-form” manner.
I used an NTR1P02LT1 and a 10kΩ resistor, but any P-channel MOSFET that can handle at least 100 mA current should work fine.
The voltage levels are fine since all the pins used on the STM32 can withstand 5V voltage.The Raspberry Pi will start when entering the application and shut down when the calculator turns off the power.Thus, it can leave or enter the Raspberry Pi application as needed.
Installing the Calculator
The calculator fits easily into the Raspberry Pi.There are no components where the connector of the Raspberry Pi is located.I fixed it in place with double-sided tape on the HDMI interface and the display interface of the calculator.But it is a bit thick, so the original back cover cannot be used (the vertical protrusion was cut off), but it can be left in place.
Software Configuration of the Raspberry Pi
GitHub repository:https://github.com/zardam/spifbSimply install the kernel header files, compile, install, and auto-load the module.
sudo apt-get install raspberrypi-kernel-headers build-essential
git clone https://github.com/zardam/spifb.git
cd spifb
make -C /lib/modules/$(uname -r)/build M=$PWD
sudo make -C /lib/modules/$(uname -r)/build M=$PWD modules_install
sudo depmod -a/etc/modules
spi-bcm2835
spifb
uinput/boot/config.txt
dtparam=spi=on
# Disable HDMI output, saves some power
hdmi_blanking=2
# Enable the mini uart (/dev/ttyS0 on a PI Zero W)
enable_uart=1
# Disable LED, saves some power
dtparam=act_led_trigger=none
dtparam=act_led_activelow=onThere are two possibilities here:Directly using the framebuffer.This is the simplest method, but hardware acceleration from the Raspberry Pi GPU will not be available.Using fbcp to copy the normal framebuffer (fb0) to the SPI framebuffer (spi1).This copy incurs some CPU overhead, but allows hardware acceleration and can scale the framebuffer since the 320×240 resolution is almost unusable.
Directly using the framebufferConfiguration is the same as using fbtft./boot/cmdline.txt
fbcon=map:10X Server
sudo apt-get install xserver-xorg-video-fbdev/usr/share/X11/xorg.conf.d/99-fbdev.conf
Section "Device"
Identifier "myfb"
Driver "fbdev"
Option "fbdev" "/dev/fb1"
EndSectionfbcpI used a branch that builds with CMake.https://github.com/Oper8or/rpi-fbcp
sudo apt-get install cmake
git clone https://github.com/Oper8or/rpi-fbcp.git
cd rpi-fbcp
mkdir build
cd build
cmake ..
make/boot/config.txt
hdmi_force_hotplug=1
hdmi_cvt=640 480 60 1 0 0 0
hdmi_group=2
hdmi_mode=87/etc/systemd/system/fbcp.service
[Unit]
Description=NumWorks input device
After=systemd-modules-load.service
[Service]
Type=simple
WorkingDirectory=/home/pi/rpi-fbcp/build
ExecStart=/home/pi/rpi-fbcp/build/fbcp
User=root
Group=root
Restart=on-failure
[Install]
WantedBy=multi-user.targetEnable and start the service
sudo systemctl daemon-reload
sudo systemctl enable fbcp
sudo systemctl start fbcpKeyboard
GitHub repository:https://github.com/zardam/uinput-serial-keyboard
git clone https://github.com/zardam/uinput-serial-keyboard
cd uinput-serial-keyboard
gcc uinput.c -o uinputNeed to disable lxkeymap in lxde session configuration (only use GUI tools).Must disable the Linux serial console.In /boot/cmdline.txt, remove:
console=serial0,115200/etc/systemd/system/nwinput.service
[Unit]
Description=NumWorks input device
[Service]
Type=simple
WorkingDirectory=/home/pi/uinput-serial-keyboard/
ExecStart=/home/pi/uinput-serial-keyboard/uinput
User=root
Group=root
Restart=on-failure
[Install]
WantedBy=multi-user.targetThen, enable and start the service:
sudo systemctl daemon-reload
sudo systemctl enable nwinput
sudo systemctl start nwinputCalculator
GitHub repository:https://github.com/zardam/epsilon/tree/rpiOn the computer, after installing the NumWorks SDK:
git clone -b rpi https://github.com/zardam/epsilon.git
cd epsilon
make epsilon_flashThen connect and reset the calculator to refresh the custom firmware.
Completion
Run the NumWorks simulator in the calculator’s browser.
Now, the entire project is complete, and I hope you enjoy this project.
BrachioGraph: Raspberry Pi Drawing Machine
