In the previous article, we conducted a simple analysis of firmware. In this article, we will supplement some knowledge of Vxworks and continue our research into firmware content.
As this involves operating system content, it is recommended that readers have a basic understanding of operating systems before reading this article, or refer to my article “The Art of Windows Debugging” to gain a simple understanding of concepts such as threads, interrupts, and drivers.
This article is an extension of “Introduction to Industrial Control Security (Part 5) – An Initial Exploration of PLC Reverse Engineering.”
What is Vxworks
The Vxworks operating system is a real-time operating system developed by Wind River Systems, Inc. Compared to Linux, Vxworks offers better real-time performance and customizability. Companies can tailor Vxworks to their needs, and the firmware we reverse-engineered is a secondary development done by Schneider on Vxworks.In the fields of national defense and industrialization, Vxworks occupies a significant share of the market. Even the Patriot missile is related to Vxworks, highlighting its importance in the industrial control field. However, due to the lack of any “beginner” books on Vxworks, learning it can be quite challenging. In the “Introduction to Industrial Control Security” series, I will try to introduce the relevant details of this system as simply as possible. However, if you want to delve deeper or conduct secondary development, you can refer to the official manuals.
The Tasks of Vxworks
As an RTOS (real-time operating system), Vxworks has a unique task system and scheduling scheme to ensure its real-time performance. Before further researching the firmware, it is necessary to clarify this part to ensure the smooth progress of subsequent work.
In Vxworks, there are four types of task queues:
-
The active queue contains all tasks, also known as the activity queue. When we run the ‘i’ command in the shell to display all tasks.
-
The tick queue uses the taskDelay function, which is intended to delay the execution of a task, meaning that it temporarily cannot preempt the CPU. These tasks are stored in the tick queue, also known as the timer queue.
-
The ready queue contains tasks that are fully prepared and waiting for the CPU, also known as the ready queue.
-
The work queue is a special circular queue, also known as the kernel delay queue.
In Vxworks, there is a function called usrAppInit (depending on a certain macro, we will assume it exists by default), which is essentially equivalent to what we commonly understand as the main function. We can “do whatever we want” here, but as a multitasking system, it is impossible to have just one main function and one main task, so several functions are needed to perform task-related operations.
Vxworks tasks have 256 priority levels (0~255), with 0 being the highest priority. For application layer programs, priorities generally range from 100 to 250, while driver programs use priorities from 51 to 99.Similar to other operating systems, each task is represented by a data structure known as TCB (Thread Control Block), which we introduced in detail in the previous article “The Art of Windows Debugging”. Although the operating systems differ, the design philosophy is consistent.
It is important to note that in Vxworks 5.x, there is no strict distinction between kernel mode and user mode (using a global variable called KernelState to indicate whether it is in kernel mode, but the stack is still a stack, and there is no essential distinction). This means that TCB is still exposed to user visibility, so if there is a stack overflow, it can be very fatal. In version 6.x, Vxworks officially distinguishes between user and kernel modes, greatly improving security issues.
int taskSpawn(
char *name,
int priority,
int options,
int stackSize,
FUNCPTR entryPtr,
int agr1,
int agr2,
...
int agr10
)
This function is used to create a new task, returning the task’s “identity card”, which is also a memory address pointing to the task’s TCB, also known as tid.
-
name: the name of the executing task.
-
priority: the priority, which is one of the 256 mentioned above.
-
options: controls certain behaviors of the task, for example, VX_DSP_TASK means that the DSP processor will be used to support this task.
-
stackSize: the size of the stack that the task will use.
-
entryPtr: a pointer to the function that the task will execute, commonly referred to as the entry function.
-
args: parameters required by the entry function. If there are more than 10, you can use pointers to transfer structures or arrays.
int taskCreate(
...)
// This function's parameters are identical to taskSpawn, and it returns the same tid, but the task created is not prepared for execution (meaning it has not entered the ready queue) and needs to be awakened using taskActivate.
STATUS taskDelete(int tid)
// This function is used to delete a task, but it is very dangerous and generally not used. For example, if we have a wrench that a task is using, and there are several other tasks waiting to use it, if you suddenly delete it, it would be equivalent to losing both the person and the wrench, leaving the other tasks in an infinite wait.
Of course, there are many more functions related to tasks; here we only briefly explain them, and we will look at them again when we encounter them later.
Vxworks Startup
In the previous article, we used the sysStartType parameter but did not explain it in detail due to space limitations. Here, we will take a look at it.
Simply put, Vxworks has two different startup methods:
-
Bootram startup is similar to the BIOS of our PC. It first has a small operating system, which then boots the actual operating system to run.This small operating system is bootram, which is stored in ROM or Flash. After it runs, it will download Vxworks to RAM via serial or network port to initiate the startup process.
-
ROM startup means that the Vxworks image is directly stored in ROM and can be started directly.
Bootram Startup (Compared to Vxworks Related Functions)
Since ROM startup and bootram are essentially identical in subsequent parts, we will choose the more complex bootram startup for explanation.
When powered on, the system automatically jumps to the bootram bootstrap program stored in ROM or Flash. Interestingly, the naming conventions of bootram and Vxworks are quite similar.One is bootConfig.c, and the other is usrConfig.c, while function names like usrInit and usrRoot are completely identical. Of course, they also have similar functionalities, and the functions mentioned below, usrInit and usrRoot, are all related to bootram unless otherwise specified.
For bootram, it can be divided into bootram.bin and bootram_uncmp.bin based on whether it has been compressed. Since the general idea is the same, we will take bootram.bin as an example for explanation.
-
romInit: initialization work, such as initializing memory registers, initializing registers, initializing stacks (this stack is used by bootram and is not related to Vxworks), disabling interrupts, etc.
-
romStart: copy work, which copies the uncompressed (the uncompressed part is romInit and romStart) part to the lower address of RAM (defined as RAM_LOW_ADRS) and copies the compressed part to the high address of RAM (defined as RAM_HIGH_ADRS) and decompresses it. For cold starts (as mentioned in the previous article, which resets data), it will also clear the data in RAM before jumping to usrInit.
-
usrInit: similar to the usrInit we analyzed previously in Vxworks.
void usrInit(int startType) {
while (trapValue1 != 0x12348765 || trapValue2 != 0x5a5ac3c3) {
;
}
cacheLibInit(0x02, 0x02);
bzero(edata, end - edata);
sysStartType = startType;
intVecBaseSet((FUNCPTR *)((char *)0x0));
excVecInit();
sysHwInit();
usrKernelInit();
cacheEnable(INSTRUCTION_CACHE);
kernelInit((FUNCPTR)usrRoot, (24000), (char *)(end), sysMemTop(), (5000), 0x0);
}
Here we take the firmware’s usrInit out for comparison and learning.
First, it enters a dead loop checking the values of two variables, which are clearly two addresses. This is essentially checking whether the copy work of romStart was successful. Vxworks does not need to check the RAM memory space again, so this step is omitted.
Next, it calls cacheLibInit. As we can see, both usrInit functions have this function. Functions prefixed with xxxLibInit are considered library initialization functions, and here it initializes the cache library function. The parameters are options for the initialization, which do not need to be focused on.
bzero(edata, end – edata) sets a section of memory to zero, which essentially clears the BSS segment.
intVecBaseSet and excVecInit were analyzed in detail in the previous article, which set up the interrupt vector table.
sysHwInit() is used to initialize devices. Intuitively, it performs simple initialization on various peripherals while keeping them “silent.” We know that the CPU responds to peripherals via interrupts, but since we have not fully established the interrupt system (only a simple interrupt vector has been established), if a device generates an interrupt now, it would lead to an awkward situation where there is no interrupt handler, resulting in system errors. Therefore, devices need to remain “silent.”
Next is rootram’s cacheEnable and Vxworks’ usrCacheEnable. Functions prefixed with xxxEnable generally mean “enable.” This is akin to the enable pin in digital circuits; only when enabled can the device be used.
Finally, the most critical function is usrKernelInit. Let’s take a look at Vxworks:
The xxxLibInit we mentioned earlier initializes library functions, followed by qInit (including workQInit), which stands for queue initialization. We have already discussed this in detail above.
void kernelInit(
FUNCPTR rootRtn, /* User starting routine */
unsigned rootMemSize, /* Memory allocated for TCB and initial task stack */
char *pMemPoolStart, /* Starting address of memory pool */
char *pMemPoolEnd, /* Ending address of memory pool */
unsigned intStackSize, /* Interrupt stack size */
int lockOutLevel /* Interrupt level to disable (1-7) */
)
The main function is to create and execute a task while setting up the task’s TCB (Thread Control Block), stack, memory pool, etc. The task created here is usrRoot, and the starting address of the allocated memory pool is (sysMemTop – end)/16, meaning one-sixteenth of the memory space is used for storage, and the interrupt level is set to 0, which disables any form of interrupts.
The usrKernelInit function of bootram is essentially the same, except that kernelInit is placed outside.
-
usrRoot: I wonder if everyone has noticed that from usrInit to usrRoot, it is done through task creation, meaning there is no return value. Furthermore, from this point onwards, the context changes. Before this, we can say we were living in the “Stone Age” where a bunch of functions were merely operating in the “Stone Age”. Now we officially enter the “civilized era”.
The image shows the decompiled usrRoot of Vxworks.
First, we see usrKernelCoreInit, which mainly initializes some functions. The functions prefixed with sem represent semaphores, wd stands for watchdog (which monitors the system for unrecoverable errors and reboots if detected), and msgQ represents message queues (similar to the message content seen in “The Art of Windows Debugging”). The taskHook is related to hooks.
Next, the memInit function initializes the system memory heap, and from now on we can start using malloc and free functions.
Next comes a very important step, the sysClkInit function, which initializes the clock. The clock definitely involves clock interrupt handling (as we mentioned above, functions like sysHwInit have not completed the actual registration of hardware device interrupt handlers, and our clock cannot operate normally yet). We will delve into this.
First is the sysClkConnect function, which takes usrClock as a parameter. This raises suspicion: could usrClock be the interrupt handler we are looking for? Let’s investigate further.
We can see that the usrClock function is simply placed in some memory location, seemingly unrelated to interrupts. Instead, we see sysHwInit2, which must be connected to the earlier sysHwInit. Let’s delve into this function step by step.
Ultimately, we discover intConnect, which registers ppc860Int as the clock interrupt handler. This differs from the Vxworks source code I found, where intConnect registers sysClkInt as the interrupt handler, and sysClkInt then calls usrClock to execute usrClock’s tickAnnounce function for task scheduling. It is unclear whether Schneider’s firmware made specific adjustments or if this is a bug in Ghidra’s analysis.
Returning to usrRoot, the next function is usrIosCoreInit, which initializes the I/O subsystem of Vxworks. We will explain this in detail later, but for now, we will only look at what it initializes.
The parameters of iosInit are three: the maximum number of supported drivers, the maximum number of files that can be opened simultaneously, and a special file where all written content is ignored (similar files exist in Linux).
Continuing deeper, we arrive at usrSerialInit.
This function is somewhat difficult to understand. To make it easier for everyone to view, I have renamed some variables.
First, tyname is initialized to 0, meaning it is empty. Then, the empty tyname is connected to /tyCo/, which is actually tyCo. It then calls an unresolved function that essentially converts ix to a string and appends it to tyname. Additionally, there is a loop for ix < 1, meaning we now have two strings, /tyCo/1 and /tyCo/0, representing the two serial ports of the PLC.
For these two devices, we first use ttyDevCreate to create the device. This might sound confusing; in fact, it is a characteristic of Vxworks. Although the serial port device already exists, the system is unaware of it and requires you to call xxxDevCreate to “register” it. We will cover this in more detail in future articles.After registration, it checks if ix is 0, which means it operates on /tyCo/0, calling the ioctl function to operate on the serial port device.
ioctl is similar to the ioctl in Linux, where basic operations like open, read, and write abstract devices as files (as Linux advocates everything as a file). For unique operations on devices (like ejecting a CD drive), the ioctl function is used.
ioctl(int fd, int function, int arg)
fd is the file descriptor returned after opening the device (often referred to as consoleFd in Vxworks, but do not confuse it with the console…). The function is the operation to be performed, and arg is the parameters needed for the operation, which can be found in ioLib.h as shown in the image below.
Finally, we exit the loop. At this point, consoleFd is the fd for /tyCo/1, and we use ioGlobalStdSet to redirect standard input, output, and error output. Now, functions like printf can be used.
Returning to usrRoot, we will not look further at the remaining initialization functions (those interested can check themselves) but will focus on usrNetworkInit. Why? Because there is a vulnerability known as CVE-2011-4859, and this version of the firmware has not yet been patched. We will analyze this part in detail in the next article.
Finally, usrRoot calls usrAppInit, and we finally arrive at the so-called “main” function.
Similarly, in comparison to the bootram system, the initialization aspects are the same. Once completed, bootram also begins to load Vxworks into memory, entrusting the work to Vxworks, completing the startup process.
Summary
This article introduced the task mechanism of Vxworks and provided a simple analysis of the firmware initialization functions. In the next article, we will study the network initialization of Vxworks and CVE-2011-4859, and begin the reverse engineering of the firmware’s “main” function.
Original Source: Security Guest
