What Are the Differences Between Embedded Programming and PC Programming?

In China, few friends in embedded programming have graduated from a formal computer science program; most come from automatic control or electronics-related majors.These individuals have strong practical experience but lack theoretical knowledge;many computer science graduates tend to work on online games, web pages, and other higher-level applications independent of operating systems.They are also reluctant to engage in the embedded industry, as this path is challenging.While they possess strong theoretical knowledge, they often lack knowledge of circuits and other related areas, making it difficult to learn specific knowledge in embedded systems.Being able to view embedded issues from the perspective of PC programming is the first step;learning to apply embedded programming concepts is the second step;combining PC and embedded thinking in practical projects is the third step.Many friends transition from PC programming to embedded programming.In China, few friends in embedded programming have graduated from a formal computer science program; most come from automatic control or electronics-related majors.These individuals have strong practical experience but lack theoretical knowledge;many computer science graduates tend to work on online games, web pages, and other higher-level applications independent of operating systems.They are also reluctant to engage in the embedded industry, as this path is challenging.While they possess strong theoretical knowledge, they often lack knowledge of circuits and other related areas, making it difficult to learn specific knowledge in embedded systems.Although I have not conducted an industry survey, from my observations and the personnel I have recruited, engineers in the embedded industry either lack theoretical knowledge or practical experience.It is rare to find individuals who possess both.The root cause lies in the issues with university education in China.I will not delve into this topic to avoid a heated debate.I would like to list a few examples from my practice.These examples aim to draw attention to certain issues when working on embedded projects.First Issue:A colleague developed a serial port driver under uC/OS-II, and both the driver and interface showed no issues during testing.In the application, a communication program was developed, and the serial port driver provided a function to query the number of characters in the driver buffer:GetRxBuffCharNum().The higher-level application needs to receive a certain number of characters before it can parse the packet.A colleague wrote the following pseudocode:

bExit = FALSE;do {    if (GetRxBuffCharNum() >= 30)           bExit = ReadRxBuff(buff, GetRxBuffCharNum());} while (!bExit);

This code checks if there are more than 30 characters in the current buffer and reads all characters from the buffer until successful.The logic is clear, and the thought process is straightforward.However, this code does not work correctly.If it were on a PC, there would be no issues; it would function normally.But in embedded systems, the outcome is uncertain.The colleague was frustrated and did not understand why.When he came to me for help, I looked at the code and asked him how GetRxBuffCharNum() was implemented.Upon inspection:

unsigned GetRxBuffCharNum(void){    cpu_register reg;    unsigned num;    reg = interrupt_disable();    num = gRxBuffCharNum;    interrupt_enable(reg);    return (num);}

It is evident that there is a global critical section between interrupt_disable() and interrupt_enable() in the loop, ensuring the integrity of gRxBufCharNum.However, due to the frequent enabling and disabling of interrupts in the outer do { } while() loop, the time is very short.In reality, the CPU may not respond to the UART interrupt correctly.Of course, this is related to the UART baud rate, the size of the hardware buffer, and the CPU speed.The baud rate we are using is very high, approximately 3Mbps.The start and stop signals of the UART occupy one bit each.One byte takes 10 cycles to transmit.At a baud rate of 3Mbps, it takes about 3.3us to transmit one byte.How many CPU instructions can be executed in 3.3us?At 100MHz ARM, about 150 instructions can be executed.How long does it take to disable interrupts?Generally, disabling interrupts on ARM requires more than 4 instructions, and enabling requires another 4 instructions.The code for receiving UART interrupts actually consists of more than 20 instructions.Thus, there is a possibility of losing communication data, which manifests as instability in the system.Modifying this code is actually quite simple; the easiest way is to change it from the higher level.That is:

bExit = FALSE;do {    DelayUs(20); // Delay 20us, generally implemented using a busy loop    num = GetRxBuffCharNum();    if (num >= 30)        bExit = ReadRxBuff(buff, num);} while (!bExit);

This allows the CPU time to execute the interrupt code, thus avoiding the issues caused by frequently disabling interrupts, which leads to information loss.In embedded systems, most RTOS applications do not come with serial port drivers.When designing code, one must fully consider the integration of the code with the kernel.This can lead to deep-seated issues in the code.The reason RTOS is called RTOS is due to its rapid response to events;the quick response to events relies on the CPU’s response speed to interrupts.Drivers in systems like Linux are highly integrated with the kernel, running in kernel mode.While RTOS cannot replicate the Linux structure, it can draw some lessons from it.From the above example, it is clear that embedded development requires developers to have a thorough understanding of all aspects of the code.Second Example:Simultaneously driving a 14094 serial-to-parallel chip.The serial signal is simulated using IO because there is no dedicated hardware.A colleague casually wrote a driver, but after debugging for 3 or 4 days, there were still issues.I couldn’t stand it anymore, so I took a look, and the control of the parallel signal was sometimes normal and sometimes not.I examined the code, which was roughly as follows:

for (i = 0; i < 8; i++){    SetData((data >> i) & 0x1);    SetClockHigh();    for (j = 0; j < 5; j++);    SetClockLow();}

This code sends the 8 bits of data from bit0 to bit7 sequentially on each high clock edge.It should work normally.I couldn’t see where the problem was!After thinking carefully and reviewing the 14094 datasheet, I understood.It turns out that the 14094 requires the clock high level to last for 10ns, and the low level must also last for 10ns.This code only implements a delay for the high level and does not implement a delay for the low level.If an interrupt occurs during the low level, this code will work.However, if the CPU does not execute during the low level, it will not work correctly.Thus, it works inconsistently.Modifying it is also quite simple:

for (i = 0; i < 8; i++){    SetData((data >> i) & 0x1);    SetClockHigh();    for (j = 0; j < 5; j++);    SetClockLow();    for (j = 0; j < 5; j++);}

This modification works perfectly.However, this code is not easily portable because if the compiler optimizes it, it may eliminate these two delay loops.If they are lost, the requirement for the high and low levels to last for 10ns cannot be guaranteed, and it will not work correctly.Therefore, truly portable code should implement this loop as a nanosecond-level DelayNs(10);Like Linux, measure how long the nop instruction takes to execute and how many nop instructions are needed for 10ns.Executing a certain number of nop instructions will suffice.Using compiler directives or special keywords to prevent the compiler from optimizing away the delay loop is also necessary.For example, in GCC:

__volatile__ __asm__("nop;\n");

This example clearly shows that writing good code requires a lot of knowledge support.What do you think?Embedded systems often lack operating system support, or even if there is operating system support, the functions provided by the operating system are often very limited.Thus, many codes cannot be as free and unrestricted as PC programming.Today, let’s talk about memory allocation issues, particularly memory fragmentation, which is something everyone is familiar with.However, in embedded systems, memory fragmentation is the most feared issue and the number one killer of system stability.I once worked on a project where there were many malloc and free calls of varying sizes, from over 60 bytes to 64KB.We used an RTOS as support.At that time, I had two choices: one was to use the malloc and free from the C standard library, and the other was to use the fixed memory allocation provided by the operating system.Our system was designed to run stably for over three months.In reality, it crashed after running continuously for about six days.We suspected various issues, but ultimately determined that the problem lay in memory allocation; after a long time of extensive memory allocation, the system’s memory became fragmented and could not be allocated contiguously.Although there was ample space, it could not allocate contiguous memory.When a large space request was made, the system would crash.To meet the original design requirements, we simulated the entire hardware on a PC, ran the embedded code on the PC, and reloaded malloc and free, creating a complex statistical program.This program analyzed the system’s memory behavior.After running for several days, we extracted and analyzed the data; although the requested memory sizes varied, there were still some patterns.We categorized requests under 100 bytes, 512B, 1KB, 2KB, and under 64KB.We counted the number of requests in each category and added a 30% margin.We implemented fixed memory allocation, significantly extending the system’s stable continuous operation time.Embedded systems are like this; they do not fear primitive methods but fear performance that does not meet requirements.Memory overflow issues are even more terrifying in embedded systems than in PC systems! They often occur unnoticed.It is hard to imagine, especially for beginners in C/C++, who are not familiar with pointers and cannot even check for them.Because PC systems have an MMU, when serious memory violations occur, the MMU provides protection, preventing severe disaster consequences.In contrast, embedded systems often lack an MMU, which makes a significant difference; the system code can be corrupted and still run.Only God and the CPU know what is actually running.Let’s take a look at this code:

char  *strcpy(char *dest, const char * src){           assert(dest != NULL && src != NULL);           while (*src != '\0')          {                   *dest++ = *src++;          }          *dest = '\0';         return (dest);}

This code is for copying a string; it works fine on a PC.However, in embedded systems, one must guard against the src not being properly terminated with ‘\0’.If it is not, it will be a tragedy.When will it end? Only God knows.If this code manages to run to completion, it is unlikely that the program will run normally.Because the memory area pointed to by dest will be largely corrupted.To maintain compatibility with the standard C/C++ library, there is really no good solution, so this issue must be left for the programmer to check.Similarly,

memcpy( dest, src, n);

the memory copy function has the same issue; one must guard against passing a negative value for n.This is the number of bytes to copy; a negative value will be forcibly converted to a positive value.This results in a very large positive number, corrupting all memory after dest…Memory pointers in embedded systems must undergo strict checks before use, and memory sizes must be rigorously debugged.Otherwise, tragedy is hard to avoid.For example, a function pointer, even if assigned NULL or 0 in embedded systems, if it is ARM, will not even throw an exception error; it will directly reset because calling this function pointer will cause the code to run from address 0.And 0 is the first instruction location executed after ARM power-up.This is especially true for ARM7.This kind of tragedy is much more tragic than on a PC, where the MMU would throw an undefined instruction error.This would draw the programmer’s attention.In embedded systems, everything is left for the programmer to find.Memory overflow can occur at any unguarded moment; how large is the heap allocated for the entire front and back-end system (or operating system)?How large is the stack?What is the usual system call depth (and maximum), and how much stack does it occupy?It is not enough to just look at the correctness of the program’s functionality; these parameters also need to be counted.Otherwise, if there is an overflow in one place, it can be fatal to the system.Embedded systems require long continuous operation times, with stringent reliability and stability requirements.It takes time to carefully refine these systems.Debugging embedded systems is often very complex, and the available means are not as abundant as in PC programming, leading to significantly higher development costs than PC systems.The main debugging methods for embedded systems are step tracing represented by JTAG, and the printf debugging method.These two debugging methods do not necessarily solve all problems in embedded systems.JTAG requires the debugger to have a debugging device (which can be quite expensive) connected to the target system.Using software like GDB Client to log into the debugging device allows for tracking the running program.To be honest, this method is the ultimate debugging method for embedded systems and is also a relatively good debugging method.However, it still has several shortcomings; when there are too many breakpoints, exceeding the hardware limits, some low-end CPUs do not support more breakpoints, requiring JTAG to use software simulation or software traps (software interrupts or exceptions) to implement breakpoints.The mechanism is quite complex; simply put, 1. long-term debugging is not feasible and is not very stable; 2. it may affect the program’s runtime behavior due to timing issues.After connecting the JTAG system, hardware-implemented breakpoints do not affect the system’s running speed, but software-implemented breakpoints will inevitably sacrifice some performance.Reliability will also be compromised.When there are too many breakpoints, and the system enters a critical area, it may cause breakpoints to become ineffective.This is because implementing global critical areas in embedded systems often requires disabling interrupts, and some CPUs do not have non-maskable interrupts; when the number of breakpoints exceeds a certain number, software breakpoints are used, and software breakpoints must be used in the context of interrupts…Especially when debugging timing issues and high-speed communication code, JTAG is not very helpful.The communication process is often very fast, and communication packets come in quick succession to complete a complete action.If it is high-speed communication, breakpoints cannot allow the program to complete its work.Thus, the printf debugging method is the only option.The printf debugging method is good.However, several issues must also be noted:Embedded systems often do not have screens, and printf output is sent via serial port.There are two working modes for the serial port: polling and interrupt or DMA.Regardless of which method, the debugging output printf can only use polling; never use interrupts or DMA.Whether in front-end or back-end programs, or in operating systems, there are inconvenient times; perhaps printing is needed in a global critical section (interrupts are disabled), perhaps printing is needed in an interrupt (nested interrupts are not allowed), or perhaps printing is needed in some drivers (many cooperative devices are not initialized, and memory allocation and interrupts cannot work well).In these cases, using UART interrupts to output characters is unwise.Thus, debugging output can only use polling methods.Do not fantasize about using some fancy methods; it is unnecessary.In short, it is unreliable!Since debugging is being done, reliable output results are the first requirement.Because of this, printf can also affect the efficiency of the code; the maximum baud rate for the serial port is 115200bps, and the faster the CPU, the more time is wasted waiting for the previous character to finish outputting; this time is entirely consumed by busy waiting.Thus, using printf requires some skills; print only in positions that do not affect critical timing, rather than randomly flooding the output… drowning out the bugs.The above two methods do not solve all problems well; in practice, if the embedded system has one or two LED lights, trying to use IO ports to turn them on and off in special situations can also indicate the program’s status.This method is suitable for debugging interrupts and critical areas.Turning on an LED light takes a very short time, basically one memory read/write command, if the IO port register is uniformly addressed by the CPU.The impact is minimal.When debugging complex timing issues, one can also use spare IO ports to pull them low or high in special situations, and then use a digital oscilloscope or logic analyzer to capture and analyze further.This is particularly significant for analyzing the execution frequency, execution time, optimization effects, etc.For overall performance improvement, it is very meaningful.For simple microcontrollers, manufacturers’ development software often has timing statistics functionality.However, for microcontrollers with cache and MMU, timing statistics are often inaccurate, and using an oscilloscope is usually more accurate.If there is no oscilloscope, the CPU’s internal timer can also be used to achieve timing statistics, which needs to be combined with printf.I had a colleague who debugged a Philips ARM7; since the external RAM of the Philips ARM7 is all static RAM, even if the CPU crashes, as long as power is not cut, the data in SRAM will not be lost. Since SRAM and internal SRAM are uniformly addressed, accessing it is just a read/write instruction, which is very fast.Using this feature, he marked all the modules and points of the program, and when the system did not run correctly, after resetting the ARM7, the first task upon powering up was to retrieve and print the data from before the reset.This method was very clever for debugging ARM7 code.If only SDRAM is available, this method cannot be used because once the system resets, SDRAM loses data without refresh.Everyone knows that the biggest challenge in embedded systems is the simultaneous maturity of hardware and software;when a problem arises, it is unclear whether it is a software issue or a hardware issue.Of course, many problems can be solved through virtual means, but virtual is ultimately virtual.When it comes to actual boards, many issues still arise.Embedded systems, especially low-level technologies, consist of both software (drivers) and hardware.Resolving issues requires knowledge of both parts, which raises the quality requirements for personnel.I have encountered many tricky problems, all of which are complex system issues.1. A system requires continuous operation for 24 hours, and even in the event of a power outage, it must retain the state before the outage.When the power is normal, it must restore the state before the power outage and continue working.In practice, we implemented software to do this, but the actual effect was not as expected.Out of ten thousand power outages, there are always a few dozen that do not function correctly;and there is no way to reproduce the issue, so we can only guess.Because the system loses power, this is also difficult to debug; with JTAG connected, the system is now powered off, and the target board has no power.Thus, it is impossible to debug or step through.The original design idea was to use capacitors in the control circuit to store some energy to continue working after a power outage, saving the state and entering standby mode.Testing the power outage detection signal showed no issues.Later, this problem became a mystery…This system is divided into two modules: the working module and the control module.The control module has capacitors to continue powering, while the working module does not have capacitors.Thus, when a power outage occurs, the entire system does not lose power at the same time.When the control module detects a power outage, the working module has already lost power, so it cannot correctly transmit relevant data back, causing the control module to malfunction.The timing of the two power outages is very close, making it impossible to determine their order.The solution is simple: synchronize the power outage detection module with the working module as the primary reference, and there will be no issues.2. Still regarding power outage protection, after simulating power outages with relays thousands of times without issues, we finally conducted a full machine test, only to frequently find that power outage protection did not function correctly.Upon careful inspection, there were no abnormalities in the circuit; everything was the same.As a result, the engineering department accused our R&D department of not testing thoroughly, claiming that the released product was problematic.Ah, how disheartening.After careful analysis, we believed that the possibility of software anomalies was very low.The main issue was still with the hardware; the supercapacitor on the hardware might not have stored enough energy during frequent power outages to complete the protection process.So what exactly caused the frequent power outages?According to design requirements, the supercapacitor should charge to 80% energy within 3 to 5 seconds, which should be sufficient.What could cause frequent power outages in less than 3 to 5 seconds?It sounds unbelievable; using a digital oscilloscope to continuously track the power supply of the control board revealed the issue.It turned out that the three-phase AC power needed to connect to a phase protector, which would frequently switch on and off during system operation (possibly related to the system’s state).The solution was simple: just connect the controller’s power supply before the phase protector.These issues seem to be hardware-related, but they are often encountered during product debugging.These problems require software personnel to confirm whether bugs in the software could cause such situations, and then hardware engineers need to verify the hardware.Of course, the hardware verification process is lengthy and complex, and debugging methods are very limited; debugging embedded software is generally more cost-effective and productive than hardware.Thus, embedded software personnel often spend a lot of time confirming software issues before suspecting hardware.As embedded developers, understanding the basic principles of hardware, combined with the working principles of software, and collaborating with hardware engineers to experiment and locate errors is a very effective approach.Some friends online often ask me questions.These questions often pertain to low-level knowledge, including some multi-processor issues.Regarding multi-processor issues, I am also somewhat inexperienced, so I would like to discuss embedded multi-CPU applications with everyone.Embedded systems are one of the application fields of computer science.Since it is an application field of computer science, one must have solid theoretical knowledge in computer science to excel in this field.First, multi-processors can be categorized into several types:Processors of the same model, all identical, connected via a communication method such as multi-port RAM, RapidIO, gigabit Ethernet, or PCI-E;Processors of different models, or even completely different architectures, connected via a communication method, as mentioned above, such as multi-port RAM, RapidIO, etc.;Multiple CPUs integrated within the same chip.These CPUs share everything, belonging to a tightly coupled system.Why use multiple processors?For large-scale parallel computing;To leverage the characteristics of multiple CPUs, such as using a DM642 solution for complex video applications.To utilize the floating-point computing capabilities of DSPs while also using the transactional computing capabilities of ARM;Simply to improve system performance.For ordinary applications, improving system performance is the basic starting point.However, applying multiple processors in embedded systems is not a simple task.The software design difficulty for multi-processors is high, and debugging is also a significant issue.If an operating system is not used, and a front-back system is adopted, one must design a communication algorithm and a result integration system.In such systems, many components must be designed independently, where the reliability and fault tolerance of the bus are crucial.Therefore, if possible, using a mature and stable operating system to support multi-processors can significantly reduce development difficulty.However, finding such an operating system is not easy.First, one must clarify their application needs: is thread/process migration required?Is processor balancing needed? For multi-processors, if thread/process migration is not supported, then dynamic balancing of processor tasks cannot be discussed; otherwise, one can only specify in advance which processor each thread/process will run on.For heterogeneous multi-processors, thread and process migration have little practical significance.For profit-driven companies, there is currently no practical value in discussing this.Thus, migration is limited to symmetric processors.However, not all processes can migrate for symmetric processors.For symmetric processors, the operating system encapsulates the underlying details, allowing users to develop as if they were developing for a single CPU; of course, it cannot be completely identical to a single CPU, but it at least alleviates many difficulties.Many friends ask me if RTEMS can run on x86 multi-processors like CMP.Of course.However, the design differs from ordinary symmetric multi-processors.Because CPUs on CMP share many resources, interrupts, memory, and buses, their addressing spaces are basically consistent.For RTEMS, which supports symmetric processors in a heterogeneous manner, if there are several CPUs, each must run its own RTEMS.Thus, communication becomes particularly important; multiple RTEMS require multiple system ticks, so where do the ticks come from? CMP shares many resources, so users must manually specify interrupt sources and allocate memory space for RTEMS, which results in multiple CPUs on CMP running RTEMS, but the drivers for each CPU are often different.This tightly coupled system is very challenging.In contrast, SMP composed of the same type of CPU is simpler because all drivers are the same; communication drivers may need special handling due to communication methods, but this greatly reduces development pressure and debugging difficulty.It is like each CPU being a core, which would be overwhelming.Especially regarding debugging issues, from an economic perspective, I prefer multi-processor systems composed of multiple identical single CPUs.Often, for heterogeneous processors, using RTEMS can also be easily managed, but there is still a problem: multiple cores require their own RTEMS support, which complicates development.Moreover, debugging operating systems is relatively complex.Thus, the practical solution is that among heterogeneous processors, the processor responsible for transactional operations runs the operating system, while the processor responsible for computation adopts a front-back system, simply communicating through shared memory to respond to the operating system’s computation requests.This greatly reduces development difficulty; after all, the operating system treats the DSP as a hardware register, writing a few registers can yield results, or inputting a set of astronomical data can produce a complex result.Anyway, in summary, this reactive processing method is the approach adopted in the vast majority of projects.It is simple, reliable, and practical.It seems that multi-processors in embedded systems are highly related to applications.-END-