Follow and star our official account for exciting content
When searching for architects on various recruitment websites, you will find various system architects, web architects, backend service architects, etc., but it is rare to see embedded software architects. Does embedded software not need architecture? Do drivers not need architecture? The answer is certainly yes, but why are there so few positions in this area?
Currently, embedded development in China mainly divides into embedded low-level development and embedded application development. Low-level embedded development is generally called driver development or BSP development, sometimes referred to as Linux kernel development, which sounds quite impressive.
Why is there no architect with such impressive titles? The architects of the Linux kernel are Linus and the other maintainers of the Linux kernel, because the Linux kernel or operating system itself is a general platform that solves common problems. The big names in the Linux open-source community have already established architectural rules, leaving little room for creativity. Most work only needs to fill in according to the rules.
Moreover, most companies in China only need to integrate peripheral devices, port embedded platforms, and build and trim, with business requirements never exceeding the functionality provided by the kernel. This results in no new architecture needing to be designed or implemented by developers. So what are embedded BSP developers doing? Besides debugging various peripherals and cleaning up after hardware, they are just fixing stability bugs (details about specific work are not elaborated here; debugging peripherals will only add some experience and breadth, but contribute little to depth; it merely follows a path from not knowing how to debug to debugging faster, while solving stability issues does require some accumulated experience).
On the application development side of embedded systems, the business logic is generally simple and often overlooked, so recruiters feel there is no need to look for architect-level candidates.
Thus, it seems that the embedded industry indeed does not require architects, and it is not surprising to be looked down upon by the internet industry.
But is this really the case? For low-level embedded development, there should not be many developers in China capable of proposing architectural optimizations for the kernel and driver architecture, so for most ordinary people, it is best not to “fantasize” about becoming a Linux kernel architect.(Of course, I believe there are certainly capable experts among our compatriots), discovering and solving some bugs is more reliable.
So do we really not need architecture for embedded application layer development?
Here, I will share my own experience regarding the architecture design and optimization of an embedded device application software:
I once took over a project that adopted a single-process multi-threaded model, which included several modules represented by a, b, c, d, and e. The business logic of this project determined that these modules had many interrelations.
For example, in the initial design, module a was a state monitoring module that would call the interfaces of modules b and c based on the monitored state to implement some functions (the advantage of multithreading is that direct calls are very convenient, so developers mostly do this, which is simple and crude). However, requirements are always changing, and a new module f needs to process the state monitored by module a. Following the previous approach, completing this function involves two steps: 1. Provide an interface in module f, 2. Call that interface in module a. Thus, the new requirement has been “perfectly” solved.
The aforementioned changing requirements continue to arise, and the client proposes a custom requirement that needs to add another module g, which also processes the state monitored by module a, but this custom requirement does not need the recently added module f. At this point, the simplest and most crude way is to define a macro to distinguish the custom requirement from the previous general requirement and build two program versions. This approach seems simple, but as the number of custom requirements gradually increases, maintaining so many custom versions of the program becomes a nightmare, and code management and generality become significant issues, with the code filled with differential processing for different macro definitions. #ifdef xxx; do_something; #endif. A better practice would be to introduce dynamic monitoring of device model versions, using one build program version to dynamically support all custom requirements, thereby reducing maintenance of different build programs. However, this approach only solves the maintenance of build program versions and does not address the issue of differential macro processing, merely changing previous macro judgments to dynamic device version number judgments. If these differential judgments are concentrated in one place, it won’t lead to significant complexity issues, but this is clearly difficult to guarantee; these differential processes may spread to all corners of the project, making project maintenance a nightmare.
No need for profound software thinking; most people would think of extracting the differential parts and placing them in a unified management location, concentrating differential modifications in this unified management area.
The general practice is to use callbacks to set hooks, and then customize differential requirements in the callback. In the above example, this means adding a hook in module a and, during system initialization, customizing the callback processing function according to different device version numbers. Moreover, these custom callback processing functions must be handled in the same place; otherwise, if they are still scattered in various corners, it becomes meaningless (the previous method of not placing hooks cannot gather these differential configurations together). This approach brings another benefit: changes to functional requirements will not affect the processing of module a, meaning that we can add functionality without modifying the code of module a (the previous approach required modifying the calling process of module a), thus achieving separation of modules.
At this point, the architecture of the second solution (which doesn’t even qualify as architecture) has improved significantly compared to the first solution, at least making it somewhat easier for developers. For other custom requirements, developers only need to modify this callback processing and focus on the differential parts.
Software needs to evolve continuously; is the second solution the optimal solution? Certainly not, is there still room for optimization?
Now let’s digress a bit and discuss the advantages and disadvantages of multi-threading/multi-processing models, mainly focusing on the advantages of multi-processing:
I won’t mention the textbook explanations; firstly, I advocate for multi-processing models for large projects, regardless of performance, primarily for the following reasons:
Decoupling of modules:Many developers maintaining multi-threaded model projects should encounter the following issues: direct calls between modules. If you don’t believe it, your project must be modular, right? Now randomly delete a module and try building it to see if it can pass (just build, no need to run). I believe that in most cases, you will encounter situations where a certain function call or global variable cannot be found, indicating strong coupling between your modules.
Due to the inherent advantages of multi-threading, where the address space is mutually visible, direct calls are very easy. Many inexperienced engineers can easily write direct call interfaces that are simple and crude. If they encounter a static interface function, they often remove the static for convenience and use it directly. As the entire project continues to add features, inter-module crossings increase, and coupling becomes stronger.
The reason I advocate for multi-processing is that it physically isolates this “convenient” communication method, leading developers to think more about whether such interactions are necessary. If necessary, they will further consider whether the interface definition is clear and straightforward (because inter-process communication is relatively cumbersome, developers will aim to reduce interactions and clearly define interfaces, otherwise they will be making things difficult for themselves). This is similar to life; if everything goes smoothly, people may not think too much, but if there are some bumps along the road, they will gain different insights.
Therefore, I believe that the multi-processing model forces you to think more about the design of the program, physically reducing the coupling of modules.
Abstracting common components and separating common functionalities from business logic functionalities: When modifying a multi-threaded model to a multi-process model, you often find that some interface codes are repeated across multiple process modules because the interface functions were in one process space and could be called directly by everyone. For example, if interface A is called by modules a and b, after separating module a and b into two independent processes, interface A needs to be implemented separately in both a and b. No need to explain; duplicated code is a big taboo in software engineering and must be eliminated. The solution is straightforward: separate the interfaces called by multiple modules into a library for other modules to call. Once you complete this task, you may notice that the separated interfaces can serve as common components for the entire project. Ideally, the code under the library is a common foundational component, while each module serves as an independent business processing module.
Convenient problem localization:In a multi-threaded model, if one thread exits abnormally, the entire process will exit. Of course, crash information can help identify which thread died, but if these thread modules are maintained by multiple teams, when the entire process crashes, determining which team should resolve it becomes a significant issue. Sometimes, a thread appears to be hanging, but it is actually caused by another thread module (the bane of coupling), leading to blame-shifting between teams (confident engineers believe their code is flawless).
However, if you use a multi-process model, if your service process crashes, you can find the reason yourself; there’s not much to argue about.
Convenient performance testing:In a multi-threaded model, it is not easy to view the resource usage of a single thread (at least some embedded systems lack comprehensive commands). When the resource consumption of the entire process is high, determining which module thread is problematic becomes just as challenging. In a multi-process model, whoever’s process consumes a lot of resources can be singled out for investigation; this is still a granularity issue. In the same system, dividing it into multiple processes will inevitably reduce the complexity of a single process, making it easier to locate and identify various issues.
Distributed deployment:The internet industry has always emphasized distribution, cloud computing, etc., while the embedded industry seems to suffer because distribution is rarely needed. However, in most cases, embedded systems use single chips for independent operation, so distribution is seldom encountered. But what if one day you need to distribute the functions of a single chip across two chips? Multi-processing expansion becomes much easier.
This is just a special example; in fact, embedded devices are inherently distributed industries, having already achieved separation from the beginning rather than evolving from a centralized to a distributed model.
Facilitating code permission isolation in companies: I actually disdain this practice; companies should trust their employees. However, given the state of integrity in China…, some isolation is not unreasonable.
In a multi-threaded model, as mentioned earlier, if you remove a module, you might not even be able to build it. So, should all code be exposed to all engineers? Clearly not, so each module can only provide libraries. However, I believe that organizing common functional interfaces into common libraries is a normal practice, while providing business-related modules as libraries is somewhat… questionable.
At this point, I should add that all the advantages mentioned above are not critical points; they do not give multi-processing an absolute advantage over multi-threaded models. They merely reflect my personal viewpoint that the multi-processing model forces engineers to think about solving certain problems (and these issues would be considered by experienced engineers regardless of the model).
Having said so much, it’s time to consider modifying the previous project example to a multi-process model, or else it’s all just theoretical discussion. Let’s begin:
The first issue to address is: selecting a communication method for multi-processing; direct calls between threads cannot be used anymore, so how do we choose a communication method for multi-processing?
Linux provides many IPC methods, which I won’t enumerate here. For non-large data control and message transmission, a good method is to use sockets, particularly UNIX sockets on the local machine (what are the benefits of this approach? If you need to turn a single system into a distributed system, the advantages become apparent).
However, merely using sockets to implement the functionality of the previous example will still encounter some issues:
Referring back to the previous example, it should be noted that the second solution we optimized is no longer applicable in the multi-process model for simple reasons that need no explanation…
A simple approach is to base it on solution one, changing direct calls to socket communication (just define the communication protocol). However, engineers familiar with socket development know that initiating socket communication requires some preliminary work (mainly establishing connections to associate the two modules). Therefore, the previous example transforms into this: module a needs to establish connections with modules b and c, and if module f is added, module a also needs to connect with module f. When you mentally draw a connection diagram, you will see that it resembles a spider web, with complex interrelations between nodes, and like solution one, adding a module associated with a requires modifying the code of module a, making this situation even more cumbersome and complex than the multi-threaded model. This approach is a definite nightmare.
Well, how to solve this? I’m sure many people have thought of using a bus distribution method. Developers familiar with Android system development will think of Binder, those familiar with OpenWRT will think of ubus, and desktop developers will think of dbus. Internet industry developers certainly know about the pub/sub modules provided by Redis.
The principles of Binder, ubus, etc., are quite simple: establish a message center and construct a forwarding routing model. All other modules do not interact directly but use the message center for forwarding and routing. The routing rules are defined using the observer pattern of subscription/publication. (Embedded developers or C language developers often mistakenly believe that design patterns are only associated with object-oriented languages. While many experts have popularized this, some developers’ information channels are relatively closed, leading to this misconception still being prevalent).
Based on this model, the requirements of our previous example can be well addressed. By adding a message center module, all modules needing communication only connect to this message center module and subscribe to the events they are interested in. When an event occurs, they only need to handle it accordingly.
Thus, modules b and c subscribe to events from module a; when module a detects a certain event, it publishes that event, which first reaches the message center and is then forwarded to modules b and c. For the newly added module f, it only needs to subscribe to that module without needing to modify the code of module a, making functional expansion very convenient.
Simultaneously, the previously mentioned custom development has also been simplified. If a custom version needs to add module g, it only needs to start module g as an independent process in the custom version and subscribe to events from module a. The difference between the custom version and the general version lies in whether to start the process of module g, thus achieving one of the goals of software engineering: adding functionality is like building blocks; you only need to insert (start) or remove (not start) a module, and changes in functionality are confined to one or a few modules without affecting the main framework.
The above roughly describes the process of gradually optimizing project requirements. Although the example seems based on an embedded project, it appears applicable to software engineering in general.
Now, let’s turn to the internet industry:
Looking at the articles shared by architects of major websites about the technological architecture transformation of their sites, the first thing mentioned is usually the breakdown of previous application server functions based on business into more refined services (for example, the breakdown of e-commerce services for login, registration, transactions, products, sellers, etc.), and then deploying the split services across multiple servers to provide concurrency. Doesn’t this sound familiar, similar to the division from multi-threading to multi-processing?
After splitting, communication issues arise, and many message middleware solutions emerge, such as Alibaba’s Dubbo. A simple understanding of these middleware principles reveals that they revolve around subscription/publication, RPC, etc.; it can be said they are fundamentally similar, while the challenges lie in protocol formulation and performance enhancement.
Comparing the load balancing solutions in the internet industry, it seems that the load balancing front end resembles a message center.
Having said all this, I only want to illustrate one point: software design is interconnected, and the underlying thoughts are the same. Although the business logic in the embedded industry is relatively simple, careful consideration can still lead to many architectural improvements and designs.
However, it saddens me that some embedded developers, given the simplicity of business logic, feel that adopting less optimal solutions can still solve problems and do not think about how to optimize and improve.For instance, in the earlier example of solution one, if there are not many custom requirements, maintaining it is manageable. Even if custom requirements increase, hiring some junior programmers can still handle it; companies where one person is responsible for one set of code for a project do exist.
Similarly, there should not be an insurmountable wall between the internet and embedded industries; we should focus more on universal software engineering principles.
Copyright Statement:This article is sourced from the internet and aims to share knowledge freely. All rights belong to the original author. If there are any copyright issues with the works, please contact me for deletion.
‧‧‧‧‧‧‧‧‧‧‧‧‧‧‧‧ END ‧‧‧‧‧‧‧‧‧‧‧‧‧‧‧
Follow my WeChat public account, reply "join group" to join the technical exchange group according to the rules.
Click "Read the original text" for more shares. Welcome to share, bookmark, like, and view.
