Ma Yinong is a software developer who is in a good mood lately because he is going to be responsible for the development of a critical embedded system software. He decided to visit his teacher of many years to hear the master’s opinion.

Master: Ma Yinong, long time no see. You look good, do you have anything new to talk about?

Ma Yinong: Hey, Master, I happen to have a new project I want to consult with you. We plan to develop a new embedded system to replace that old antique that has been in service for nearly 20 years.

Master: Oh, it’s finally time for an upgrade, that’s great news. That old system really can’t keep up with the times, and it’s a headache to maintain.

Ma Yinong: Yeah, it’s a tearful story. But I’m a bit worried; designing the new system isn’t an easy task.

Master: Don’t worry, we’ll take it step by step. Tell me about this new system.

Ma Yinong: This new system is mainly a task scheduling system that will run on specific embedded devices, responsible for orchestrating and executing various tasks. Some tasks need to be executed on a schedule, while others execute immediately upon receiving a command. Moreover, these tasks are all related to network I/O and need to access external resources to fetch data for processing.

Master: It does sound like a complex system. What architectural style are you planning to use for the design?

Ma Yinong: That’s exactly what’s troubling me. The old system uses a multi-threaded architecture, and while everyone is familiar with this style, it comes with many issues, such as shared data problems, deadlocks, difficulty in reproducing issues, and debugging challenges. So I’m wondering if the new system should continue with a multi-threaded architecture or if there are better alternatives.

Master: Multi-threading does have its limitations. However, you might want to consider an event-driven architectural style. Event-driven architecture can provide higher efficiency and better scalability in many scenarios.

Ma Yinong: Event-driven? That sounds like a good choice. Can you explain the principles and advantages of event-driven architecture in detail?

Master: Of course. Simply put, event-driven architecture executes tasks based on event triggers. Let’s use a simple analogy. Multi-threading is like a high-end restaurant where each customer (task) has a dedicated waiter (thread) serving them. When a guest arrives, the waiter takes their order, and from the moment the guest sits down until the order is completed, the waiter is waiting there. This is blocking I/O. In contrast, event-driven is like self-service, where there is no waiter to take your order; you scan the code to place your order. The system automatically notifies the kitchen. Once the meal is ready, the system will notify you to pick it up yourself.

Ma Yinong: I see, event-driven does have many advantages. But how do we implement event-driven architecture? Are there existing models to refer to?

Master: The key to implementing event-driven architecture is designing an efficient event loop mechanism. In this mechanism, the system continuously listens for event sources, and once an event occurs, it triggers the corresponding handling function. Thus, task execution no longer depends on thread switching but is driven by event triggers. As for existing models, you can refer to the Reactor model, which is a common event-driven model that has been successfully applied in many systems.

Ma Yinong: Sounds impressive. But my system seems like it could be simpler. It runs on a bus architecture that encapsulates MQTT, so MQTT messages can serve as events. These events can drive the entire task execution process. Do you think this is feasible?

Master: Haha, it looks like you already have your own ideas. Indeed, if your system runs on a bus architecture that encapsulates MQTT, then using MQTT messages as events to drive task execution is entirely feasible. In this way, you can utilize MQTT’s publish/subscribe model for task scheduling and execution. However, the event-driven plus multi-threading solution is indeed more complex than the old system, which may adversely affect system stability. Therefore, you need to pay special attention to thread safety and concurrency control during implementation.

Ma Yinong: You’re right. I’m also worried about this issue. So, is it possible to implement event-driven architecture using a single thread? This might simplify the system complexity.

Master: Implementing event-driven architecture with a single thread? That’s indeed a bold idea. However, in certain scenarios, single-threaded architecture can achieve efficient concurrent processing. For example, Node.js is a typical case; it uses a single-threaded asynchronous I/O model to achieve high concurrency. However, this requires your system to have asynchronous characteristics, meaning all I/O operations must be non-blocking to fully utilize the advantages of a single thread.

Ma Yinong: You’re right. In fact, our system already has asynchronous characteristics. The cost of asynchronous calls is about 10ms or less. As an edge device, our concurrency is not as high as that of a server, so a cost of 10ms is entirely acceptable. In this case, performance shouldn’t be an issue, right?

Master: Haha, it looks like you’ve thought it through. Indeed, if the cost of asynchronous calls is within an acceptable range, then implementing event-driven architecture with a single thread is entirely feasible. Moreover, event-driven architecture can bring an additional benefit: decoupling. Since event triggering and handling are separated, you can easily add, remove, or modify event handling functions without affecting other parts of the code. This is very beneficial for the system’s scalability and maintainability.

Ma Yinong: That’s great! Decoupling is indeed a feature we need for this system. Because the needs of different provinces across the country vary, the new system needs to meet the demands of each province with a single system. This is indeed a significant business challenge. Do you have any suggestions?

Master: In response to this business challenge, I suggest that you fully consider design principles such as separation of concerns, single responsibility, open/closed principle, and the Unix mechanism and strategy separation principle when designing the system. By dividing the system into multiple independent components, each responsible for a specific function or business logic, you can achieve high scalability and configurability. This way, you can customize and extend the corresponding components according to the specific needs of each province. Additionally, the event-driven architectural style can help you achieve decoupling and communication between components.

Ma Yinong: Your advice is very insightful! I will think carefully about these principles and try to apply them to our system design when I get back. Thank you!

Ma Yinong returned home, still very excited, with thoughts of event-driven, single-threaded, scalable, replaceable, and orchestratable running through his mind.

That night, he dreamed of a round moon, which hung in the sky and was reflected in multiple rivers on the ground.

The next day, Ma Yinong visited the master again.

Ma Yinong: Master, I’m back. After our discussion yesterday, I have a clearer understanding of the new system design. I want to share my ideas with you: this task scheduling system is like a natural ecosystem, with multiple rivers, each running tasks corresponding to parallel task flows. In each river, tasks flow serially (execute) and are sorted by priority. What do you think of this design?

Master: Haha, your idea is very creative! Comparing the task scheduling system to rivers can vividly describe the flow and execution of tasks.

Ma Yinong: Yes. Each task consists of multiple atomic tasks. Atomic tasks are the smallest tasks that cannot be split. Atomic tasks can be extended or replaced, and tasks can orchestrate atomic tasks. This allows for a standard implementation, and the needs of each province can be realized based on this, extending or replacing certain atomic tasks, or re-orchestrating atomic tasks. Finally, we form an atomic task library, orchestrating atomic tasks into various tasks based on the needs of each province, with these tasks running at different priorities in multiple rivers.

Master: The idea of atomic capabilities and orchestration is excellent; the system will have powerful scalability. Coupled with the decoupling capability brought by event-driven architecture, it will certainly meet various demand differences!

Ma Yinong: The master’s ideas on component scalability, replaceability, and orchestrability have inspired me. With these three features, our architecture can meet any challenge.

Master: Very good! However, I have one question. If a higher-priority task suddenly arrives while a task is being executed, how should you handle it?

Ma Yinong: My idea is to treat atomic tasks as the smallest scheduling unit. When a high-priority task arrives, if the current atomic task has not yet completed, it should continue to execute until it is finished. Then, immediately after the current atomic task is completed, the high-priority task should be inserted for execution. In this way, atomic tasks are both the smallest business components in design and the smallest business components at runtime.

Master: Hmm, your approach is very reasonable. Treating atomic tasks as the smallest scheduling unit indeed matches your business characteristics and simplifies the management and scheduling of tasks. Furthermore, your design also considers the priority of tasks and the timing of insertion, which is also very important in practical applications. Ultimately, they provide the necessary scalability for the system, meeting the differentiated needs of various provinces across the country. And all this is built on a single-threaded foundation, which makes development and debugging much easier. Overall, your design thinking is clear and creative. I believe that implementing the new system according to this design will yield excellent results!

Ma Yinong: Thank you for your affirmation! With your guidance and suggestions, I am more confident in the design of the new system. I look forward to building this system and laying a solid foundation for our products!

Master: I wish you success!

As Ma Yinong left, his heart was filled with excitement, and his eyes revealed an unwavering determination. Let us all send him our blessings!

Series of Blogs on Embedded Systems:

• Quality of Embedded Systems – How to Do Things Right the First Time?

• Requirements Management of Embedded Systems – How to Double Delivery Efficiency?

• Team Design of Embedded Systems – Team Topology

• Team Design of Embedded Systems – Team Priority