Follow the Embedded Learning Station for more fresh hotspots every day.
🤟 Eating Tip: This article has 1311 words and an estimated reading time of 7 minutes~
Today, I am sharing an open source project suitable for C++ developers to advance their skills. This project is called Workflow, and the project address is as follows: https://github.com/sogou/workflow
“
Project Use Cases
This project is the server engine of Sogou, and it is used by almost all backend C++ services of Sogou and dozens of other companies, processing over 10 billion requests daily.
Moreover, this project is also very suitable for implementing network framework design in embedded applications. Framework design is a crucial aspect of a project’s success, and networking is especially important in AIOT, smart homes, and smart hardware.
For example, if the project needs to use HTTP, Workflow can be configured via CMake to support compilation across multiple platforms such as Windows, Linux, or MacOS, while maintaining the same interface on different platforms.
If you add an embedded device, such as ESP32, you only need to maintain one set of code on both the server and client to fulfill your application requirements.
This truly achieves the goal of porting cloud technology to run on embedded devices.
“
Key Features for Embedded Systems:
1. Multi-Platform and Multi-Architecture Support
The multi-platform code takes into account many design details in configuration and coupling. Currently, Workflow can run not only on Linux, Windows, MacOS, and Android but also happily on Raspberry Pi, domestic Longxin processors, and other architectures.
2. Fast Compilation
Workflow does not depend on other libraries except OpenSSL, and there are no templates at the interface layer, thus the compilation speed is very fast, allowing a usable lib to be built in less than a minute.
3. Small Size and Support for Compilation Trimming
The Kafka protocol in Workflow is not compiled by default, and other less commonly used modules can also be trimmed. Additionally, you can use the strip command to remove symbol links, reducing the library file size to around 400k.
4. Small Runtime Memory and Fast Scheduling
As an asynchronous scheduling library, Workflow has consistently excellent scheduling performance. Furthermore, its runtime memory usage is very small.
5. Very Convenient Custom Protocols
The active community and the project manager’s prompt responses to questions are also highlights. If issues arise during framework setup and porting, questions posted in the community will receive quick responses.
“
Open Source Support for Porting to Embedded Devices
It is reported that many users are already using it on embedded systems. GitHub has also provided patient explanations.
“
What Can the Framework Do?
1. Easily Build a Server
Needless to say, if a server framework cannot build a server, what’s the point? However, using this framework is very convenient. Take the HTTP server as an example; it only requires a few simple lines of code:
#include <stdio.h>
#include "workflow/WFHttpServer.h"
int main() {
WFHttpServer server([](WFHttpTask *task) {
task->get_resp()->append_output_body("Hello World!");
});
if (server.start(8888) == 0) { // start server on port 8888
getchar(); // press "Enter" to end.
server.stop();
}
return 0;
}2. Easily and Efficiently Initiate Client Requests
The project claims to serve as a universal asynchronous client, currently supporting HTTP, Redis, MySQL, WebSocket, and Kafka protocols. Below is an official example of a MySQL client:
int main(int argc, char *argv[]) {
...
WFMySQLTask *task = WFTaskFactory::create_mysql_task(url, RETRY_MAX, mysql_callback);
task->get_req()->set_query("SHOW TABLES;");
...
task->start();
...
}Previously, when C++ servers needed to access MySQL, they might use traditional clients. In a single thread, they would wait for data to arrive in a synchronous blocking manner. If multiple network requests were to be handled concurrently, users needed to manage multiple MySQL CLI objects.
Workflow perfectly solves this series of problems by delegating all such user requests to an internal poller thread for unified management, achieving efficient non-blocking IO behavior, and improving server performance when making client requests for data. No longer do you have to worry about this client behavior affecting the overall performance of the server.
Supports custom protocol client/server: Users can build their own RPC systems, and Sogou has an open-source project called SRPC based on this framework.
3. Construct Asynchronous Task Flows
Supports chaining, parallel execution, and combinations of serial and parallel, as well as complex DAG structures.
4. Asynchronous IO
Can be used as an asynchronous IO tool for files in Linux systems, outperforming any standard calls.
5. Integration of Communication and Computation
Most frameworks focus on the efficiency of network IO, while computation and task scheduling need to be implemented by the user. Workflow automatically schedules tasks, bridging resources such as networks and disks, making it especially suitable for computational modules requiring network communication.
“
Task Flow Framework Design
In the author’s design philosophy, all business logic is a task, and multiple tasks will form a task flow, which can form a graph. This graph can be a serial graph, parallel graph, or a combination of both, similar to this:
Or this complex DAG graph:
Of course, the hierarchical structure of the graph can be user-defined, and a remarkable feature is the support for dynamically creating task flows.
That’s the end of the sharing. I hope you all like it. Embedded systems truly require continuous learning to enrich your brain. I believe you can all become the most capable programmers.
Recently, many friends have received the programmer knowledge map, and the feedback has been great. We know that every technical field has its complete knowledge system, and a good knowledge system helps technical personnel grow and develop, reducing the detours taken during the learning process. So for those who haven’t received it yet, hurry up and claim it! (There is an entry at the bottom to claim it.)
Previous Recommendations
●
Amazon is reported to lay off 10,000 employees, the largest scale in history.
●
In 2022, the difficulty of employment under the pandemic, this is how to break through.
●
Monthly salary of 16k! A senior’s personal experience tells you whether to pursue a master’s degree or employment~