Abstract: In systems with high reliability requirements for network communication, dual redundancy backup for network cards is necessary. When the active network card or line fails, it can automatically switch to the backup network card. This article details the implementation of a dual network card redundancy backup technology in the real-time operating system VxWorks.
Keywords: Real-time Operating System VxWorks; Network Controller; Dual Redundancy Backup
1 Introduction
With the maturity of network technology, Ethernet has become the main medium for interconnecting various control system interfaces. In certain special applications, to improve the reliability and fault tolerance of the system, dual redundancy network technology is required. In a dual redundancy network, each node uses two network cards (or a single board with dual cards), interconnected by two HUBs or switches. When one network card fails, or the network cable is damaged, or one of the HUBs or switches fails, the network can still operate normally.
Although a dual redundancy network has two network cards and two channels, for high-level application systems, it still presents the characteristics of a single network card. Specifically, each node’s two network cards can only have one physical address and one IP address; otherwise, normal communication cannot occur.
Currently, most operating systems (such as Windows, Unix, Linux, etc.) support multiple network cards, but they are not designed for redundancy. Each network card has an independent physical address and IP address, allowing the application system to use them independently. According to the current requirements of the Navy for high-performance real-time systems, we will introduce the design methods and key points for switching and redundancy backup of dual network cards in the VxWorks environment.
2 Introduction to VxWorks
The VxWorks operating system is a high-performance embedded real-time operating system with industrial leadership and is a key component of the embedded development environment Tornado. It provides programmers with good reliability and excellent real-time performance, thus being widely used in communication, military, aerospace, and other high-tech fields with extremely high real-time requirements.
In comparison to widely used operating systems like Unix and Windows, which are excellent platforms for application development and interactive applications, they are not suitable for real-time applications. On the other hand, previous real-time operating systems did not provide a good operating environment for application development and non-real-time components (such as to GUI). Wind River’s philosophy is to utilize two mutually cooperating operating systems to complement each other’s shortcomings (such as VxWorks and Windows), allowing them to each play to their strengths. VxWorks provides real-time performance, while the host is used for application development and running non-real-time applications.
3 Redundant Switching
In the operation of the system, devices may fail due to hardware or software reasons. Implementing effective error prevention and fault tolerance mechanisms is essential for a system that needs to operate stably. Redundancy backup technology provides redundancy for error-prone hardware devices. When one device cannot operate normally for some reason, another device can immediately take over to perform the same function. This article analyzes a feasible error protection mechanism for network adapters in cases where network factors lead to system failures.
Typically, if two network cards are installed in a system, they have different physical and IP addresses. When one network card fails, the other network card cannot take over in real-time because their physical addresses are different, preventing it from receiving data meant for the first network card. However, in systems with high reliability requirements for network communication, dual redundancy backup for network cards is needed, where one network card is used during normal operation, while the other serves as a backup. The backup network card does not send or receive data but is already activated. When the active network card fails, the backup network card can automatically switch over to continue operation. Clearly, this requires the two network cards to use the same physical address and the same IP address. From the perspective of the application program, only one network card is seen as operational. The application program does not care which network card is working or how the switching occurs between the network cards.
In the VxWorks system, network cards of the same type use the same driver, and the network cards are distinguished by handles provided by NDIS. When NDIS calls the NIC driver interface functions, it passes the handle of the network card into the function. This provides the basis for implementing redundancy backup in the driver program. Once the dual network card driver is implemented, it is bound with the high-level protocol driver, making it completely transparent to the application program. Therefore, the ideal way to implement dual redundancy backup for network cards is in the NIC driver. However, if this approach is taken, we must have the driver program for all network cards under VxWorks, which can be very challenging for the average user. Currently, some research institutions have completed this work, but their prices are too high, with the software and hardware costing around 15,000 yuan, which is quite high for typical small users. Therefore, we choose to implement this goal in the high-level user program without significantly affecting the redundancy backup effect.
In early VxWorks 5.1.x, the system did not support multiple network cards. Typically, VxWorks initializes only the network card that booted it when starting. If we need to switch from one network card to another on a specific target machine (for example, from an ENE type to an ENP type network card), we can achieve the switching between network cards using functions provided by the system. What we need to do is connect the required network card and delete the previous network card’s route. We can call the connection functions for specific network cards: enpattach(), enettach(), etc. For example, the enpattach() function connects the network card interrupt and establishes enpsoftc, and the system will initialize that network card accordingly.
enpattach (0, 0xffde0000, 192, 3); /* Parameters are: network card unit number in the system, enp's shared memory address, interrupt vector to connect, interrupt level */It should be noted that although in VxWorks 5.1 these connection functions have a unitNum parameter, it seems to support multiple network cards, but this is not the case. In the following description, we can see that it does not define the IP MAX UNITS macro, so it does not support multiple network cards; the value of unitNum remains 0, which readers can check in the source code.
When VxWorks starts, it loads a default network interface into the network interface table, so after connecting to the required network card, we need to call the ifRouteDelete() function to delete the route connected to a specific network card.
ifRouteDelete("ln", 0); /* ln is the name of the network card to be closed */
Then use ifMaskSet() to define the subnet mask
ifMaskSet("enp0", 0xFFFFFF00);
Finally, use ifAddrSet() to assign an IP address to the specified network card
ifAddrSet("enp0", "147.11.1.230");
This completes the switching between multiple network cards in VxWorks 5.1.
As for VxWorks 5.4, it inherently supports multiple network cards. Below, we describe how the SBS PC104 system supports software redundancy switching for multiple network adapters under VxWorks (taking the NE2000 type as an example).
First, we need to configure the IO base address and IO interrupt for all network adapters in the hardware environment using the network card configuration program. This article assumes there are only two network adapters in the system, with the following IO base addresses and IO interrupt numbers. In actual systems, there may be multiple network adapters, and the configuration method can be extrapolated as long as the network card’s IO base address and IO interrupt do not conflict with other devices in the system.
Modify the config.h file in the \\Tornado\\target\\config\\pc486 directory,
#define IO_ADRS_ENE 0x320 /* IO base address of the first network adapter */
#define INT_LVL_ENE 0x05 /* IO interrupt number of the first network adapter */
#define IO_ADRS_ENE1 0x300 /* IO base address of the second network adapter */
#define INT_LVL_ENE1 0x09 /* IO interrupt number of the second network adapter */Modify the configNet.h file in the \\Tornado\\target\\config\\pc486 directory,
In the driver parameter table endDevTb1 of VxWorks’s enhanced network devices, add the driver entries for the two network adapters.
#ifdef INCLUDE_ENE_END
{0, END_ENE_LOAD_FUNC, ....., FALSE}, /* 0 indicates loading the first network card */
{1, END_ENE_LOAD_FUNC, ....., FALSE}, /* 1 indicates loading the second network card */
#endifEND_ENE_LOAD_FUNC is the device loading callback function. Since the two network adapters in the system are of the same type, the same loading callback function is used.
Modify the sysNe2000End.c file in the \\Tornado\\target\\config\\pc486 directory:
SysNe2000End.c contains the actual loading code for the network adapter, and the function defined by END_ENE_LOAD_FUNC, sysNe2000EndLoad(char*,void*), is in this file, and it needs to be modified as follows:
if(pParamStr[0] == '0')
sprintf(cp, ne2000ParamTemplate,
IO_ADRS_ENE,
......
ENE_OFFSET);
else if(pParamStr[0] == '1')
sprintf(cp, ne2000ParamTemplate,
IO_ADRS_ENE,
......
ENE_OFFSET);Next, in the Bootable BSP project directory, modify the prjParams.h file to change IPMAXUNITS to the number of network cards you need, such as 2, to require the system to support multiple network cards simultaneously.
The above is the modification to the BSP; below we will illustrate the application program using the Shenzhen Shengbo Company embedded DETH module as an example. SysExpanModule/DETH is fully compatible with Novell’s NE2000 and complies with the standard DETH (Ethernet) protocol to complete the sending and receiving of data packets on the board logic. It provides two 10Base-T interfaces to meet the needs of multiple network cards, but we can set them to the same MAC address as needed.
From a practical perspective, we need to query the interface currently in use to determine whether it is in normal working condition. Generally, network cards have self-diagnostic capabilities and modify the values of their internal registers accordingly. We can continuously read the values of the interface’s internal registers using VxWorks’s low-level functions sysOutByte() and sysInByte() in a querying manner.
sysOutByte(intNum, 0xc0);
enable=sysInByte(intNum+3)&0x04;When the interface is abnormal (enable==4), we will switch the interface.
As a dual redundancy backup, our sender should be transparent to the receiver. Therefore, our two interfaces need the same IP address, which creates an address conflict problem. Thus, before using the new interface, we must remove the original interface’s IP and delete it from the host list. Otherwise, we cannot drive the second interface with the same address.
hostDelete("host table", "192.168.0.1");
ifRouteDelete("ene", 0);
ipDetach(0, "ene"); /* Remove the first network card from the system */
ipAttach(1, "ene"); /* Establish a new connection between the IP layer and the destination interface link layer */
/* If the network layer and data link layer connection is successful, set the currently used network adapter */
ifMaskSet("ene1", 0xFFFFFF00);
ifAddrSet("ene1", "192.168.0.17"); /* Configure the IP address of the switched network adapter */
ifBroadcastSet("ene1", "192.168.0.255");After completing the above steps, we can still send using the same broadcast sending program, and the receiving end can only recognize the sender’s route. Tests conducted by the author indicate that this switching method can achieve a switching time within 10ms, while the normal switching time reflected in the application program is around 100ms, which basically meets the requirements of dual redundancy backup technology (completing redundancy switching in the driver program takes about 20-80ms).
4 Conclusion
Using the method described in this article, we have successfully implemented dual NIC redundancy backup technology in the VxWorks environment for a certain missile boat comprehensive navigation display project, meeting the system’s operational requirements.