Click the blue word to follow us
With the official launch of the ‘East Data West Computing’ project, new infrastructure, the big data industry, and surrounding supporting industries are ushering in significant benefits and investment opportunities. Facing a trillion-level industry space, major cloud computing vendors and ICT (Information and Communication Technology) companies are scrambling to seize the high ground, and the ‘East Data West Computing’ track continues to be hot. So, is all ‘East Data’ suitable for ‘West Computing’? What are the core technologies and applications supporting ‘East Data West Computing’? Not all ‘East Data’ needs ‘West Computing’ Information transmission has two key indicators—bandwidth and latency. Limited by physical laws, regardless of how large the bandwidth is or how fast the speed is, latency objectively exists and can never be overcome. This also means that in the process of transmitting eastern data to the west for storage and processing, if it is a real-time application that requires rapid feedback of results, it cannot take into account timeliness through ‘East Data West Computing’, so it is not suitable for data-intensive and time-sensitive data applications. In other words, online ‘hot data’ that needs to be frequently accessed by computing nodes and has high network latency requirements is not suitable for ‘West Computing’, such as disaster warning, remote medical care, financial securities, etc.; while offline data that is not frequently accessed and has low network latency requirements, as well as ‘warm data’ that is in between, are more suitable for ‘West Computing’, such as offline analysis, backend processing, and storage backup. ‘Cold data’ and ‘warm data’ account for the vast majority of the current total data volume. These data mainly include two major types: one is application data used for retrieval, such as e-commerce retrieval data; the other is long-term recorded data, such as digital archives. Although these data have a small transmission volume, over time, they can produce astonishing effects with data volume + computing power. In the applications of these two major types of data, ‘East Data West Computing’ can fully apply the tiered computing principle. For different organizational structures or application scenarios, the business processing and resource services of each tier can be automatically and flexibly organized or reorganized according to demand, and microservices that adapt to them can be deployed without being limited by hardware resources, thus achieving shared configuration and load optimization of computing resources at all levels. For example, constructing a ‘new type of facial recognition application service system’ based on tiered computing can improve the intelligent processing capabilities of front-end devices by migrating part of the video image analysis processing to the edge, placing global analysis functions on the cloud to reduce the resource requirements for computation, storage, and networking on the cloud platform core end, thereby increasing the speed and efficiency of large-scale facial recognition analysis and enhancing social management capabilities. Another example is the ‘smart traffic optimization system’ based on tiered computing, which collects and processes real-time traffic data using smart streetlight companions, traffic flow cameras, and other edge devices, uploads it to a big data platform in the cloud for real-time congestion analysis, and then sends the intelligently analyzed results back to the edge, realizing that signal timing optimization shifts from passive to active, improving traffic scheduling efficiency overall from local to macro. Core technologies supporting ‘East Data West Computing’ The implementation of ‘East Data West Computing’ is not achieved overnight. As a national-level strategy radiating across the country, the ‘East Data West Computing’ project faces many challenges while bringing development opportunities. Key links such as computing power platforms, network transmission, and data governance are all critical to the advancement of the project and also require a rigorous process of overcoming difficulties. In this process, fully leveraging the synergy of cloud storage technology, virtualization technology, cloud database technology, cloud transmission technology, cloud video technology, and cloud-native technology can further support the national new infrastructure layout and digital innovation of ‘East Data West Computing’. These hardcore technologies are not out of reach for ordinary people; their applications are gradually entering our daily lives.
“
Efficiently storing massive data:
Cloud Storage + Virtualization
Whether it is ‘hot data’ hosted in data centers in eastern cities or ‘cold data’ migrated to computing hubs in the west, the foremost issue faced is data storage, which must rely on cloud storage and virtualization technology. In recent years, domestic cloud storage technology has developed rapidly, transitioning from learning open-source cloud storage technology to gradually developing cloud storage technology with independent intellectual property rights. Among them, the cStor cloud storage system can be said to be a typical representative. At the same time, the cStor cloud storage system, which combines software and hardware, adopts a distributed storage mechanism, dispersing data across multiple independent storage servers and providing a virtual massive storage volume externally. It has advantages such as high cost-effectiveness, low power consumption, high reliability, generality, and maintenance-free, making it widely applicable in scenarios with massive data storage needs. The cStor A8000 low-power cloud storage system can support a total storage capacity of up to 5376TB (equivalent to storing more than 5 million movies of 1GB each) while being three times more energy-efficient than traditional cloud storage products. It has been successfully applied in various fields such as safe cities, smart transportation, and healthcare, meeting the demands for both the large-scale storage of ‘East Data West Computing’ and green energy-saving needs. In the implementation of ‘East Data West Computing’, the power of virtualization technology also needs to be fully utilized to achieve resource optimization and dynamic expansion. Virtualization technology was introduced by IBM in the 1960s, primarily for server virtualization of IBM mainframes. Its core idea is to use software or firmware management programs to form a virtualization layer, mapping physical resources into virtual resources, and installing and deploying multiple virtual machines on virtual resources to achieve multi-user sharing of physical resources. For data centers, virtualization can achieve dynamic allocation and scheduling of resources, improve the utilization rate and service reliability of existing resources, provide automated service activation capabilities, reduce operational costs, and have effective security and reliability mechanisms to meet the security needs of public and enterprise customers. It can also facilitate system upgrades, migrations, and transformations. With the development of cloud computing, traditional data centers are gradually transitioning to virtualized data centers, which abstract and integrate the physical resources of the original data center using virtualization technology. This also provides important references for the construction of data centers for ‘East Data West Computing’.
“
Centralized processing of massive data:
Cloud Database Technology
The processing and analysis of massive data run through the entire lifecycle of the ‘East Data West Computing’ project, and the requirements for cloud database technology are quite high. In recent years, with the explosive growth of data, the development of domestic and international cloud computing big database technology has entered a rapid growth phase, from Google’s Bigtable, Amazon’s Simple DB, Oracle’s Exadata, EMC’s Greenplum, to Alibaba’s OceanBase and Cloud Creation Big Data’s DataCube, a hundred flowers bloom. OceanBase was created to solve the large-scale data of Taobao. It is a high-performance distributed database system that supports massive data and can manage hundreds of billions of records, featuring strong data consistency, high availability, high performance, and online scalability. It is also a domestically produced native distributed database that refreshes world records. OceanBase not only provides relational database functions but also offers transaction processing and business intelligence analysis for customers, having continuously supported ‘Double 11’ for 8 years. In addition to Alibaba, OceanBase’s clients include the Industrial and Commercial Bank of China, Sinopec, Bank of Communications, Shanghai Pudong Development Bank, China People’s Insurance, and China Mobile. The DataCube distributes large-scale operations on data sets to each node on the network by introducing indexing modules, parallel execution architecture, and local disk reading execution methods, achieving data processing. Each node periodically reports the completed work and the status update back, and as the number of nodes increases, its processing capacity will grow exponentially, allowing queries to be completed in real-time with high performance, simplicity, and reliability, enabling EB-level data to be processed in seconds. Not only does it have a significant performance advantage in querying and retrieving this part of the data, but it can also support data warehouse storage, deep data mining, and business intelligence analysis. For example, in mobile communication real-time processing analysis applications, the DataCube can provide high-performance signaling real-time query and analysis services, achieving second-level processing speeds while the application algorithm redundancy ensures data security and reliability, significantly reducing hardware, time, and labor costs, making it frequently favored by operators. 
“
Cross-domain interaction of massive data:
Cloud Transmission Technology
In ‘East Data West Computing’, how to achieve cross-regional transmission of massive data is of utmost importance. Based on a high-performance reliable file transfer protocol, the cTrans cloud transmission system adopts a parallel pipeline approach to jointly optimize transmission and storage, making it particularly suitable for remote transmission of massive data, capable of increasing the efficiency of remote data transmission by 4 to 80 times, significantly improving the efficiency of data transmission between data centers. Specifically, by applying the cTrans cloud transmission system, data from both local servers and cloud can flow unobstructed, breaking the physical barriers between IT infrastructures and solving the scene data flow problem, making many business application scenarios feasible; it can unify management of dispersed storage resources and better utilize existing data assets; real-time monitoring of transmission tasks ensures that the entire transmission process is ‘visible and controllable’; auditing the entire network transmission history and generating statistical reports regularly facilitates user archiving and analysis. At different stages of development, the cTrans cloud transmission system can help address changing data transmission challenges, reduce costs, dig deeper into value, and drive growth, continuously promoting improvements in business operation efficiency. Currently, the system has been widely used in video transmission, program distribution, data disaster recovery, and large-scale data transmission, etc.
“
Intelligent understanding of video data:
Cloud Video Technology
In ‘West Computing’s’ ‘East Data’, the proportion of video data is not to be underestimated. This data comes from numerous heterogeneous monitoring platforms, with different standards, facing challenges of technological intelligence and data sharing. To this end, the cVideo cloud video system adopts a hyper-converged architecture, utilizing the powerful processing and massive storage capabilities of cloud computing, allowing it to receive video while also well supporting the storage, analysis, indexing, retrieval, transcoding, and application of massive video data within the same cluster, avoiding the storage and application of video being divided by region and link, and consolidating video resources from different devices, platforms, and standards for effective sharing and integration. Currently, it has supported over 100,000 video streams in two cities, equivalent to accessing more than 100,000 monitoring images, providing application support for video data integration in ‘East Data West Computing’. In the construction of the ‘Snow Bright Project’, the cVideo cloud video system helps build a comprehensive scheduling video integration platform, integrating monitoring systems from cities to counties, meeting the requirements for large-scale video monitoring, massive data storage, and long-distance monitoring. For the collected monitoring data (including public security, traffic management, stores, etc.), it is uniformly summarized to the cStor cloud storage platform. Staff can upload, store, download, browse, and analyze a large amount of monitoring video materials, and through intelligent analysis of various video monitoring information, provide accurate and complete video information for smart applications such as social management, urban management, traffic command, and emergency management. Currently, the intelligent analysis algorithms paired with the cVideo cloud video system fully combine cutting-edge image processing technology and pattern recognition technology, while carrying and analyzing massive video data, achieving event retrieval for massive video, slicing playback of event occurrence videos, motion frame extraction, and object tracking. It has already realized flame detection, smoke detection, fighting event detection, traffic volume statistical analysis, vehicle speed determination, traffic accident determination, abandoned object detection, and intrusion detection. Through optimized intelligent recognition algorithms, the false detection rate and error detection rate can be significantly reduced, improving the detection rate and issuing corresponding alarm signals. This intelligent analysis algorithm is also applied in large-scale facial comparison, license plate recognition, real-time vehicle tracking, and other fields, receiving full recognition in the industry, and holds construction significance in the video data processing scenarios of ‘East Data West Computing’.
“
More innovative application scenarios:
Cloud Native Technology
In the digital wave, software developers and IT operations management personnel are gradually changing the past mode of local independent development and operation, and the concept of cloud-native has emerged. ‘East Data West Computing’, as an important part of digital infrastructure and a landing project of the digital economy, can fully absorb the cutting-edge ideas of cloud-native and further accelerate the innovation of digital models. In the digital world, the internet can be viewed as a web of ‘trade routes’, while software and data are like ‘ships’ and ‘cargo’ sailing along these routes. In traditional IT architecture, each enterprise deploys IT infrastructure such as computing, storage, and networking on its own ship. If there are new business needs, additional equipment needs to be added, and when demand decreases or disappears, various facilities can only sit idle, leading to low equipment utilization. Later, with the introduction of cloud computing models, it is akin to having several large freight companies provide standardized ships. If an enterprise’s ship has business needs, it can hire one of these freight companies to ‘ship’ and pay according to demand, saving the enterprise a lot of infrastructure acquisition costs. The introduction of cloud-native technology further equips these standardized ships with flexible containers that are highly available, resilient, easy to load and unload, and operate at a fast pace, further improving the efficiency and experience of cloud computing services. In other words, cloud-native technology is ‘born in the cloud’, directly using cloud infrastructure for development, granting agility to applications through microservices, using lightweight container technology as a carrier, and reducing development risks through DevOps (a method that automates processes between development and operations). Specifically, it applies the low-cost advantages of cloud computing service models to agilely build fault-tolerant, easy-to-manage, and observable loosely coupled systems, dynamically combining different services to build applications in the cloud, achieving transparency in the operating environment, streamlined development processes, service-oriented infrastructure, and integration of development and operations. Cloud-native platforms have characteristics such as reliability, availability, and resilience, and their resource usage is not limited to specific physical machines’ storage, CPU, memory, etc., such as Alibaba Cloud. Alibaba Cloud’s cloud-native application platform provides a complete set of container services, function computing, microservice systems, message middleware systems, and a series of basic service application tools, allowing easy completion of application and operating environment decoupling through product selection or combination, improving R&D and operational efficiency by ten times, while providing full lifecycle management and diagnostic operation intelligence, offering one-stop cloud computing elasticity and distributed architecture technical dividends. For example, in the currently common e-commerce live streaming sales, many merchants frequently hold promotions, and the system faces pressure to ensure stability. How to ensure the stable and smooth operation of the mall system is a key prerequisite. To address this, through the introduction of container applications and supporting services, optimizing testing, capacity assessment, and expansion processes, while improving production and research efficiency, further ensures system stability; for online education, business access has tidal characteristics, with peak access generally occurring in the evenings and holidays. Through cloud-native solutions, it can quickly respond to R&D needs in cases of sudden surges in users and traffic, enhancing user experience while avoiding resource waste and reducing technical costs. ‘East Data West Computing’, as another important development strategy following ‘South-to-North Water Diversion’, ‘West-to-East Gas Transmission’, and ‘West-to-East Power Transmission’, coordinates resources and energy consumption between the ‘East’ and ‘West’, utilizing the natural land, energy, and climate advantages of the West to meet the surging computing power needs of the East, thereby enhancing the overall computing power efficiency of the country and promoting green development. The comprehensive application of the aforementioned technologies can provide strong technical support for the implementation of ‘East Data West Computing’.■Author Unit: Nanjing Cloud Creation Big Data Technology Co., Ltd.
We welcome submissions to the ‘Enterprise Management’ magazine, sharing your thoughts and practical results!
Consultation phone:
010-68414646
Submission email:
Live Broadcast Announcement
Starts at 8 PM
The article is good, give it a‘Look’☟☟☟