|
The “Performance Evaluation Action for Typical Server Application Scenarios” has officially launched!—— In order to standardize the construction of a full-stack standard system, promote product technology iteration and upgrade, better showcase the advantages of the entire architecture and computing power, and support excellent products and solutions to enhance their brand and market expansion through official authoritative certification, the Green Computing Industry Alliance’s “Performance Evaluation Action for Typical Server Application Scenarios” (abbreviated as: Evaluation Action) has recently been officially launched! This evaluation action includesbig data, distributed storage, web applications, high-performance computing, centralized relational database management systems, ARM native cloud phones, and virtualization seven typical server application scenarios. |
Today, we bring you a detailed interpretation of “Performance Testing Methods for Server Application Scenarios: Centralized Relational Databases”.
The drafting units of “Performance Testing Methods for Server Application Scenarios: Centralized Relational Databases” includeThe China Electronics Standardization Institute, Beijing Mass Data Technology Co., Ltd., Yunhe Enmo (Beijing) Information Technology Co., Ltd., Wuhan Yangtze River Computing Technology Co., Ltd., Huawei Technologies Co., Ltd., Arm Technology (China) Co., Ltd., and Anshengpei Semiconductor Technology Co., Ltd., we thank the above units for their strong support.
01
Background
With the gradual implementation of applications and the rapid rise of the industry, the importance and scale of data are continuously increasing in the tide of informatization development. The process of digital industrialization and industrial digitalization is accelerating, and the introduction of new infrastructure and other industrial policies has sparked a new wave of computing infrastructure construction. Modern data centers, driven by market demand, have entered a period of rapid development. The scale of data and computing in centralized database applications is continuously increasing, raising higher requirements for the functionality and performance of data center infrastructure.
The database is the core of information systems, and the entire industry mainly includes three sub-sectors: database products, technical services, and support systems. China’s database market is entering a golden development period. Due to the long development cycle, high technical difficulty, and challenges in ecological construction, the development of database products has become a “bottleneck” problem for basic software. Under policy guidance, China has begun to carry out iterative upgrades of self-controllable database products, which will accelerate the industry transformation of databases in China.
According to Gartner, the global database market size was $64.8 billion in 2020. In the future, the database market size will continue to grow, expected to reach $100 billion by 2024. In China, with the explosive growth of data scale and the rapid deepening of data applications, as well as the rapid development of the information technology industry and the deep layout of new infrastructure, China’s database industry is entering a significant development opportunity period. According to estimates by the China Academy of Information and Communications Technology, the size of China’s database market was approximately 24.1 billion yuan in 2020, and it is expected to reach 68.8 billion yuan by 2025, with a compound annual growth rate of 23.4%.
From the demand side, the information construction needs of industry customers are continuously increasing, and the willingness to pay for genuine basic software has significantly improved; from the supply side, database products are continuously innovating and iterating, and favorable policies are being introduced. With the joint action of both sides, the domestic database market is thriving.
02
Testing Methods
The performance testing process of centralized relational databases is conducted using open-source, general-purpose, and industry-recognized performance testing tools as clients to generate load through multiple concurrent operations, systematically evaluating the overall performance of centralized relational databases on different architectures with equal computing power of the CPU.
Among them, simple load testing and batch processing testing, except for single-row inserts, all other scenarios use files to pre-load data into the target table, with a data volume of 100 million, using JMeter as the client tool. In simple load testing, the performance metric for equality query is QPS (queries per second), while the performance metrics for single-row inserts and updates are TPS (transactions per second);
In batch processing testing, the local command line is used to execute batch testing commands, and the performance metrics for batch loading, updating, and deleting are measured in seconds (response time), meaning that the performance of batch processing testing is measured by execution time. The average value is taken from three measurements.
The testing method for transaction performance refers to the TPC-C benchmark, using the BenchmarkSQL tool to pre-generate data and conduct stress tests. In transaction performance testing, the tpmC value (the number of new orders processed per minute) is used as a performance metric, with the average value taken from three measurements.
Test cases are selected based on different data structure definitions for various testing scenarios to simulate business patterns in production, evaluating the overall performance metrics of the database under different concurrent loads, thereby assessing whether centralized database products can meet future business performance requirements.
-
Test Configuration Description
This test used two Arm servers and two X86 servers for comparative testing. The Arm servers under test used Beijing Mass Data Technology Co., Ltd.’s centralized database product Vastbase G100 with the domestic openEuler operating system version 20.03, while the X86 server is a Dell R460, also using the Vastbase G100 centralized database product.
Detailed server and hardware configuration information is shown in the table below:
03
Testing Environment and Results
|
Testing Environment Introduction—— This evaluation action is undertaken by the China Electronics Standardization Institute. The alliance collaborates with the China Electronics Standardization Institute to create the open laboratory “GCC Open Lab”. Participating units can apply for the professional typical application scenario hardware and software compatibility testing environment from the China Electronics Standardization Institute, or they can build their own testing environment, with experts arranged by the institute for remote guidance and review to ensure authoritative processes and results. |
All test data or functional implementation results in this report come from the joint laboratory of the Green Computing Industry Alliance, providing objective and fair testing data while ensuring that the testing environment is as similar as possible.
The test results demonstrate that the Vastbase G100 has been optimized for multi-NUMA node processor architecture characteristics, ARMv8 instruction set, and other aspects, involving a series of multi-level optimizations from the operating system, software architecture, lock concurrency, logging, atomic operations, and cache access, significantly improving the processing performance of the Vastbase G100 database on multi-NUMA node processors. Running on Arm servers allows for better utilization of multi-core high concurrency performance advantages compared to the x86 platform.
The NVMe drives used in the test also greatly enhanced system performance.
04
Conclusion and Outlook
From the testing results of this centralized relational database, the green computing servers based on Arm architecture are highly competitive compared to traditional architectures. In terms of overall solutions, green computing servers, when combined with commercial software solutions, demonstrated significant advantages over traditional architecture servers combined with open-source software solutions in this evaluation.
In the future, the Green Computing Industry Alliance will continue to promote typical computing business scenario performance evaluations, drive joint optimization of underlying server hardware and upper-level application software, and reflect the overall competitiveness of full-stack solutions. Additionally, the alliance will collect more industry needs from alliance members, incorporate more typical server application scenarios (such as edge computing, etc.), and more green computing server configurations, making typical application scenario evaluations a core competitiveness of the Green Computing Industry Alliance.
Furthermore, the alliance will conduct evaluations and research on green computing servers from more dimensions, such as artificial intelligence servers, green energy-saving (power management), etc. The alliance will also study the system and mechanism for promoting the implementation of alliance standards through evaluation certification, and carry out preliminary research on the latest technology standards for green computing and the upgrading of testing environments to lay the foundation for subsequent standard compliance testing certification services.
The Green Computing Industry Alliance will continue to promote certification activities for green computing technology products, services, and management, enhancing the compliance, compatibility, and quality consistency of products and services, and facilitating product interoperability, thereby building trust across the entire industry chain. The alliance is committed to encouraging end users and relevant parties to trust the certification results based on alliance standards, fully reflecting the value of standards and ensuring their practical application!
|
Welcome to register for the “Evaluation Action”! Boost brand enhancement, customer recognition, and business opportunity acquisition——
|
|
Don’t hesitate… participate in the “Evaluation Action”! How to Participate? The “Performance Evaluation Action for Typical Server Application Scenarios” has received enthusiastic responses from alliance members since its launch. The first batch of products and solutions that pass the evaluation will be officially released at the “Green Computing Industry Alliance White Paper and Standard Results Release Conference” in July. Additionally, the Green Computing Industry Alliance GCC will provide marketing support for evaluated products through various means such as “continuous online publicity, release of evaluation lists, and excellent case tours”.
|
Click below on “Read the Original” to visit the alliance’s official website and download the standard document GCC-7006-2022 “Performance Testing Methods for Server Application Scenarios: Centralized Relational Databases”.
Thank you for your long-term attention and support for the Green Computing Industry Alliance GCC! The alliance values your thoughts and feelings, and we welcome you to scan the QR code above to enter the alliance survey questionnaire, providing valuable feedback and suggestions for our work, and work together with the alliance to build a new development in the green computing industry! (This questionnaire will close on August 10, 2022, and after it ends, the alliance will draw lucky winners to receive exquisite gifts!) Come scan the code to participate~~
——— End of Full Text ———
About the Green Computing Industry Alliance Since its establishment in 2016, the Green Computing Industry Alliance aims to collaboratively build a green, open, independent, and shared ecological system, dedicated to promoting the development of the green computing industry, constructing a platform for industrial exchange and cooperation, enhancing enterprises in fields such as PCs, servers, storage, operating systems, databases, and promoting win-win cooperation in the computing field. It has now become a global alliance with the most complete partners of Arm infrastructure server chips, including Kunpeng, Feiteng, Ampere, Marvell, etc.
