With the rapid development of digital construction and transformation in China, the scale and volume of data have been increasing year by year. Coupled with the trend of open source and cloud deployment, more and more organizations are considering replacing their existing traditional databases to meet new and more complex business requirements.
The replacement of databases inevitably brings about massive data migration and a large amount of data processing program migration and transformation work. How to achieve smooth and seamless migration while ensuring business continuity and data consistency has become an important factor that enterprises must consider when selecting a database replacement.
Minsheng Bank Embraces openGauss
As the first national commercial bank established by private capital in mainland China, Minsheng Bank has always adhered to its original intention and mission of “born for the people, coexisting with the people,” committed to providing customers with professional and distinctive modern financial services, and exploring the path of modern commercial bank construction for the Chinese banking industry.
It is well known that users’ financial status, transaction records, and various data information are the “lifeline” of banks, and ensuring their security is of utmost importance. At the same time, as business continues to diversify, complicate, and move to the Internet, the centralized traditional commercial database originally deployed by Minsheng Bank is struggling to bear the sudden peak pressure and cannot cope with challenges such as mixed loads. Additionally, it is costly and difficult to scale, and relying on resource stacking to ensure business continuity and stability is no longer sustainable. To ensure data security and achieve cost reduction and efficiency improvement while addressing new business challenges, Minsheng Bank conducted rigorous testing and evaluation of candidate products and ultimately chose the high-performance openGauss for the critical business scenario database replacement transformation.
Database Migration Keywords: Compatibility, Stability, Continuity
After determining that openGauss would replace the original database, a challenge facing Minsheng Bank was how to efficiently achieve the complete migration of database objects and full data between heterogeneous databases.
According to Minsheng Bank’s 2021 annual report, the number of retail customers (referring to normal status individual customers [including Class I, II, and III accounts], pure credit card customers, and micro-enterprise legal entity customers) reached 110.1378 million, an increase of 5.41% compared to the end of the previous year. Coupled with other types of customers, the total volume of business data at the bank is considerable. If migration relies solely on manual processes, the workload would be enormous and would inevitably impact the normal operations of the company. Therefore, customers need mature migration tools to enhance the efficiency of heterogeneous database migration.
Minsheng Bank primarily migrates data from Oracle, MySQL, DB2, and other databases to openGauss. The client requires the migration tool to adapt well to different types of databases and integrate into the openGauss ecosystem. It is understood that the amount of data to be migrated from various systems ranges from tens of GB to tens of TB. After migration, all data must be complete and accurate to ensure usability. Additionally, to better serve bank customers, the migration process needs to ensure system stability and business continuity.
Of course, Minsheng Bank also encountered some common challenges in database migration, such as SQL syntax conversion. During the migration process, SQL-related database objects, such as tables, storage, functions, etc., may face compatibility issues, and the migration tool must accurately identify incompatible SQL.
MTK Safeguards Smooth Replacement for openGauss
In fact, to ensure that the database replacement proceeds steadily and far, Minsheng Bank adopts a prudent three-party cooperation model for system construction. The bank is responsible for database product testing, application transformation, and online operations and maintenance; the openGauss community mainly undertakes tasks such as core function development and ecosystem construction; Cloud and Enmo, as a third-party vendor, is responsible for product defect repair, technical support, and peripheral tool development, with MTK being one of them.
MTK is the database migration tool (Database Migration Toolkit) that supports the MogDB database of Cloud and Enmo. Since MogDB is an enterprise-level relational database commercial distribution based on the openGauss kernel, MTK has a natural advantage in compatibility and adaptation for migration tasks targeting openGauss as the destination database. In addition, MTK supports database migration from Oracle, DB2, and MySQL, perfectly matching Minsheng Bank’s migration needs and becoming a powerful tool for promoting customer database replacement.
After MTK was deployed in the Minsheng Bank system, the migration task could be completed with only six commands, specifically:
1. Configuration file check: ./mtk -c config.json config-check, check if the file has syntax errors.
2. Pre-migration test: ./mtk -c config.json –preRun, check for potential issues during the migration process in advance, estimate the time required to complete the entire migration task, and coordinate the business pause time window.
Next, Minsheng Bank performs step-by-step migration according to its business needs, achieving accurate problem location during the migration process and reducing the impact of migration tasks on the system.
3. Table structure: ./mtk -c config.json mig-tab-pre
4. Data migration: ./mtk -c config.json mig-tab-data
5. Index/constraint object migration: ./mtk -c config.json mig-tab-post
6. Stored procedure/function migration: ./mtk -c config.json mig-tab-other
To address the SQL syntax conversion challenges in database migration, MTK has embedded multiple SQL rules and has achieved the conversion of most syntax. Additionally, to allow users to intuitively understand the migration status and any issues encountered, MTK generates an HTML report after configuring the reportFile parameter, which contains detailed content, showing each SQL statement during the migration process, allowing users to directly view and modify problematic SQL.
The work of migrating data from Oracle, DB2, and other databases to openGauss using MTK has been recognized by Minsheng Bank. During the entire migration process, MTK performed stably, maintaining a migration speed of 1TB per hour, ensuring data consistency and smooth business operations. As of the end of 2022, Cloud and Enmo has completed the migration of 100 sets of heterogeneous databases to openGauss for Minsheng Bank.
In the details lies the effort, and in the utmost pursuit lies the excellence. Data is the top priority of business systems, and ensuring zero data loss, immediate data retrieval, and data security is key in the database migration process. In fact, in addition to MTK, Cloud and Enmo’s Migrate to openGauss / MogDB solution also includes SQL compatibility assessment tools SCA before heterogeneous database migration, data consistency verification tools MVD, and heterogeneous data synchronization tools MDB. Through continuous innovation of the MogDB database and its supporting migration solutions, Cloud and Enmo will continue to promote the smooth progress of Minsheng Bank’s database replacement work.
The 12th “Data Technology Carnival” (DTC 2023), co-hosted by the China DBA Alliance (ACDU) and the Mo Tianlun community, will be grandly held on April 7-8 at the Crowne Plaza Hotel in Beijing. The conference gathers many excellent vendors, advanced technologies, outstanding products, and excellent cases, with leading figures, academic elites, technical experts, industry practitioners, and ecological evangelists from the data field bringing over 60 keynote speeches. Experts from Minsheng Bank will also share their insights at this conference. When purchasing tickets on the conference official website (click the “Read the original text” in the lower left corner), you can use the instructor’s exclusive discount code to get tickets for 0 yuan (limited quantity, please register as soon as possible). We look forward to seeing you at the conference!
Hou Yu
Database Expert at China Minsheng Bank
Venue丨Innovative Application: Database Migration Practice
Title丨Building a Tool System for Innovation at Minsheng Bank
Exclusive Discount Code丨HOUYU
Kong Zaihua
Senior Database Expert at China Minsheng Bank
Venue丨Financial Technology: Financial Industry Practice
Title丨Application Practice of openGauss at Minsheng Bank
Exclusive Discount Code丨KONGZH
Data-driven, achieving the future, Cloud and Enmo, living up to expectations!
Founded in 2011, Cloud and Enmo is a smart data technology provider with the mission of “data-driven, achieving the future.” We are committed to bringing data technology to every industry, every organization, and every individual, building an intelligent future driven by data.
Cloud and Enmo provides reliable products, services, and solutions in areas such as data carrying (distributed storage, continuous data protection), management (database software, database cloud management platform, data technology services), processing (application development quality control, data model control, digital transformation consulting), and application (data service management platform, data intelligence, privacy computing data federation platform), continuously creating value for customers around user needs, stimulating data potential, and striving for an agile and efficient digital world in the future.
