【Abstract】 With the advancement and deepening of digital transformation in finance, organizations are beginning to consider longer-term and more cautious approaches when selecting cloud architectures. On one hand, they plan comprehensively from the perspective of enterprise-level, group-level, and industry-level overall development to avoid the emergence of isolated cloud or resource pools; on the other hand, adopting a self-controlled, innovative cloud architecture has become an important choice for many financial enterprises. This article analyzes and compares the types of data storage in cloud platforms and the selection of cloud storage routes.
【Author】Cheng Zongjing, Senior Engineer in Banking
Introduction
Currently, innovation has become a critical part of China’s socio-economic development, and during the promotion of the innovation industry, “cloud” plays a core role in bridging various aspects. With the arrival of the digital economy era, the financial sector’s dependence on information technology hardware and software is deepening, providing a good opportunity for the development of the cloud service industry. As the digital transformation in finance progresses, organizations are beginning to consider longer-term and more cautious approaches when selecting cloud architectures. On one hand, they plan comprehensively from the perspective of enterprise-level, group-level, and industry-level overall development to avoid the emergence of isolated cloud or resource pools; on the other hand, adopting a self-controlled, innovative cloud architecture has become an important choice for many financial enterprises. The following discusses the selection of innovative cloud platform storage routes.
1. Types of Data Storage in Cloud Platforms
From the perspective of data format, cloud platform data storage can generally be divided into three categories: block storage, file storage, and object storage. Different types of storage technologies have different characteristics and applicable scenarios in cloud platforms. Enterprises should choose the most suitable storage type based on their business needs and data characteristics to meet requirements for data security, high performance, scalability, and cost-effectiveness. However, in practical environments, the main distinctions based on usage scenarios include virtual machine storage, database storage, and unstructured data storage.
1. Virtual Machine Storage
Virtual machine storage refers to the storage used for virtual machine images, operating systems, applications, and user data. Generally, virtual machine storage is large, often exceeding 10GB, with low IOPS latency requirements for individual virtual machines, while availability requirements are high. The number of virtual machines in enterprises typically ranges from hundreds to thousands, and the substantial storage demand poses challenges for configuring cloud storage. From a usage perspective, virtual machine storage can be further divided into the following categories.
(1) Virtual machine storage, where virtual machine disks are the primary storage medium for saving the operating system, applications, and user data of virtual machines.
(2) Image storage, used for saving virtual machine images, including predefined virtual machine templates, operating system images, and application images.
(3) Snapshot storage, used for saving virtual machine snapshots, which are backups of the current state of the virtual machine and can quickly restore the virtual machine to a previous state when needed.
(4) Backup storage, used for saving backup data of virtual machines, which is a periodic backup of virtual machine disks and data to ensure data security and reliability.
(5) Cold storage, used for storing infrequently accessed virtual machines or data that need to be preserved for a long time.
When selecting a cloud platform storage technology route, it is essential to consider the relationship between demand and cost. The main factors to consider include the following aspects.
(1) Performance, as the performance of virtual machine storage directly affects the operational efficiency and response time of virtual machines. For applications requiring high performance, it is necessary to choose high-performance virtual machine storage, which incurs higher costs.
(2) Availability, virtual machine storage should have high availability to ensure the persistence of virtual machines and data; redundancy and backup mechanisms are key to ensuring the availability of virtual machine storage.
(3) Cost-effectiveness, different types of virtual machine storage solutions may have significant cost differences. Enterprises should choose the most cost-effective cloud platform storage solution based on actual needs and budget constraints.
Therefore, in actual construction, it is advisable to consider building a unified management of different categories of resource pools, with each resource pool supporting dynamic expansion and effectively improving the management and utilization efficiency of storage resources. Based on the usage characteristics of virtual machine storage, storage resource pools can utilize SSDs, SAS interface disks, or even SATA interface disks.
2. Database Storage
Database storage is a storage system used for storing and managing database data. Cloud platforms provide various types of database storage services, including relational databases, non-relational databases, and data warehouses. Cloud platform database storage typically supports multiple database engines to meet different application needs.
(1) Elastic scaling, allowing for elastic scaling based on business needs, automatically adjusting storage capacity and computing resources. This enables databases to adapt to the growing volume of data and access loads without manual intervention.
(2) High availability and redundancy, usually providing high availability and redundancy mechanisms, with data automatically replicated to multiple storage nodes or data centers to prevent data loss and catastrophic failures.
(3) Security and access management, emphasizing the importance of data security, providing features such as data encryption and access control, allowing for permission management of databases to restrict access for different users or roles.
When enterprises choose cloud platform database storage, they must first clarify their business needs. Different types of databases are suitable for different application scenarios; relational databases are suitable for transaction processing, while non-relational databases are better suited for handling large-scale unstructured data. Performance must also be considered, as relational databases typically have higher transaction processing capabilities, while non-relational databases excel at handling a large number of concurrent read and write operations. The high availability and redundancy mechanisms of cloud platform database storage are crucial for critical business operations. Enterprises need to ensure that the selected database storage can provide high availability and data backup capabilities to prevent data loss and business interruptions. Of course, from a cost control perspective, it is necessary to choose suitable database storage services based on actual budgets.
3. Unstructured Data Storage
NAS, distributed file storage, and object storage are technologies that can be used for managing unstructured data storage. If the data is not large in volume and has performance and high availability requirements, NAS storage is typically used, such as achieving persistent storage in a container cloud environment. If the data consists of large volumes, such as video data, or if the data volume exceeds PB levels, and the application supports the S3 protocol, object storage offers better scalability and lower maintenance costs, such as achieving log data archiving in a container cloud environment.
II. Main Cloud Storage Routes
1. Storage Virtualization
Storage virtualization abstracts the underlying physical storage devices into logical storage pools in a cloud computing environment, providing more flexible, efficient, and simplified storage resource management and configuration. Through storage virtualization, enterprises can achieve centralized management of different storage devices and aggregate them into a unified storage pool, thereby improving the utilization of storage resources and making data migration and adjustment more convenient. Its implementation is primarily a virtualization of traditional block storage (SAN). To better achieve unified management of heterogeneous storage, a unified storage gateway is generally used to implement storage virtualization.
The storage virtualization gateway is an intermediate layer that connects local storage with cloud storage, seamlessly integrating local storage resources with cloud storage services. It acts as a bridge between local storage and cloud storage, allowing enterprises to manage and access data from different storage systems through a unified interface. Data backup migration, replication, synchronization, centralized management, data security, and even protocol conversion can all be achieved through the virtualization gateway. The benefits of the storage virtualization gateway include helping enterprises make full use of existing local storage resources and extending them to cloud storage, thus achieving efficient management and utilization of storage resources. However, it also introduces performance bottlenecks under massive concurrent data requests, especially in data transmission and processing; introduces single points of failure, as the failure of the gateway will affect the availability of the entire storage system; and increases costs, as procurement and configuration management of devices increase the enterprise’s investment.
In summary, the main usage scenarios for storage virtualization are still in traditional IT environments and private cloud environments. Naturally, it is aimed at important application business environments with high IO loads and low read/write latency, such as databases. It is an upgrade and expansion of traditional centralized storage.
2. Distributed Storage
Distributed storage is in contrast to centralized storage. It disperses data across multiple nodes or devices rather than storing it centrally in a single storage device. In cloud platforms, distributed storage is a common data storage method, particularly suitable for large-scale data storage and processing scenarios.
Compared to centralized storage virtualization, distributed storage includes: ① High availability, as distributed storage disperses data replicas across multiple nodes, when a node fails, other nodes can continue to provide data access and services, thus ensuring system availability; ② Performance improvement, allowing data to be accessed in parallel across multiple nodes, thereby increasing data read and write speeds and overall system performance; ③ Data redundancy, as data is stored in multiple nodes with multiple replicas, providing data redundancy to prevent data loss and damage. However, the system is relatively complex; ensuring data consistency poses certain challenges, as maintaining consistency among multiple replicas is a significant challenge for distributed storage systems, and the requirement for strong consistency among multiple replicas can introduce network latency issues.
Therefore, distributed storage is more suitable for scenarios requiring large-scale data storage and complex data processing, such as big data analysis and artificial intelligence applications; it can also be used in cloud platforms to support the data storage and access needs of a large number of users. Distributed storage and object storage are inclusive, with distributed storage being a physical organization form of storage nodes, and distributed storage can provide block, file, and object storage services. Object storage is a model based on objects, assigning each object a unique identifier, unlike traditional file systems that store data in a hierarchical directory structure. In distributed object storage services, data is dispersed across multiple nodes or devices, achieving high availability and reliability of data.
3. Network Shared File Storage
Here, we mention NAS data storage, which is also commonly used in cloud platforms. NAS storage is specifically designed for storing shared data, providing file-level storage and access services to multiple users and clients over a network. It typically uses independent hardware devices as a form of cloud service.
In practical applications, some data storage scenarios are more suitable for Ethernet transmission and are better suited for file systems as carriers. Achieving cross-regional transmission of data from storage virtualization is exceptionally challenging in terms of cost and flexibility. Therefore, when selecting storage technology routes, it is essential to consider the storage needs of this data.
In summary, selecting data storage routes in cloud platforms is a critical and complex task, involving aspects such as data security, performance, scalability, and cost-effectiveness. First, it is essential to clarify the types of data for enterprises or users, including structured data, unstructured data, and semi-structured data. Determine the data access frequency, number of concurrent users, and read/write operations, as these factors will affect the performance requirements of the storage system; consider data security, especially for sensitive data; based on data characteristics and needs, different types of storage technologies can be selected, such as storage virtualization, NAS network storage, or distributed storage. Additionally, consider whether to use local storage devices or cloud storage services, or a hybrid storage approach combining both; consider performance requirements, and select appropriate storage performance based on data access needs and application requirements, including read/write speeds, IOPS, etc.
If you find this article useful, pleaseshare, likeor click“♡” so that more peers can see it
Recommended materials/articles:
-
What is the relationship between containers, container clouds, and containerized PaaS platforms?
-
Consensus framework for evaluating the selection of innovative cloud bases in the financial industry (12 dimensions to comprehensively assess cloud base capabilities, available for download)
-
Comparison of the cloud frameworks of the three major domestic cloud vendors (Huawei, Tencent, Alibaba)
-
Practice sharing on the construction of a “local dual-active + remote asynchronous replication” disaster recovery architecture based on NAS dual-active architecture for over 100 business systems in a city commercial bank
-
Construction of a private cloud platform 2.0 for a certain bank’s headquarters data center
Welcome to follow the community “Cloud Computing”technology theme , which will continuously update high-quality materials and articles. Address:
https://www.talkwithtrend.com/Channel/77/
Download the twt community client APP
Long press to recognize the QR code to download
Or search for “twt” in the app store
Long press the QR code to follow the public account
* The content published by this public account only represents the author’s views and does not represent the community’s position