From discussions within the twt community, we welcome more peers to join the conversation.
What are the pros and cons of using RDM raw disk mapping versus VMFS for storage?
If building a database RAC on a virtualization platform with shared storage, what are the pros and cons of using RDM raw disk mapping versus VMFS?
This question comes from community member @wangdd, a systems engineer at Hebei Provincial People’s Hospital. The following content is based on the shared practical experiences of many peers in the twt community, and everyone is welcome to participate in the discussion and express their opinions.
* The content of the “Controversy” column comes from firsthand experiences and observations shared by peers and represents personal opinions.
@Zhao Hai, Technical Manager:
RDM: Raw disk mapping is generally used in Oracle RAC virtualization scenarios.
VMFS: Virtual File System can be used in most virtual machine disk scenarios.
In VMware environments, what is the need to support RDM? The main purpose is to support shared virtual machine disks across multiple VMs, especially in scenarios where read/write performance is critical, such as Oracle RAC. If this method is used, some features of VMware are lost, such as VM migration and control parameters for disk I/O. Additionally, it brings many operational challenges to cloud environments, such as batch migrations being hindered by RDM on a particular server, making it impossible to execute migrations in bulk.
Generally, when implementing business scenarios on VMware platforms, we consider flexibility and high availability more. From a performance perspective, even if we use RDM, there are still performance bottlenecks due to the VM. Therefore, for heavyweight OLTP workloads, it is more reasonable to consider physical machines as resource pools to support RAC, even if it is through a 12C method to form a DB resource pool from physical machines. If you really want to move the database to VMs on the VMware platform, it is recommended to choose less critical databases with lower loads and smaller data volumes, and there is no need to implement RAC.
For other business scenarios, there is even less need to use RDM instead of VMFS.
@Zhang Peng, Senior Technical Supervisor at China Financial Electronic Corporation – Disaster Recovery Service Center for Small and Medium Financial Institutions:
Some believe that RDM directly maps LUNs from SAN to virtual machines, which can improve performance to some extent, allowing for the deployment of relatively high I/O database services. However, in reality, the difference is not significant.
Moreover, RDM has some limitations.
There are two types of RDM: Virtual Compatibility Mode RDM and Physical Compatibility Mode RDM. Especially, Physical Compatibility Mode RDM has some significant limitations: for example, VMware snapshots cannot be used, VCB is not supported because VCB requires VMware snapshots, physical mode RDM cannot clone virtual machines, and using physical mode RDM in templates cannot convert virtual machines. If migration includes disk copying, physical mode RDM cannot migrate virtual machines, and physical mode RDM cannot use VMotion functionality. Virtual mode RDM can solve these issues, allowing the original LUN to be seen as a virtual disk and enabling features like VMotion, snapshots, and cloning. If RDM is needed, virtual mode RDM is more popular in most cases. Virtual mode RDM can be used for V2V clustering across physical hosts. However, note that P2V migration across physical hosts requires using physical mode RDM.
@zyp8365, Database Administrator at Guangdong Provincial Hospital of Traditional Chinese Medicine:
Personally, I do not recommend building a database RAC on a virtualization platform. The virtualization platform is a cloud computing solution designed to fully utilize server computing and memory resources, effectively addressing the siloed development of IT information. The main goals of database RAC are to achieve high reliability with dual machines and load balancing for business. The virtualization platform itself consists of multiple servers and has its own HA mechanism. Additionally, there are many ways to protect virtual machines on the virtualization platform, such as VMware’s FT (although there are CPU limitations), continuous data protection mechanisms like RP4VM, and various backup and recovery methods that can achieve high reliability without relying on RAC. If you want to implement disaster recovery, you should consider using Data Guard.
Furthermore, regarding business load balancing, it is even less advisable to implement this on a virtualization platform. The contradiction between RAC’s multiple virtual machines and the virtualization platform’s multiple virtual machines makes such operations inadvisable. Two physical machines would be better.
Another important reason is that although Oracle RAC can theoretically be implemented on virtualization platforms, it is not recommended because the additional virtualization layer complicates troubleshooting and leads to suboptimal information flow. Returning to the original question, the most important difference between RDM and VMFS is that RDM is a direct pass-through, while VMFS goes through a virtualization layer and uses the virtualization platform’s file system format.
The pros and cons are that with RDM, data goes directly to the disk, providing a shorter path and faster speed, but because it does not go through the virtualization platform, management is cumbersome, and some advanced features of the platform cannot be utilized. In contrast, VMFS is the opposite; data must be parsed through the virtualization platform before reaching the backend disk, resulting in a longer path and relatively slower speed, but management is simpler and more convenient, allowing the use of advanced features of the virtualization platform for disks.
@wuliangy, Engineer in the Information Engineering Department at Zhejiang Cancer Hospital:
I personally support exploring the new approach of building RAC on virtualization layers.
Before 2019, the common way to build database RAC was through physical machines + optical interconnect + storage (active-active) for several reasons:
1. Oracle does not recognize RAC built on virtualization; if issues arise, Oracle does not provide support.
2. This model has been tested over time and is stable and reliable.
3. Building on virtualization introduces an additional virtualization layer, raising questions about overhead.
4. Many medical institutions may use small machine RACs and do not have high trust in X86.
However, since 2020, at least in the Zhejiang region, more and more institutions have begun to attempt building databases on virtualization, even adopting RAC over vSphere for the following reasons:
1. Since September 2019, Oracle and VMware have recognized each other, allowing for direct case formation, gaining recognition from the original manufacturer.
2. The performance and stability of X86 hosts have reached a high level, with single-machine computing performance fully capable of bearing the database pressure of large institutions.
3. Many large medical institutions have core business databases such as HIS DB, LIS, EMR, PACS, and integration platforms beyond just the database. If databases are deployed virtually, hardware resources can be utilized more efficiently.
4. The price drop of all-flash storage and all-flash hyper-converged hardware has opened up many possibilities, and practical tests show that the I/O and latency of virtual machine RAC fully meet requirements.
Returning to the main question, regarding RDM and VMFS, one is raw device mapping, which has a shorter storage path and is more efficient, while the other is a virtualization file format. However, today’s all-flash storage has reached a high level of performance, and practical tests show no significant difference. If you want to utilize VM features, it is recommended to use VMFS for mounting, or as a compromise, use RDM for database VMs to maximize performance, while allocating remaining storage resources to VMFS for efficient utilization.
@cpc1989, Storage Engineer at an Insurance Company:
My personal view is that in most scenarios, there is no significant difference between using RDM and VMFS for storage access on virtualization platforms. However, VMFS is more compatible with virtualization platforms, while RDM is mainly used in very specific scenarios requiring SAN raw disk mapping, and such usage scenarios often lack stability.
In the deployment of database RAC, from a technical architecture perspective, physical machines are more suitable for deploying database RAC. If server computing resources are abundant, multiple databases can be integrated to achieve effective resource utilization. The virtualization platform’s deployment of RAC does not align with the high availability characteristics of virtualization clusters. From practical experience, under high load conditions, system stability may decrease. If the load is not high and system availability requirements are low, deploying a single machine with database backup or single machine with database replication on a virtualization platform is acceptable; otherwise, a physical machine deployment is more recommended.
@Wang Qiaolei, Systems Engineer at Sino-Bridge:
VMware currently supports both VMFS and RDM general modes, with RDM further divided into virtual compatibility and physical compatibility modes.
The characteristics of these modes are as follows:
(Click the image to enlarge)
Based on the above explanations and some project experiences, my personal opinion is:
1. VMFS leans towards VMware’s perspective, covering most scenarios, especially under current hardware surplus conditions, VMFS can maximize the use of all advanced features of VMware, such as snapshots and VMotion, which can bring great convenience.
2. RDM leans towards the storage perspective and has its corresponding applicable scenarios, such as needing direct exchange with storage management hosts or having special SAN requirements; needing to use advanced storage features (snapshots, replication, cloning, etc.); using NPIV functionality in virtualization environments, etc.
Of course, some say that since it has come to this, it is better to use physical machines directly. However, some applications do not have high computing resource demands, making virtualization a more cost-effective and manageable option. For applications with particularly high computing and storage demands, such as Oracle RAC, I personally still prefer using physical machines. It ultimately depends on the requirements.
@hacmp, Systems Engineer at Sichuan Huaxin Fuheng:
I believe it depends on the business. Currently, most applications running on virtualization platforms are lightweight. Even if there are many databases, the I/O is not very high, and VMFS can fully meet the needs. However, it is essential to plan VMFS reasonably. If the database is core, it is still recommended to run RAC on physical machines. RDM is indeed a bit awkward; although it performs better than VMFS, I have encountered few cases where RDM was used.
I have worked on a business system that stored images, which used RDM and had dozens of TB.
@Liu Dong, IT Technical Consultant at Neusoft Group:
Deploying a database RAC on a virtualization platform built with shared storage is technically feasible. However, from a performance and management perspective, it is not recommended. If you must use a virtualization platform to deploy Oracle, it is advisable to deploy only a single machine or Oracle DG, as Oracle RAC is a clustering and high availability technology, while the virtualization platform itself has HA, VMotion, and other high availability technologies, which can conflict.
Although the high availability technology of the virtualization platform cannot achieve the fault-tolerant switching of RAC, implementing RAC on a virtualization platform does not yield good results.
First, from a performance perspective, the additional virtualization layer prevents full utilization of the physical machine’s hardware performance, including CPU and I/O performance, which will be affected. Secondly, if the two selected virtual machines drift and end up on the same physical server, the deployed Oracle RAC becomes meaningless. If this physical machine fails, both RAC nodes will go down simultaneously, so it is better to deploy a single machine and utilize the high availability technology of the virtualization platform to provide services, or directly deploy RAC on physical servers.
Regarding RDM raw disks and VMFS, if you insist on using a virtualization platform for RAC, I personally recommend VMFS or virtual RDM as a better choice, as it allows the use of disk management features of the virtualization platform, such as snapshots and migrations. If you directly use physical RDM, it is better not to use a virtualization platform at all and directly use physical machines with SAN storage, as there is no need to deploy on a virtualization platform. Since you are deploying RAC on a virtualization platform, you should fully utilize the features of the virtualization platform.
Additionally, if conditions allow, it is best to configure the hardware of the virtualization platform where RAC is deployed to be higher, such as using SSDs and 10G core networks, because RAC differs from single-machine deployment modes. The cluster nodes of RAC will generate a large amount of data exchange and synchronization. Without 10G networks and SSD support, problems can easily arise, which will also affect other business systems deployed on the virtualization platform. This point needs special attention.
We welcome you to click at the end of the articleto read the original text in the community for discussion and to express your views.
If you find this article useful, please share, like or click to read more, so that more peers can see it.
Recommended materials/articles:
-
Essentials! 68 Selected Articles Related to Storage | Weekend Material Delivery
-
All About Oracle RAC
We welcome you to follow the community’s “Storage” technology theme , which will continuously update quality materials and articles. Address:
https://www.talkwithtrend.com/Channel/179
Download the twt community client APP
Long press to recognize the QR code to download
or search for “twt” in the app store
Long press the QR code to follow the public account
* The content published by this public account only represents the author’s views and does not represent the community’s position.