Guidelines and Best Practices for Virtualization Planning

This article discusses how to plan computing resources, storage resources, network resources, availability, and management and monitoring in VMware-based virtualization construction. Shared by skey_deng.

1. Advantages of Virtualization

Virtualization technology can greatly reduce the number of devices that need to be maintained and managed. Virtualization can improve resource utilization, reduce hardware procurement costs, and is more energy-efficient and space-saving.

After server virtualization, we can manage through a virtualized cluster. It makes full use of existing devices while allowing for vertical and horizontal scaling of new devices.

The in-depth application of virtualization will dynamically increase the scale of the virtualized cluster, forming a healthier IT architecture.

On the client side, the original access mode will not change, and there will be no adverse impact on the business system, allowing for a smooth migration of existing systems.

The advantages of virtualization compared to traditional architecture are summarized as follows:

Planning work is easier to control
Relatively low hardware investment
High resource utilization, easy integration
Low server power consumption, minimal associated infrastructure consumption
Fast application deployment, short cycle for new business upper limits
Excellent business continuity, strong disaster resistance

2. Applicability of Virtualization

The applicability of virtualization platforms is strong; except for a few types of applications that are not suitable for virtualization platforms, the rest can be applicable.

Applications not suitable for virtualization

Applications with special hardware access requirements

Requires high-performance graphics cards

Not suitable for virtualization, such as OCR text recognition systems
Special serial/parallel encryption devices

Not suitable for virtualization, such as unified encryption platforms
USB device connection requirements

May not be suitable; external USB devices can be used instead, but need testing, such as certain monitoring systems using USB tokens for authorization

Applications with high loads even on high-spec servers

May not be suitable; current server configuration needs to be analyzed

3. Virtualization Construction Planning and Analysis

Virtual machines run applications and services that support various users and the entire business line, many of which are critical business applications, so it is essential to correctly design, allocate, and manage virtual machines to ensure these applications and services can operate efficiently.

3.1 Computing Resource Planning

VMware ESXi hosts are the basic computing building blocks of the data center. These host resources can be aggregated to create a highly available dynamic resource pool environment, serving as the overall computing resources for various applications in the data center.

This section mainly introduces the requirements for overall planning of computing resources, including physical servers, virtual machines, and other resources.

Guidelines and Best Practices:

Unless multiple virtual CPUs (vCPUs) are really needed, default to one, using as few virtual CPUs as possible. The operating system must support symmetric multiprocessing (SMP) functionality. Applications must be multithreaded to benefit from multiple virtual CPUs. The number of virtual CPUs must not exceed the number of physical CPU cores on the host.

Do not plan to use all CPU or memory resources of the host; reserve some available resources in the design. To optimize virtual machine memory performance, it is critical to retain the active memory of the virtual machine in physical RAM and avoid over-allocating active memory.

Always keep transparent page sharing enabled, always load VMware Tools, and enable memory reclamation.

Resource pool CPU and memory shares settings cannot be used to configure virtual machine priority. Resource pools can be used to allocate dedicated CPU and memory resources to virtual machines.

Configure vSphere DPM in environments where workloads are highly variable to reduce energy consumption and heat costs.

Deploy one system disk and one separate application data disk. If the system disk and data disk require the same I/O characteristics (RAID level, storage bandwidth, and latency), they should be placed together in one datastore.

Application requirements should be the primary indicator for allocating resources to virtual machines.

Protect virtual machines’ security as you would for physical machines. Ensure antivirus, anti-spyware, intrusion detection, and firewalls are enabled for each virtual machine in the virtual infrastructure. Ensure all security measures are updated regularly. Apply appropriate latest patches, keep virtual machine software and applications up to date, and use patch management tools or install and configure Update Manager. Intrusion detection and firewalls can be configured on the network.

To avoid management connection issues, assign static IP addresses and hostnames to each ESXi host. For management convenience, configure DNS with hostnames and IP addresses for each ESXi host; if cost is a concern, a simple DNS service can be set up in a VM.

Ensure the data center has sufficient power and cooling capacity to avoid service interruptions.

Regardless of the hardware platform chosen, a consistent platform configuration should be designed, especially in VMware clusters. Consistency includes CPU type, memory capacity and memory slot allocation, network card and host bus adapter type, as well as PCI slot allocation; refer to VMware’s compatibility list.

Use one or more clusters enabled with vSphere HA and DRS to increase availability and scalability.

Whether to use horizontal or vertical scaling of the cluster is determined by factors such as cluster purpose, infrastructure size, vSphere limitations, and funding and operating costs.

The computing capacity of each server should be estimated based on peak rather than average values to ensure it can support the maximum value of virtual machine applications running simultaneously.

CPU and memory estimates need to reserve 20% of space for burst computing capacity.

Memory sharing should be estimated at a rate of 50%.

Use the following formula to estimate the number of ESXi hosts required to meet the normal operation of virtual machines at CPU peak in the data center: Total CPU peak frequency demand of all virtual machines / Available CPU per ESXi host = Number of ESXi hosts required.

Use the following formula to estimate the number of ESXi hosts required to meet the normal operation of these virtual machines at memory peak in the data center: Total peak memory demand of all virtual machines / Available memory per ESXi host = Number of ESXi hosts required.

To use the high availability features of vSphere, we need to consider increasing the number of ESXi hosts accordingly.

3.2 Storage Resource Planning

Correct storage design has a positive impact on organizations achieving business goals and can lay the foundation for a well-performing virtual data center. It can protect data from malicious or accidental destruction while preventing unauthorized users from accessing data. Storage design must be reasonably optimized to meet the diverse needs of applications, services, administrators, and users.

The goal of storage resource planning is to strategically coordinate business applications with storage infrastructure to reduce costs, improve performance, enhance availability, provide security, and enhance functionality.

This section provides overall planning for storage resources, including shared storage logical planning, storage space planning, storage I/O control planning, and storage tiering planning.

Guidelines and Best Practices:When planning storage resources, we will follow the following guidelines and best practices.

Build a modular storage solution that can continuously expand over time to meet organizational needs without requiring users to replace existing storage infrastructure. Both capacity and performance should be considered in the modular storage solution.

Each storage tier has different performance, capacity, and availability characteristics; as long as not every application requires expensive, high-performance, highly available storage, designing different storage tiers will be cost-effective.

Configure storage multipathing and redundancy at the host, switch, and storage array levels to improve availability, scalability, and performance.

Allow all hosts in the cluster to access the same datastore.

Configure storage DRS to balance based on usage and latency.

For Fibre Channel, NFS, and iSCSI storage, appropriate designs should be made to reduce latency and improve availability. It is particularly important to distribute workloads to different locations for workloads that process a large number of transactions per second (e.g., data collection or transaction logging systems). Reduce latency by minimizing the number of hops in the storage path.

The maximum capacity of NFS storage depends on the array vendor. The capacity of a single NFS datastore depends on the space required by each virtual machine accessing the datastore, multiplied by the number of virtual machines that can access the datastore within acceptable latency. Consider configuring storage DRS to maintain usage of member datastores at a balanced level of 80% (default setting).

The capacity of a single VMFS datastore depends on the space required by each virtual machine accessing the datastore, multiplied by the number of virtual machines that can access the datastore within acceptable latency. Consider configuring storage DRS to maintain datastore usage at a balanced level of 80%. Reserve an additional 10% to 20% capacity for accommodating snapshots, swap files, and log files.

To facilitate stable access to iSCSI resources, static IP addresses should be configured for iSCSI initiators and targets.

For IP-based storage, a separate dedicated network or VLAN should be used to isolate storage traffic, avoiding contention for resources with other types of traffic, thereby reducing latency and improving performance.

Select a RAID level based on availability requirements; for most virtual machine workloads, if the array has sufficient battery-backed cache, the RAID level will not affect performance.

For most applications, unless there is a specific need for RDM, use VMDK disks.

Shared Storage Logical Planning

For the connection between the host and storage, multiple links should be used to improve load and redundancy.

Ensure that the concurrent I/O queue length of virtual machines within each ESXi host matches the settings of the HBA adapter card.

The underlying LUN requirements should be planned based on the actual storage IOPS demand of virtual machine applications.

Set the RAID structure of LUNs according to the application’s needs; for databases such as Oracle and SQL that involve random read and write, it is recommended to use RAID10 structure at the LUN level, while for database logs that typically involve sequential writing or continuous reading during recovery, it is suggested to use RAID5 structure at the LUN level.

For I/O-intensive applications, try to use separate VMFS storage to avoid I/O contention with other applications on the storage side.

When multiple virtual machines share a datastore or multiple hosts share a datastore, enable storage queue QoS to ensure that the latency of core applications is within controllable limits and prioritize read and write operations to the datastore.

Generally, LUN sizes of 1-2TB provide good performance and manageability.

The choice of disk array should meet the throughput requirements of the maximum IOPS of the entire virtualization environment and configure sufficient storage processors, cache, and port numbers.

Storage Space Planning

When planning LUN capacity, it is recommended to run 10 to 20 VMs per LUN (data transaction applications can be appropriately reduced), and the usage of each LUN should not exceed 80% of its capacity.

If a VM needs direct access to a storage volume, such as NTFS or EXT3, an additional LUN should be created in storage, mapped to the VM in RDM mode, allowing the VM to use it as a raw disk.

The formula for LUN capacity planning is as follows.

LUN Capacity = (Z x (X + Y) * 1.25)

Where:

Z = 10 virtual machines residing on each LUN

Y = virtual disk file capacity

X = memory size

Based on the actual production environment of XXX customer, for example:

1GB memory, single VMDK file requires 80GB, the calculation of LUN capacity is as follows:

LUN Capacity = (10 x (1 + 80) * 1.25) ≈ 1000 GB

Storage Tiering Planning

Each storage tier has different performance, capacity, and availability characteristics; as long as not every application requires expensive, high-performance, highly available storage, designing different storage tiers will be cost-effective.

When planning storage tiering, we mainly consider the following storage characteristics of applications and services.

IOPS requirements per second

MBps requirements per second

Capacity requirements

Availability requirements

Latency requirements

And based on the following information, applications and services are moved to storage tiers with matching characteristics.

Consider any existing service level agreements (SLAs)

Data may move between storage tiers during its information lifecycle

Data Store Cluster Planning

Data stores and hosts associated with data store clusters must meet specific requirements to successfully use data store cluster features.

When creating a data store cluster, the following guidelines should be followed.

Data store clusters must contain similar or interchangeable data stores.

Different sizes and I/O capabilities of data stores can be mixed within a data store cluster, and data stores from different arrays and vendors can also be mixed. However, the following types of data stores cannot coexist in the same data store cluster.

NFS and VMFS data stores cannot be combined in the same data store cluster.
In a data store cluster enabled with storage DRS, replicated data stores and non-replicated data stores cannot be combined.

All hosts connected to data stores in the data store cluster must be ESXi 5.0 or higher. If the data stores in the data store cluster are connected to ESX/ESXi 4.x or earlier hosts, storage DRS will not operate.

Data store clusters cannot contain data stores that are shared across multiple data centers.

The best practice is that data stores enabled with hardware acceleration should not be placed in the same data store cluster as those not enabled with hardware acceleration. Data stores in the data store cluster must belong to the same category to ensure that hardware acceleration support behavior is achieved.

3.3 Network Resource Planning

Correct network design has a positive impact on organizations achieving their business goals, ensuring authorized users can timely access business data while preventing unauthorized users from accessing data. Network design must be reasonably optimized to meet various needs of applications, services, storage, administrators, and users.

The goal of network resource planning is to design a virtual network infrastructure that can reduce costs, improve performance, enhance availability, provide security, and enhance functionality, allowing for smooth data transfer between applications, storage, users, and administrators.

This section provides overall planning for network resources, including virtual switches, NIC teaming, etc.

When planning network design, we mainly consider the following aspects and conduct relevant designs.

Connection requirements

Bandwidth requirements

Latency requirements

Availability requirements

Cost requirements

Guidelines and Best Practices:When planning network resources, we will follow the following guidelines and best practices.

Build a modular network solution that can continuously expand over time to meet organizational needs, enabling users to avoid replacing existing network infrastructure, thereby reducing costs.

To reduce contention and enhance security, network traffic should be logically separated according to traffic types (vSphere management network (HA heartbeat interconnect network), vMotion online migration network, network providing services for virtual machines, FT, IP storage).

VLANs can reduce the number of required network ports and cables but need support from the physical network infrastructure.

Prefer distributed switches and configure as few virtual switches as possible. For each virtual switch (vSwitch), at least two uplink physical network ports should be configured.

Network adapters can be added to or removed from standard or distributed switches without affecting virtual machines or network services running on the backend of the switch. If all running hardware is removed, virtual machines can still communicate with each other. If one network adapter remains intact, all virtual machines can still connect to the physical network.

Each physical network adapter connected to the same vSphere standard switch or distributed switch should also connect to the same physical network. Configure all VMkernel network adapters to the same MTU.

Implement redundancy for network components and paths to support availability and load distribution.

Use NIC teaming with active/standby port configuration to reduce the number of required ports while maintaining redundancy.

For multi-port redundancy configurations, physical network card ports should be configured across different PCI slots.

For the physical switching network, redundancy settings should also be made to avoid single points of failure. It is recommended to use Gigabit Ethernet switching networks to avoid network bottlenecks.

In cases with high throughput and concurrent network bandwidth requirements, consider using 10GbE, although the investment costs for ten-gigabit networks on adapters and switches will also increase accordingly. A simple method is to achieve load balancing by bundling multiple 1GbE ports on the virtual machine network vSwitch or vPortGroup.

Avoid using MSI and MSI-X modes when using passthrough devices with Linux kernel 2.6.20 or lower, as this significantly affects performance.

To protect most sensitive virtual machines, deploy firewalls within the virtual machines to route between virtual networks with uplinks (connected to the physical network) and pure virtual networks without uplinks.

Virtual Switch Planning

The selected network cards must be on the vSphere server’s network I/O device compatibility list.

For dual-port redundancy of virtual switches, if the network card supports it, NIC teaming can be implemented at the ESXi operating system level. This plan recommends configuring dual network cards’ load balancing or active/standby switching strategy at the vSwitch level; the load balancing strategy can be based on the virtual machine’s source or destination IP hash or set based on MAC address hash.

For the network of virtual machine applications, to ensure that virtual machines maintain their original VLAN state after being migrated to another physical host via vMotion, it is recommended to enable 802.1q VLAN tagging (VST) on the virtual switch port as needed. This method ensures that the migrating host retains its original network configuration, such as gateway settings, and it is recommended to enable the notify physical switch feature in the network settings, which ensures that the migrating host informs the physical switch of changes to the virtual machine port through reverse ARP, ensuring that new user sessions can be established correctly.

For virtual machine storage, use IPSAN networks to aggregate storage paths and select fault tolerance strategies through the PSA multipathing module included in the VMkernel.

NIC Teaming

Server consolidation mixes various failure impacts, increasing the need for redundancy. Using port configurations from multiple network cards and motherboard interfaces for NIC teaming can further reduce the number of single points of failure. Additionally, NIC teaming can increase the available bandwidth of network paths.

NIC teaming requires the following conditions to be met:

Assign two or more network cards to the same virtual switch
All network cards in the same port group must be in the same Layer 2 broadcast domain

3.4 Availability Planning

This planning fully considers the availability design of the virtualization environment, for example: Utilizing VMware vSphere’s built-in network redundancy and storage multipathing control at the network and storage levels to ensure high availability. On server high availability, vSphere has built-in HA, DRS, and vMotion features to address various planned and unplanned outages of virtual machine applications at local sites.

This section provides overall planning for availability, and the explanation and configuration guidelines for availability-related technologies are as follows:

Guidelines and Best Practices:

Configure all vSphere hosts in high availability clusters to achieve at least n+1 redundancy.

Enable HA policies as needed, and for systems with high business continuity requirements, set up business auto-start scripts to ensure virtual machines can start services in the shortest time after being restarted on other hosts.

Enable DRS policies as needed and set mutual exclusion rules, especially for applications using multi-machine load technology, to avoid multiple machines being on the same host.

VMware HA provides a simple, efficient, and highly available virtual machine application runtime environment. When a physical machine fails, it can be detected by other physical nodes in the cluster and automatically start the failed node’s online virtual machine on a backup physical machine or another physical machine with available resources. Additionally, if the virtual machine’s operating system fails, VMware HA can also detect it and attempt to restart the virtual machine, maximizing the availability of virtual machine applications.

Utilize VMware DRS dynamic resource allocation to collect the usage of resources (CPU, memory, etc.) from each physical host and virtual machine, providing optimal placement strategies for virtual machines, and enabling online migration of virtual machines automatically or manually to meet optimal load balancing needs. Establish resource pools with DRS to maximize the guarantee of core applications in the XXXX information center’s virtualization environment, for example, setting high priority for SQL databases for office systems to ensure optimal allocation of CPU, memory, etc. in the resource pool. At the same time, a DRS HA cluster can be built to ensure load balancing while meeting high availability requirements.

For virtual machines that need to upgrade and maintain the physical machine’s operating environment, VMware VMotion technology can be used to migrate the virtual machines running on that physical machine to other physical hosts over the network, ensuring that there is no impact on virtual machine applications during the migration. After migration, all sessions connected to clients will not be interrupted; currently, Gigabit network vSphere can concurrently migrate four virtual hosts.

3.5 Management and Monitoring Planning

The VMware vSphere virtual infrastructure must continuously operate efficiently every day. Maintaining this efficiency starts with correctly designing management and monitoring components.

This section provides overall planning for management and monitoring components, including vCenter Server and its database, alarms, and installation and configuration of ESXi hosts.

Guidelines and Best Practices:When conducting management and monitoring planning, we will follow the following guidelines and best practices.

Prefer deploying vCenter Server and database systems as virtual machines. Deploy one or more vCenter Server systems based on the size of the virtual infrastructure.

Configure static IP addresses and host names to avoid connection interruptions with vCenter Server instances.

If downtime of one or two minutes is acceptable, use vSphere HA to protect the vCenter Server system. If downtime of more than one or two minutes is unacceptable, use products like vCenter Server Heartbeat or third-party cluster solutions to protect the vCenter Server system.

If DRS is enabled for the vSphere HA cluster, disable the migration of the vCenter Server virtual machine.

For all infrastructures except for small infrastructures with a few hosts, do not place the vCenter Server database system and vCenter Server on the same system.

Use availability methods provided by the database vendor (if possible); if the database vendor does not provide specific methods, use vSphere HA to protect the database server.

If the organization is concerned about man-in-the-middle attacks when connecting to the vCenter Server system or ESXi hosts using management interfaces, use certificates. If vCenter Linked Mode is configured (for Windows-based vCenter Server systems only) or vSphere Fault Tolerance (FT), vCenter Server certificate checks must be enabled.

Use automated methods to install and configure ESXi hosts. Create a separate management cluster for infrastructure services.

Limit the number of users with access to vCenter Server. Apply the principle of least privilege for users with access to vCenter Server.

Add all servers hosting the vCenter Server system and management interfaces (VMware vSphere Client, vSphere PowerCLI, etc.) to the directory service. Then create users and groups in the directory service.

Use folders to assign roles to objects that require similar access permissions, simplifying management and enhancing security by streamlining permission allocation.

Install optional modules for vCenter Server on systems separate from the vCenter Server system. It is preferable to install optional modules on virtual machines.

If the design includes multiple vCenter Server systems, configure vCenter Linked Mode. vCenter Linked Mode requires all vCenter Server systems to be members of the same domain’s Active Directory or at least members of trusted domains.

Time synchronization must be maintained across virtual machines, ESX/ESXi hosts, and management systems.

Snapshots are not a backup solution but are useful for undo operations. For production environments, it is recommended to have one snapshot per virtual machine and a cleanup policy:

Automate performance monitoring as much as possible and create alerts for notifications when performance thresholds are exceeded. However, avoid overly strict vCenter Server alert settings.

In large infrastructures with 70-100 hosts or more, consider creating at least one management cluster enabled with vSphere HA and DRS containing three hosts.

If a management cluster already exists, prioritize using existing management tools and processes to perform operating system and application updates. Otherwise, use Update Manager.

Original title: Scenarios Not Suitable for Virtualization and Planning and Analysis of Virtualization ConstructionIf you have any questions, please click the end of the article Read the original text to comment and communicate in the community

If you find this article useful, please share, like or click “View” to let more peers see it

Recommended materials/articles:

VMware Resource Collection | 37 Practical Documents and Articles

Welcome to follow the community “Virtualization” Technical Topic, which will continuously update quality materials and articles. You can also go to raise difficult questions and communicate with peers. Address:http://www.talkwithtrend.com/Topic/23

Download the twt community client APP

Long press to recognize the QR code to download

Or search for “twt” in the app store

Long press the QR code to follow the public account

*The content published by this public account only represents the author’s views and does not represent the community’s position