Five Key Aspects of Virtualization Planning

This article introduces how to plan for computing resource, storage resource, network resource, availability, and management and monitoring in VMware-based virtualization construction. Shared by skey_deng.

1. Advantages of Virtualization

Virtualization technology can significantly reduce the number of devices that need to be maintained and managed. Virtualization can improve resource utilization, lower hardware procurement costs, while being more energy-efficient and space-saving.

After server virtualization, we can manage them uniformly through a virtualization cluster. This makes full use of existing devices and allows for both vertical and horizontal scaling of new devices.

The in-depth application of virtualization will dynamically increase the scale of the virtualization cluster, forming a healthier IT architecture.

On the client side, the original access mode will not change, and there will be no adverse impact on business systems, allowing for a smooth migration of existing systems.

The advantages of virtualization compared to traditional architectures are summarized as follows:

Planning work is easier to control
Lower hardware investment
High resource utilization, easy integration
Low server power consumption, minimal supporting infrastructure consumption
Fast application deployment, short time to market for new businesses
Excellent business continuity and strong disaster recovery capabilities

2. Applicability of Virtualization

The applicability of virtualization platforms is quite strong; except for a few types of applications that are not suitable for virtualization platforms, the rest can be applied.

Applications Not Suitable for Virtualization

Applications with special hardware access requirements

Requires high-performance graphics card

Not suitable for virtualization, such as OCR text recognition systems
Special serial/parallel encryption devices

Not suitable for virtualization, such as unified encryption platforms
USB device connection requirements

May not be suitable, external USB devices can be used instead, but testing is required, such as some monitoring that uses USB keys for authorization

Applications that still have high loads even on high-end servers

May not be suitable, current server configuration needs to be analyzed

3. Planning and Analysis of Virtualization Construction

Virtual machines run applications and services that support various users and entire business lines, many of which are critical business applications, so it is essential to design, allocate, and manage virtual machines correctly to ensure these applications and services can operate efficiently.

3.1 Computing Resource Planning

VMware ESXi hosts are the basic computing building blocks of the data center, and these host resources can be aggregated to build a highly available dynamic resource pool environment, serving as the overall computing resources for various applications in the data center.

This section mainly introduces the requirements for overall planning of computing resources, including physical servers, virtual machines, etc.

Guidelines and Best Practices:

Unless multiple virtual CPUs (vCPUs) are truly needed, default to configuring one and use as few vCPUs as possible. The operating system must support symmetric multiprocessing (SMP) features. Applications must be multithreaded to benefit from multiple vCPUs. The number of vCPUs should not exceed the number of physical CPU cores on the host.

Do not plan to use all CPU or memory resources of the host, leaving some available resources in the design. To optimize virtual machine memory performance, it is critical to reserve active memory for the virtual machine in physical RAM and avoid overcommitting active memory.

Always keep transparent page sharing enabled, always load VMware Tools, and enable memory reclamation.

Resource pool CPU and memory shares settings cannot be used to configure virtual machine priority. Resource pools can be used to allocate dedicated CPU and memory resources to virtual machines.

In environments where workloads are highly variable, configure vSphere DPM to reduce energy consumption and heat costs.

Deploy one system disk and one separate application data disk. If the system disk and data disk require the same I/O characteristics (RAID level, storage bandwidth, and latency), they should be placed together in one datastore.

Application requirements should be the primary metric for allocating resources to virtual machines.

Protect virtual machines’ security as you would physical machines. Ensure that antivirus, anti-spyware, intrusion detection, and firewalls are enabled for each virtual machine in the virtual infrastructure. Ensure that all security measures are updated regularly. Apply appropriate latest patches, keep virtual machine software and applications up to date, and use patch management tools or install and configure Update Manager. Intrusion detection and firewalls can be configured on the network.

To avoid management connection issues, assign static IP addresses and hostnames to each ESXi host. For ease of management, configure DNS with each ESXi host’s hostname and IP address; if cost is a concern, a simple DNS service can be set up in a VM.

Ensure that the data center has sufficient power and cooling capacity to avoid service interruptions.

Regardless of the chosen hardware platform, a consistent platform configuration should be designed, especially in VMware clusters. Consistency includes CPU type, memory capacity and slot allocation, network card and host bus adapter types, and PCI slot allocation, refer to VMware’s compatibility list.

Use one or more clusters enabled with vSphere HA and DRS to increase availability and scalability.

Whether to use horizontal or vertical scaling of clusters is determined by the cluster’s purpose, infrastructure scale, vSphere limitations, and financial and operational costs.

Estimate each server’s computing power based on peak rather than average values to ensure it can support the maximum number of virtual machine applications running simultaneously.

CPU and memory estimates need to reserve 20% of space for burst computing capacity.

Memory sharing is estimated at 50%.

Use the following formula to estimate the number of ESXi hosts needed to meet the normal operation of virtual machines at CPU peak demand in the data center.Sum of all virtual machines’ CPU peak frequency requirements / Available CPU per ESXi host = Number of ESXi hosts needed

Use the following formula to estimate the number of ESXi hosts needed to meet the normal operation of these virtual machines at memory peak demand in the data center.Sum of all virtual machines’ memory peak total demand / Available memory per ESXi host = Number of ESXi hosts needed

To utilize the high availability features of vSphere, we need to consider increasing the number of ESXi hosts.

3.2 Storage Resource Planning

Correct storage design positively impacts an organization’s ability to achieve business goals, laying the foundation for a well-performing virtual data center. It can protect data from malicious or accidental destruction while preventing unauthorized users from accessing data. Storage design must be reasonably optimized to meet the diverse needs of applications, services, administrators, and users.

The goal of storage resource planning is to strategically align business applications with storage infrastructure to reduce costs, improve performance, increase availability, provide security, and enhance functionality.

This section covers the overall planning of storage resources, including shared storage logical planning, storage space planning, storage I/O control planning, and storage tiering planning.

Guidelines and Best Practices:When planning storage resources, we will follow the following guidelines and best practices.

Build a modular storage solution that can continuously expand over time to meet organizational needs, allowing users to avoid replacing existing storage infrastructure. Both capacity and performance should be considered in modular storage solutions.

Each storage tier has different performance, capacity, and availability characteristics; as not every application requires expensive, high-performance, highly available storage, designing different storage tiers will be economically efficient.

Configure storage multipathing features, and establish redundancy at the host, switch, and storage array levels to improve availability, scalability, and performance.

Allow all hosts in the cluster to access the same data store.

Configure storage DRS to balance based on usage and latency.

For Fibre Channel, NFS, and iSCSI storage, appropriate designs can reduce latency and improve availability. For workloads that require processing a large number of transactions per second (e.g., data collection or transaction logging systems), distributing workloads to different locations is particularly important. Reduce latency by minimizing the number of hops in the storage path.

The maximum capacity of NFS storage depends on the array vendor. The capacity of a single NFS datastore depends on the space required by each virtual machine accessing the datastore, multiplied by the number of virtual machines that can access the datastore under acceptable latency. Consider configuring storage DRS to keep the usage of member datastores at a balanced level of 80% (default setting).

The capacity of a single VMFS datastore depends on the space required by each virtual machine accessing the datastore, multiplied by the number of virtual machines that can access the datastore under acceptable latency. Consider configuring storage DRS to keep datastore usage at a balanced level of 80%. Reserve 10% to 20% additional capacity for snapshots, swap files, and log files.

To facilitate stable access to iSCSI resources, static IP addresses should be configured for iSCSI initiators and targets.

For IP-based storage, use a separate dedicated network or VLAN to isolate storage traffic, avoiding contention for resources with other types of traffic, thereby reducing latency and improving performance.

Select a RAID level based on availability requirements. For most virtual machine workloads, if the array has sufficient battery-backed cache, the RAID level does not affect performance.

For most applications, unless there is a specific requirement for RDM, use VMDK disks.

Shared Storage Logical Planning

For host-to-storage connections, multiple links should be used to improve load balancing and redundancy.

Ensure that the concurrent I/O queue length of virtual machines within each ESXi host is consistent with the settings of the HBA adapter cards.

The requirements for underlying LUNs should be planned based on the actual storage IOPS needs of virtual machine applications.

Set the RAID structure of LUNs according to the needs of applications, e.g., for databases like Oracle and SQL that require random read/write, it is recommended to use RAID10 structure at the LUN level; for database logs that typically involve sequential writes or continuous reads during recovery, it is recommended to use RAID5 structure at the LUN level.

For I/O intensive applications, it is advisable to use separate VMFS storage to avoid I/O contention with other applications at the storage end.

When multiple virtual machines share a datastore or multiple hosts share a datastore, enable storage queue QoS to ensure that the latency of core applications is within control and prioritize read/write access to the datastore.

Typically, a LUN size of 1-2TB has good performance and manageability.

The selection of disk arrays should meet the throughput requirements for maximum IOPS in the entire virtualization environment and configure enough storage processors, cache, and ports.

Storage Space Planning

When planning LUN capacity, it is recommended to run 10 to 20 VMs per LUN (data transaction applications can be reduced accordingly), and the usage of each LUN should not exceed 80% of its capacity.

If VMs need direct access to storage volumes, such as NTFS or EXT3, a separate LUN should be created in storage to be mapped to the VM in RDM mode, allowing the VM to use it as a raw disk.

The formula for LUN capacity planning is as follows:

LUN Capacity = (Z x (X + Y) * 1.25)

Where:

Z = 10 virtual machines residing on each LUN

Y = Virtual disk file capacity

X = Memory size

Based on the actual production environment of XXX customer, for example:

1GB memory, a single VMDK file needs 80GB, the calculation for LUN capacity is as follows:

LUN Capacity = (10 x (1 + 80) * 1.25) ≈ 1000 GB

Storage Tiering Planning

Each storage tier has different performance, capacity, and availability characteristics; as not every application requires expensive, high-performance, highly available storage, designing different storage tiers will be economically efficient.

When planning storage tiering, we mainly consider the following storage characteristics of applications and services.

IOPS requirements per second

MBps requirements per second

Capacity requirements

Availability requirements

Latency requirements

And based on the following information, move applications and services to storage tiers designed with matching characteristics.

Consider any existing service level agreements (SLAs)

Data may move between storage tiers during its information lifecycle

Data Store Cluster Planning

Data stores and hosts associated with data store clusters must meet specific requirements to successfully use the data store cluster feature.

When creating a data store cluster, the following guidelines should be followed.

Data store clusters must contain similar or interchangeable data stores.

A data store cluster can mix data stores of different sizes and I/O capabilities, as well as data stores from different arrays and vendors. However, the following types of data stores cannot coexist in the same data store cluster.

In the same data store cluster, NFS and VMFS data stores cannot be combined.
In a data store cluster enabled with storage DRS, replicated data stores and non-replicated data stores cannot be combined.

All hosts connected to data stores in the data store cluster must be ESXi 5.0 or higher. If data stores in the data store cluster are connected to ESX/ESXi 4.x or earlier hosts, storage DRS will not function.

Data store clusters cannot contain data stores shared across multiple data centers.

Best practice is that data stores enabled with hardware acceleration should not be placed in the same data store cluster as those not enabled with hardware acceleration. Data stores in a data store cluster must belong to the same class to ensure hardware acceleration support behavior.

3.3 Network Resource Planning

Correct network design positively impacts an organization’s ability to achieve its business goals, ensuring authorized users can access business data in a timely manner while preventing unauthorized users from accessing data. Network design must be reasonably optimized to meet the various needs of applications, services, storage, administrators, and users.

The goal of network resource planning is to design a virtual network infrastructure that can reduce costs, improve performance, increase availability, provide security, and enhance functionality, allowing for smoother data flow between applications, storage, users, and administrators.

This section covers the overall planning of network resources, including virtual switches, NIC teaming, etc.

When planning network design, we mainly consider the following aspects and conduct related designs.

Connection requirements

Bandwidth requirements

Latency requirements

Availability requirements

Cost requirements

Guidelines and Best Practices:When planning network resources, we will follow the following guidelines and best practices.

Build modular network solutions that can continuously expand over time to meet organizational needs, allowing users to avoid replacing existing network infrastructure, thereby reducing costs.

To reduce contention and enhance security, network traffic should be logically separated according to traffic types (vSphere management network (HA heartbeat interconnect), vMotion online migration network, networks serving virtual machines, FT, IP storage).

VLANs can reduce the number of required network ports and cables, but they require support from the physical network infrastructure.

Prefer distributed switches and configure as few virtual switches as possible. For each virtual switch (vSwitch), at least two uplink physical network ports should be configured.

Network adapters can be added or removed from standard or distributed switches without affecting virtual machines or network services running behind the switches. If all running hardware is removed, virtual machines can still communicate with each other. If one network adapter remains intact, all virtual machines can still connect to the physical network.

Each physical network adapter connected to the same vSphere standard switch or distributed switch should also be connected to the same physical network. Configure all VMkernel network adapters to the same MTU.

Implement redundancy for network components and paths to support availability and load balancing.

Use NIC teaming with active/standby port configuration to reduce the number of required ports while maintaining redundancy.

For multi-port redundancy configurations, physical network adapter ports should be configured across different PCI slots.

Redundancy settings should also be made for the physical switch network to avoid single points of failure. It is recommended to use Gigabit Ethernet switching networks to avoid network bottlenecks.

In cases with high throughput and concurrent network bandwidth requirements, consider using 10GbE, but the investment costs for 10GbE networks on adapters and switches will also increase accordingly. A simple method is to achieve load balancing by bundling multiple 1GbE ports on the virtual machine network vSwitch or vPortGroup.

When using passthrough devices with Linux kernels 2.6.20 or lower, avoid using MSI and MSI-X modes as they can significantly impact performance.

To protect most sensitive virtual machines, deploy firewalls in the virtual machines to route between virtual networks with uplinks (connected to physical networks) and pure virtual networks without uplinks.

Virtual Switch Planning

The selected network cards must be on the vSphere server’s network I/O device compatibility list.

For dual-port redundancy of virtual switches, if the network card supports NIC teaming at the ESXi operating system level, this scheme recommends configuring load balancing or active/standby switching strategies at the vSwitch level for dual network cards. Load balancing strategies can be based on the source or destination IP hash of the virtual machine, or set based on the MAC address hash.

For network applications of virtual machines, to ensure that virtual machines maintain their original VLAN status when performing vMotion migration to another physical host, it is recommended to enable 802.1q VLAN tagging (VST) on the virtual switch ports as needed. This method ensures that the migration host can retain the original network configuration such as gateways, and it is also recommended to enable the notification to the physical switch feature in the network settings, which can ensure that the migration host notifies the physical switch of changes to the virtual machine port through reverse ARP, ensuring that new user sessions can be established correctly.

For virtual machine storage, use IPSAN networks, utilizing the PSA multipathing module included in the VMkernel for storage path aggregation and fault policy selection.

NIC Teaming

Server consolidation mixes various fault impacts together, increasing the need for redundancy. Configuring NIC teaming using ports from multiple network cards and motherboard interfaces can further reduce the number of single points of failure. Additionally, NIC teaming can increase the available bandwidth of network paths.

NIC teaming must meet the following conditions:

Assign two or more network cards to the same virtual switch
All network cards in the same port group must be in the same second-layer broadcast domain

3.4 Availability Planning

This planning fully considers the availability design of the virtualization environment, for example: utilizing VMware vSphere’s built-in network redundancy and storage multipathing control at the network and storage levels to ensure high availability. For server high availability, vSphere includes features like HA, DRS, and vMotion to address various planned and unplanned downtime issues for virtual machine applications at the local site.

This section covers overall planning for availability, with explanations and configuration guidelines for availability-related technologies as follows:

Guidelines and Best Practices:

Configure all vSphere hosts in high availability clusters, achieving at least n+1 redundancy.

Enable HA policies as needed, and for systems with high business continuity requirements, set business auto-start scripts to ensure virtual machines can start services in the shortest time on other hosts after a restart.

Enable DRS policies as needed and set mutual exclusion rules, especially for applications using multi-machine load technology, to avoid multiple machines being on the same host.

VMware HA provides an easy-to-use, efficient, high-availability virtual machine application runtime environment. In the event of a physical machine failure, it can be detected by other physical nodes in the cluster and automatically start the failed node’s online virtual machine on a backup physical machine or another physical machine with available resources. Additionally, if the virtual machine operating system fails, VMware HA can also detect it and attempt to restart the virtual machine, maximizing the availability of virtual machine applications.

Utilizing VMware DRS for dynamic resource allocation can collect the resource usage (CPU, memory, etc.) of physical hosts and virtual machines and provide optimal placement strategies for virtual machines, allowing for online migration functionality to meet optimal load balancing needs. Using DRS to establish resource pools can maximize the guarantee of core applications in the XXXX information center’s virtualization environment, for example, setting high priority for SQL databases for office systems, ensuring optimal allocation of CPU and memory resources in the resource pool. At the same time, a DRS HA cluster can be built to ensure load balancing while meeting high availability requirements.

For virtual machines that need to upgrade and maintain the physical machine’s operating environment, VMware VMotion technology can be used to migrate the virtual machines running on that physical machine to other physical hosts over the network online, ensuring that there is no impact on the virtual machine applications during the migration. After migration, all sessions connected to clients will not be interrupted; currently, Gigabit network vSphere can concurrently migrate four virtual hosts.

3.5 Management and Monitoring Planning

VMware vSphere virtual infrastructure must operate continuously and efficiently every day. Maintaining this efficiency starts with correctly designing management and monitoring components.

This section covers overall planning for management and monitoring components, including vCenter Server and its database, alerts, and ESXi host installation and configuration, etc.

Guidelines and Best Practices:When planning management and monitoring, we will follow the following guidelines and best practices.

Prefer deploying vCenter Server and database systems as virtual machines. Deploy one or more vCenter Server systems based on the size of the virtual infrastructure.

Configure static IP addresses and hostnames to avoid connection interruptions with vCenter Server instances.

If downtime of one or two minutes is permissible, use vSphere HA to protect the vCenter Server system. If downtime of more than one or two minutes is intolerable, use products like vCenter Server Heartbeat or third-party clustering solutions to protect the vCenter Server system.

If DRS is enabled for vSphere HA clusters, disable migration of the vCenter Server virtual machine.

For all infrastructures except small ones with a limited number of hosts, do not place the vCenter Server database system and vCenter Server on the same system.

Use the availability methods provided by the database vendor (if possible); if the database vendor does not provide a specific method, use vSphere HA to protect the database server.

If the organization is concerned about man-in-the-middle attacks when connecting to the vCenter Server system or ESXi hosts using management interfaces, use certificates. If vCenter linked mode (for Windows-based vCenter Server systems only) or vSphere Fault Tolerance (FT) is configured, vCenter Server certificate checking must be enabled.

Use automated methods to install and configure ESXi hosts. Create a separate management cluster for infrastructure services.

Limit the number of users with access to vCenter Server. Apply the principle of least privilege to users with access to vCenter Server.

Add all servers hosting the vCenter Server system and management interfaces (VMware vSphere Client, vSphere PowerCLI, etc.) to directory services. Then create users and groups in the directory service.

Use folders to assign roles to objects that require similar access permissions, simplifying management and enhancing security by streamlining permission assignments.

Install optional modules for vCenter Server on systems independent of the vCenter Server system. It is best to install optional modules on virtual machines.

If the design includes multiple vCenter Server systems, configure vCenter linked mode. vCenter linked mode requires all vCenter Server systems to be members of the same Active Directory domain or at least members of trusted domains.

Time synchronization must be maintained across virtual machines, ESX/ESXi hosts, and management systems.

Snapshots are not a backup solution but are a useful rollback solution. For production environments, it is recommended that each virtual machine corresponds to a snapshot and cleanup policy:

Automate performance monitoring as much as possible and create alerts for notifications when performance thresholds are exceeded. However, avoid overly strict vCenter Server alert settings.

In large infrastructures containing 70-100 hosts or more, consider creating at least one management cluster enabled with vSphere HA and DRS containing three hosts.

If a management cluster already exists, prioritize using existing management tools and processes to perform updates to the operating system and applications. Otherwise, use Update Manager.

If you have any questions, please click at the end of the articleRead the original text, and comment in the community original text for discussion

If you find this article useful, pleaseshare or click “Looking” so more peers can see it

Recommended materials/articles:

34 practical articles and documents on virtualization
Which of these three implementations for mirroring network traffic in a virtualization environment is the best?

http://www.talkwithtrend.com/Article/246279

Welcome to follow the community “Virtualization Technology Topics, which will continuously update high-quality materials and articles. You can also go to raise difficult questions and discuss with peers. Address:

http://www.talkwithtrend.com/Topic/23

Download the twt community client APP

Join more peers

Experts will answer your difficult questions anytime

Easily subscribe to technical topics in various fields

Browse and download the latest articles and materials

Long press to identify the QR code to download

Or search for “twt” in the app store

Long press the QR code to follow the public account

* The content published by this public account only represents the author’s views and does not represent the community’s position