Performance Comparison of Virtualization Software

Produced by Kong Sang

First published on WeChat account Kong Sang

WeChat ID: OpenDataFoundation

After tinkering with the home network, it’s almost time to timeout. Here’s a simple article.

Test Results

CPU

The detailed entries come from the GPGPU testing section of AIDA64. Due to many peculiar issues in this section, and the big-little core scheduling being a major challenge, it needs to be discussed in detail.

  • Normal scheduling

    • Used in bare-metal order (big cores first, then little cores): Winserver2022, Workstation 17

    • Big cores first, then little cores, finally big cores with hyper-threading: PVE (the default scheduling in Linux, which I believe is the optimal solution in terms of performance)

  • Abnormal scheduling

    • Started with little cores: winserver2022 (should be equivalent to win10 21h2), very weak, and unable to bind cores. Win10 and win11 22h2 should not have this issue.

    • Random scheduling jumps: ESXi, unfortunately, version 8.0u1 has not resolved this issue. It seems VMware will not care about consumer-grade anymore.

It is clear that PVE’s compatibility and performance loss are exceptional. Moreover, it is worth noting that the performance of soft routing, which is strongly related to AES encryption and decryption speed, still incurs losses in virtual machines.

  • Due to the 6.2GHz clearly overheating, I later adjusted the voltage settings, and the scores for ESXi and PVE seem to be slightly lower.

  • Hyper-V has a problem where it prioritizes using little cores (reportedly, this issue does not exist in win11).

  • ESXi cannot recognize big core hyper-threading, and threads may actually be randomly scheduled to big and little cores. This time, CPUZ only took the big core scores, but R23 single-core scores are averaged across all cores since each core basically ran once.

  • When using big and little cores, PVE is incompatible with AIDA64. It should not be counted in the scores.

Test Item windows Hyper-V Workstation ESXi PVE
cpuz-Single 1007.5 944.9 938.1 914.3 988.9
cpuz-Multi 17665.1 17177.3 17460.8 15486.1 16767.5
R23-Single 2442 2073 2320 1468 2347
R23-Multi 42262 40802 40624 35300 38718
Single Float 2552 2530 2527 2573 134.1
Double Float 1276 1264 1254 1282 92.64
24bit Int 937.8 939.9 933.8 917.5 69.23
32bit Int 936.3 938.7 940.9 917.3 69.83
64bit Int 311.6 293.3 290.1 295.0 21.48
AES-256 308957 60233 170786 142080 7875
SHA-1 79797 54958 63531 56031 3923
Julia 951.8 937.9 878.4 856.2 65.7
Mandel 485.4 0 480.4 465.9 36.04

Here’s a supplement from tests I did with a 5900x in a similar environment.

Test Item windows ESXi PVE
cpuz-Single 631.5 637.4 635.1
cpuz-Multi 8995.9 9210.3 9203.7
MEM-Read 46346 43237 45939
MEM-Write 46351 40004 45275
MEM-Copy 45266 42283 44383
MEM-Latency 79.1 80.7 60.9
Single Float 1315 1606 1631
Double Float 825.6 798 810.9
24bit Int 1315 1286 1303
32bit Int 1307 1280 1298
64bit Int 105.7 104.8 106
AES-256 201179 136510 92500
SHA-1 47263 42123 36948
Julia 707.1 533.3 543.5
Mandel 386.9 289.5 299.3

MEM

The scores come from AIDA64, and I set the timings loosely. The unit for the first item is GiB/S, and the second item is MiB/S, which are actually quite similar.

  • When using big and little cores, PVE is incompatible with AIDA64. It should not be counted in the scores.

Test Item windows Hyper-V Workstation ESXi PVE
MEM-Read 105.40 102.91 103.44 100.47 8124
MEM-Write 90843 91450 91111 89628 6798
MEM-Copy 93650 89807 91215 96925 7062
MEM-Latency 70.3 86.9 75.1 76.9 781.5

DISK

Testing was conducted using Crystal Disk Mark 8.0.4, with default test items. The parameters were set to 64GiB, 5.

Only VMware Workstation, which is a fully virtualized VM, cannot fully utilize performance. Note: Do not use old and inferior SATA and SCSI controllers, as their performance is poor.

  • The incredible 4k read of VMware Workstation should be due to memory acceleration being enabled.

Test Item windows Hyper-V Workstation ESXi PVE
SEQ1M-Q8T1-Read 6309.68 6316.14 4681.29 6800.49 6641.09
SEQ1M-Q8T1-Write 4057.99 4060.33 2557.69 4147.90 4141.40
SEQ1M-Q1T1-Read 3354.80 3113.56 2751.92 3244.73 3245.54
SEQ1M-Q1T1-Write 3663.85 3380.36 2478.29 3351.48 3393.28
RND4k-Q32T1-Read 1521.80 725.47 422.60 501.97 819.89
RND4k-Q32T1-Write 1292.43 741.58 291.87 902.25 605.58
RND4k-Q1T1-Read 57.29 43.18 74.42 39.69 45.17
RND4k-Q1T1-Write 312.17 99.16 66.70 85.72 116.78

NET

The test items involve reading between virtual machines through a memory bridge, and the speed of local loopback tested directly on bare metal.

This test seems to have significant fluctuations in results, so it is for reference only. Anyway, Workstation is just getting beaten.

Test Item windows Hyper-V Workstation ESXi PVE
Iperf3-T1 w2w 68.8 22.0 6.53 13.1 23.8
Iperf3-T32 w2w 125 35.9 6.58 50.6 42.9
Iperf3-T32 l2w 31.3 6.27 50.9 43.7

Performance Comparison of Virtualization Software

Performance Comparison of Virtualization Software

Leave a Comment