ESXi 7 and 8 Installation fails with "Fatal CPU mismatch on feature"
- by Florian Grehl
- November 27, 2022
- 27 Comments
When you try to install ESXi 7.0 or 8.0 on a system with a 12th Gen Intel CPU, the installation fails with a purple diagnostics screen:
HW feature incompatibility detected; cannot start
Fatal CPU mismatch on feature "Hyperthreads per core"
Fatal CPU mismatch on feature "Cores per package"
Fatal CPU mismatch on feature "Cores per die"
When you try to power on Virtual Machines on 13th Gen Intel CPUs, ESXi crashes with the following PSOD:
GP Exception 13 in world
This problem is caused by the new architecture of Intel CPUs which are equipped with different types of cores - Performance-cores and Efficient-cores. With vSphere 7.0 Update 2, the kernel parameter cpuUniformityHardCheckPanic has been implemented to address the issue. This article explains the manual and automated methods to properly implement the kernel option.
This article is valid for:
- ESXi 8.0 and later
- ESXi 7.0 Update 2 and later
Manual Method
The parameter needs to be set prior to installation and the first boot of ESXi.
- When ESXi installation starts, press SHIFT+O to edit boot options.
- Append cpuUniformityHardCheckPanic=FALSE
- Press ENTER
- Install ESXi
- When the installation is finished, reboot the system and press SHIFT+O to edit the boot options again.
- Append cpuUniformityHardCheckPanic=FALSE and press ENTER
- To make the kernel option permanent, run the following command on your ESXi host:
# esxcli system settings kernel set -s cpuUniformityHardCheckPanic -v FALSE
- For 13th Gen Intel CPUs (Raptor Lake), you also have to enable kernel option ignoreMsrFaults to prevent PSOD during VM startups. (Credit to William Lam for providing a solution for the PSOD Issue)
# esxcli system settings kernel set -s ignoreMsrFaults -v TRUE
This setting allows ESXi to work with different P-Cores and E-Cores, present in 12th Gen Intel CPUs.
Automated Kickstart Method
If you prefer an automated approach, you can also use a scripted kickstart installation to fix the issue for you. In this example, I'm installing ESXi 8.0 using a USB flash drive on an Intel 12th Gen Pro NUC.
- Create a bootable USB flash drive using the ESXi ISO (See this article for instructions).
- On the flash drive, open the /efi/boot/boot.cfg file with an editor and add the following options to the kernelopt= line: ks=usb:/KS.CFG cpuUniformityHardCheckPanic=FALSE
This will instruct the ESXi installer to perform a scripted installation from the /KS.CFG file. - As ESXi will not be able to boot without the kernel option and thus does not reach the %firstboot section, you have to manipulate the installed ESXi on the USB flash drive post-installation but before the first reboot. This happens in the %post section. Create the /KS.CFG file with the following content (fix parameters to match your environment):
vmaccepteula install --firstdisk=usb --overwritevmfs network --bootproto=static --ip=192.168.0.26 --netmask=255.255.255.0 --gateway=192.168.0.1 --hostname=esx12.virten.lab --nameserver=192.168.0.1 rootpw VMware1! %post --interpreter=busybox /bin/mcopy -o -i /dev/disks/mpx.vmhba32\:C0\:T0\:L0\:5 ::BOOT.CFG /tmp/BOOT.cfg /bin/sed -i '/kernelopt/s/$/ cpuUniformityHardCheckPanic=FALSE/' /tmp/BOOT.cfg /bin/mcopy -o -i /dev/disks/mpx.vmhba32\:C0\:T0\:L0\:5 /tmp/BOOT.cfg ::BOOT.CFG /bin/reboot %firstboot --interpreter=busybox # Enable SSH vim-cmd hostsvc/enable_ssh vim-cmd hostsvc/start_ssh # Enable ESXi Shell vim-cmd hostsvc/enable_esx_shell vim-cmd hostsvc/start_esx_shell # Suppress Shell warning esxcli system settings advanced set -o /UserVars/SuppressShellWarning -i 1 # Disable CPU Uniformity Check localcli system settings kernel set -s cpuUniformityHardCheckPanic -v FALSE # Ignore general protection faults as a result of rdmsr and wrmsr instructions localcli system settings kernel set -s ignoreMsrFaults -v TRUE # SSH Key cat > /etc/ssh/keys-root/authorized_keys <<EOF ssh-rsa AAAAB3N[...] yourname EOF # NTP esxcli system ntp set -s pool.ntp.org esxcli system ntp set -e 1
- Insert the flash drive into your System and power it on. The installation should start automatically and after a couple of minutes, you should be able to access the system with SSH or UI.
- Please note that the %firstboot section is not applied when Secure Boot is enabled, which is the default for most systems. Consider disabling Secure Boot for the full kickstart pleasure, or run these commands manually. Secure Boot does not affect the %post section, so the mcopy and sed magic will work.
Alternative Option - Disable Performance or Efficient Cores
If you do not want to change kernel options, you can also disable P or E cores to completely get rid of the non-uniform core configuration. For performance, I recommend disabling Efficient Cores. Make also sure to enable Hyperthreading.
- Make sure to get the latest BIOS version for your NUC
- Power On the NUC
- Immediately press F2 to enter BIOS
- Navigate to Power, Performance and Cooling and set External Ambient Temperature Tolerance to User Defined
- Navigate to Power, Performance and Cooling > Performance > Processor
- Set Active Efficient Cores to 0
- Press F10 to Save and Exit