TL;DR: You need to boot the vm then run sudo nvram boot-args=”-nehalem_error_disable” to disable the ECC memory check, then you can edit the .vmx file and add the hw.model, board-id and serialNumber to it.
The Apple Xserve is already end of life, but as anyone that still has one will tell you it is still a solid and reliable machine. However, it is getting harder and harder to run recent versions of macOS. To add insult to injury macOS server really isn’t much of a server anymore.
This coupled with outgrowing my bare metal os setup is what pushed me to migrate my Xserve pair I didn’t want to retire or replace just yet to ESXi. These machines are running vSphere 6.5 with dual W5590’s and 192GB of Ram (12x 16GB sticks) so they have more than enough power for what I need. I was also able to get Monterey running on ESXi in a VM so I am able to run the latest version of macOS on a VM without even breaking the Apple EULA!
As part of this migration to ESXi I converted the bare metal OS on my xserve to a VM within it. During the process I realized that the vm didn’t know it used to be a Xserve bare metal install and was just a “VMware” model identifier so I decided to fix it and give it some of it’s past back. This does a couple things for me. For shares and other things it still shows as an Xserve and it makes sure anything that was on there that was just for an Xserve still works ( even though there isn’t I know of left at this point), but really I just wanted the Xserve icons to show up whenever users connected to it. 😉
The Process
Unlike other Macs of that era the Xserve and Mac Pro use ECC memory. When OS X or the OS X installer boots it checks to see if the machine is an Xserve or Mac Pro. If it is one of these machines then it load that AppleTyMCEDriver in a way that it expects ECC memory. If that kext doesn’t find ECC memory to talk to it causes a kernel panic on the boot logo screen. The good news is there is a boot flag you can set that will prevent the system from doing this check, but as of right now I have not found a way to set the boot flag in the VM config directly, however you can set it in the both the NVRAM and the OS boot settings. Although you just need 1 of these I highly recommend doing both so that you can always boot in case one of the 2 gets reset somehow.
Setting the boot flag in the NVRAM
In order to set the flag in the NVRAM you just need to boot into OS X or MacOS and run the following from the terminal sudo nvram boot-args="-nehalem_error_disable" if you are in recovery mode or using the installer the terminal is sudo by default so you will get an error that says it doesn’t recognize the sudo command. All you need to do is just drop the sudo part and run this instead. nvram boot-args="-nehalem_error_disable" Once you do this you should confirm it took the boot arg flag by running nvram boot-args="-nehalem_error_disable" When you run that you should get the following output.
boot-args -nehalem_error_disableIf you don’t get this output then it didn’t take the flag. With this done you can now boot that machine as an Xserve or MacPro without the kernel panic from the ECC, however we still need to set the model type and board ID.
Adding the flag to emulate the machine
We all be editing the VM’s files in this step so make a backup before you start this process. The machines I use run ESXi 6.5 so if you are using Fusion or a different setup the exact steps here may be a bit different for you, however the process should be about the same. In my case I navigated to the datastore that the Virtual Machine was on then went into the folder for it and downloaded the <MACHINE NAME>.vmx file. It should have the same name as your vm. IMPORTANT NOTE: There is a .vxmf file you DO NOT want that one you want the .vmx. Once you have downloaded the file you have two options for adding the flags. If you are on the type of machine you want to emulate for example an Xserve 3,1 and you just want to emulate the parent you just need to add a new line with the following. smbios.reflectHost = "TRUE" This will pass through the model, board-id and serial number of the parent for you. If however, you are not emulating the parent or just want to be able to take the VM to other hosts you can set each manually as follows:
board-id.reflectHost = "FALSE"
board-id = "MAC-F223BEC8"
hw.model.reflectHost = "FALSE"
hw.model = "Xserve3,1"
serialNumber.reflectHost = "FALSE"
serialNumber = "<PUT_A_REAL_SERIAL_NUMBER_HERE">
smbios.reflectHost = "FALSE"
As far as I can tell it doesn’t matter where you put these, but my suggestion is to just add the lines to the bottom of the file. If you downloaded the file like I did and didn’t edit it directly at this point you will also need to delete the old .vmx you downloaded and upload the new one.
Adding the boot arguments to the plist
Once you have installed the operating system you will be using or if you have already completed the other 2 steps you will want to add the boot flag to the com.apple.Boot.plist file so that the OS will set the -nehalem_error_disable boot flag even if it isn’t in the NVRAM. This makes sure you can still boot if your NVRAM somehow gets cleared. This plist is a SIP protected file so first you will need to boot into the recovery partition or the OS X installer. Go to the terminal and run csrutil disable this will disable SIP. Once you run this command you can reboot back to the regular OS you are using and edit the plist with sudo vi /Library/Preferences/SystemConfiguration/com.apple.Boot.plist or if your like me and prefer nano. You can replace vi in that command with nano, but you will need to make sure it is installed on your machine. In the file you will want to replace <string></string> with <string>-nehalem_error_disable</string> then save the file. Since we still need to re-enable SIP make sure to double check that your changes were saved and that you got it all correct. Once you do that, then boot back into the recovery partition or the OS installer and run csrutil enable to turn SIP back on. This has the added benefit of locking in the boot plist so you don’t accidentally delete or reset it. Then you can boot back into the OS and you should be good to go.
Notes and Caveats
Although you will see the machine name and identifier in System Profiler and other places Time Machine will not recognize this as a true Xserve3,1 so if you try to restore a backup that was made on a bare metal Xserve with Time Machine on this VM it will tell you it isn’t the same machine and won’t allow you to do it. However, you can still import the data from the backup just like you would any other Mac.
It should also be noted that wasn’t able to get iCloud working even with a valid serial and Apple Mac address. The error I got was that the machine could not be validated. However, I really didn’t try too hard to trouble shoot it since I didn’t need it.
This was done on an Xserve3,1 so if you are looking to emulate an older Xserve or Mac Pro there may be extra steps or this may or may not apply. I don’t need to emulate a Mac Pro so I don’t know what models this apples to, however my guess is that it would be the Mac Pro 4,1 and 5,1 since they can use the Intel Nehalem chips. If someone does testing or figures this out please leave a comment and I will update the article.