A few days ago, virtualization software provider VMware announced a major update to its flagship product, vSphere 7. In addition to updates and fixes, Update 3 also brings a change that will force many administrators to take radical action.
The popular SD cards and USB flash drives that have been the standard boot media for the ESXi hypervisor for years due to price and availability, will no longer be supported with the release of Update 3.
Why such a decision?
The reason is the media stability issues that users have been reporting for a long time. Frequent writing causes USB cards and memory sticks to wear out, risking the loss of data stored on the card and the entire configuration of a particular server running ESXi. To replace the worn media, the server must be shut down, which is equivalent to downtime.
VMware has long discouraged the use of USB sticks and SD cards as bootable media. In Update 3, however, it has drastically changed the situation by severely limiting the ability of flash drives to function.
After the update, the media will still work, but only in degraded mode, which limits writing to sensitive memory (to protect the media). The server will also boot more slowly in this mode. A warning will be displayed in the user interface:
ALERT: No persistent storage available for system logs and data. ESX is operating with limited system storage space, logs and system data will be lost on reboot.
VMware’s announced patch is scheduled to take effect at the end of October 2021, leaving administrators little time to adapt.
[UPDATE 06.10.2021] VMware has accelerated the previously announced late October release of Update 3. The fixes are now available for download from the VMware website.
What can be done?
VMware’s knowledge base Vlists moving to a different storage medium, preferably M.2 flash, SAS, SATA or PCIe NVMe, as the preferred solution. This will work well for systems where it is possible to expand servers with ESXi.
Unfortunately, in systems that have been designed with minimal hardware for economic reasons, the options are more limited. Here, administrators need to seriously consider upgrading servers by adding hard drives or special PCI cards. It is also possible to move to running servers via a virtual SAN/NAS boot. As a last resort, it may be necessary to replace the servers with newer ones.
“Degraded mode” will not last forever
In degraded mode, the logs and current state of the vSphere operating system are stored in ephemeral memory on the ESXi server. Therefore, they will not be available (saved) after a reboot. However, VMware says, that the ability to store files on the RAM disk will be removed in the next major vSphere update.
The solution to the card problem is not to stop the update. In the long run, this could significantly compromise the security of our virtualisation system.
The announcement of the vSphere changes should therefore be a call to action for administrators. If you need help developing your strategy, please contact us.
MORE:
What’s new in Update 3 – a post on the official VMware blog
More background on the decision to drop USB and SD card support – vSphere blog post
