On July 24th 2023, AMD has issued a security bulletin disclosing a vulnerability in its Zen2 computer processor microarchitecture. Named “Cross-Process Information Leak” by AMD, the vulnerability is also known as “Zenbleed”. Labelled CVE-2023-20593 and rated by AMD as Medium, the issue allows an attacker to potentially access sensitive information processed by the CPU in specific circumstances. The issue affects all software running on the AMD Zen2 based processors, including virtual machines, sandboxes, containers, and processes. Exploitation software is likely available, and we expect that attacks based on this vulnerability will occur soon.
All AMD Zen2 CPUs, including EPYC Rome processors, are vulnerable:
- AMD Ryzen 3000 Series Processors
- AMD Ryzen PRO 3000 Series Processors
- AMD Ryzen Threadripper 3000 Series Processors
- AMD Ryzen 4000 Series Processors with Radeon Graphics
- AMD Ryzen PRO 4000 Series Processors
- AMD Ryzen 5000 Series Processors with Radeon Graphics
- AMD Ryzen 7020 Series Processors with Radeon Graphics
- 2nd Gen AMD EPYC “Rome” Processors
Impacts on OVHcloud products
In response to that event, we immediately reviewed the security bulletin and technical information and determined the following potential impact on our products.
Public Cloud | All products | Not impacted |
Hosted Private Cloud | All products | Not impacted |
Web Hosting & Domains | All products | Not impacted |
Bare Metal cloud | ADVANCE-1 Advance STOR-1 Advance STOR-2 Game-1 Game-2 HGR-SDS-2 HGR-HCI-4 HGR-HCI-5 HGR-HCI-6 Infra-2 Rise-GAME-1 Rise-GAME-2 Scale-1 Scale-2 Scale-3 | Potentially impacted (only AMD Zen2 powered servers) |
Other commercial ranges of dedicated servers | Not impacted |
How to mitigate the vulnerability:
Customer-initiated mitigation
Loading a patched microcode at boot with a firmware package update
This solution will trigger the update of the processor microcode through an operating system update (the linux-firmware package for instance). You might do it as soon your OS editor or community distribute the updated package. This method is dependent on your distribution or Operating system editor and will only work if the appropriate microcode has been provided by AMD. As of today, only “Zen2 Rome” and “Zen2 Castle Peak” are covered by this method.
Mitigation with an updated Kernel
When an update of the microcode is not available via a firmware update package, you may update the Kernel with a version that implements a mitigation by configuring a so-called “chicken bit” to deactivate the faulty processor feature. It might impact the performance of the system. This solution will be included by OS editors when they backport a new version of the Linux kernel. We recommend our customers to follow this mitigation strategy in priority since it is the most efficient as it doesn’t depend on whether an updated microcode is provided by the hardware vendor.
As an alternative, you might set the chicken bit manually without relying on the kernel update. However, we do not recommend this solution that may be risky for your system.
OVHcloud-initiated mitigation
OVHcloud teams are working to implement transparent solutions that will ensure the patched microcode is updated in a transparent way for our customers. Those solutions will be deployed progressively on our servers. Two main options are being evaluated.
Using OVHcloud iPXE
The microcode update may be loaded by the bootloader when the standard OVHcloud netboot is used by customers (the most common configuration). Once it is available, rebooting the server through the OVHcloud customer interface will cause it to load the updated microcode before booting to disk, which will mitigate the vulnerability. However, if you’re booting on disk without using the OVHcloud netboot system, the mitigation will not be applied and you should consider relying on the Operating-System-level mitigation.
Using UEFI
The UEFI firmware update may update the CPU microcode at boot. UEFI firmware updates including the patched microcode will likely be made available by motherboard manufacturers within the next months. Once available, OVHcloud will include this patched microcode on the UEFI for any new delivered server. Customers will then be able to request an UEFI firmware update by contacting the support.
As an administrator of a potentially vulnerable server, what should I do?
The first action is to check if your server is impacted by the vulnerability using the following tool (Linux-only) developed by our team:
# wget https://raw.githubusercontent.com/speed47/spectre-meltdown-checker/master/spectre-meltdown-checker.sh
# sh spectre-meltdown-checker.sh --variant zenbleed --explain
If the tool says “NOT VULNERABLE”, then you are already safe and no further action is needed.
If the tool says “VULNERABLE”, you should then evaluate your exposition to the threat.
It is necessary to determine if the server context allows to run code from an untrusted origin. If the server is used to provide services to untrusted end-users that can execute code (VPS, Container, mutualized hosting, etc.), or is used as a desktop in the cloud browsing the Web (hence possibly running 3rd party Javascript payloads), then your server might be at risk. If the server is used only by trusted users and/or does not allow to run untrusted code, the risk of exploitation is probably quite low. Please note however that this vulnerability might allow an attacker to gain extra privilege in a chained attack, it could be used for persistence or lateral movement in a complex kill chain.
Based on this evaluation, you should determine the emergency to trigger a mitigation and choose the most appropriate one.
What OVHcloud is working on:
Our technical and support teams are working to ensure the risk is lowered for every of our customer impacted by the vulnerability. We mostly focus on:
- Informing impacted customers to ensure they take the risk into account in their operations and implement mitigation appropriately
- Developing and integrating updates in our automation to cover the risk in a transparent way for our customers.
- Security watch of the vulnerability exploitation in the wild to define the appropriate extra mitigations we can implement to protect our customer infrastructures
External references
https://www.amd.com/en/resources/product-security/bulletin/amd-sb-7008.html
https://lock.cmpxchg8b.com/zenbleed.html
https://www.openwall.com/lists/oss-security/2023/07/24/1
https://github.com/torvalds/linux/commit/522b1d69219d8f083173819fde04f994aa051a98
https://github.com/speed47/spectre-meltdown-checker/
https://lists.debian.org/debian-security-announce/2023/msg00151.html
https://packages.debian.org/bullseye/amd64/amd64-microcode
https://ubuntu.com/security/CVE-2023-20593
https://almalinux.org/fr/blog/zenbleed-patch-call-for-testing/
https://access.redhat.com/security/cve/cve-2023-20593
https://github.com/google/security-research/tree/master/pocs/cpus/zenbleed