At OVHcloud, we are constantly looking for ways to improve our operations and reduce our impact on the environment. This has been a defining part of the company since 1999 and is a key part of our organisational DNA and our commercial model.

We are very proud to present the new Smart Datacenter cooling system, which significantly improves energy and water efficiency while delivering a significant reduction in carbon impact across the entire cooling chain, from manufacturing and transport to daily operations.

The system is a new way of building and deploying datacenter infrastructure, changing how we manage and monitor water supply and demand, using a combination of industrial design, IoT sensors and AI innovation, specifically in our smart racks, advanced cooling distribution units (CDUs) and intelligent dry coolers.

Smart Datacenter delivers a reduction in power consumption of up to 50% across the entire cooling loop, from server water blocks to dry coolers, and consumes 30% less water compared to OVHcloud’s earliest design, driving major sustainability benefits. The system also uses complex mathematical models capturing detailed rack-level and environmental data to optimize cooling performance in real time. Furthermore, all operational data is fed into a centralized data lake, enabling cutting-edge artificial intelligence to predict, adapt, and enhance system efficiency and reliability.

Let’s get into the detail.

The system has three main components:

1. Smart Racks: These are designed with an innovative hydraulic “pull” architecture, where each rack autonomously draws exactly the water flow, pressure, and temperature it needs, dynamically adapting to server load and performance.
2. Advanced Cooling Distribution Unit (CDU): This is a compact, next-generation primary loop unit that autonomously balances flow and pressure across all racks without manual intervention or any electrical communication. It uses only hydraulic signals (pressure, flow and temperature of water) to “understand” rack demands and continuously optimizes operation for lowest power consumption and extended pump lifespan.
3. Intelligent Dry Cooler: This is operated seamlessly by the CDU, eliminating the need for separate control systems (“brains”) on both the dry cooler and the CDU. This unified control architecture ensures optimized, coordinated performance across the entire cooling infrastructure.

OVHcloud’s new Single-Circuit System (SCS) replaces the previous Dual-Circuit System cooling architecture (DCS), which consisted of a primary facility loop and a secondary in-rack loop separated by an in-rack Coolant Distribution Unit (CDU), installed inline directly after the rear door heat exchangers (RDHX), as shown in Figure 1. The CDU housed multiple pumps, several plate heat exchangers (PHEX), and a network of valves and sensors.

Figure 1. Dual-Circuit System cooling architecture (DCS) vs Single-Circuit system (SCS).

That previous design maintained turbulent flow through water blocks (WBs) using the in-rack CDU to regulate flow and temperature differences, ensuring performance despite OVHcloud’s ΔT of 20 K on the primary loop (far higher than the typical market value around 5 K).

Removing the in-rack CDU — replaced by a Pressure Independent Control Valve (PICV), a flow meter, and two temperature sensors on each rack — simplifies the system to a single closed-loop, where the flow rate through servers is dictated directly by the primary loop, adapting dynamically to rack load density. On the rack side, the system adapts the exact flow the rack requires by analyzing the water behavior and performing iterative, predictive thermal optimization considering IT components and the supplied water temperature and flow. This results in lower inlet water temperatures at the server level due to the elimination of the in-rack CDU’s approach temperature difference, and reduces electrical consumption, CAPEX, carbon footprint, and rack footprint.

To prevent laminar flow and maintain heat transfer efficiency at low flow rates, OVHcloud introduced a passive hydraulic innovation by arranging servers into clusters connected in series with servers inside each cluster connected in parallel, rather than all servers in parallel. This ensures higher water flow through individual servers even when the rack density is low. While this increases system pressure drops depending on cluster configuration, it results in better thermal performance and all servers receive water at temperatures equal to or lower than in the previous DCS design.

The racks operate on a novel hydraulic “pull” principle — where each rack draws exactly the hydraulic power it requires, rather than being pushed by the system. The  CDU then dynamically adapts the overall hydraulic performance of the primary loop, balancing flow and pressure in real time to match the actual demand of the entire data center.

A key breakthrough is the CDU’s communication-free operation: it requires no cables, radio waves, or other electronic communication with racks. Instead, it analyzes hydraulic signals — pressure, flow, and temperature fluctuations within the water itself — to understand each rack’s cooling needs and adapt accordingly. This eliminates complex telemetry infrastructure, reduces operational risks, and enhances system reliability. To ensure water quality and system longevity, water supplied to the data center is filtered at 25 microns, and multiple sophisticated high-precision sensors continuously monitors water quality in real time.

The CDU is 50% smaller than the previous generation and manages the entire thermal path — from chip-level water blocks, through the racks and CDU, to the dry coolers.

The newly designed dry cooler is also 50% smaller than the previous model and features one of the lowest density footprints worldwide. Thanks to years of thermal studies on heat exchangers by the OVHcloud R&D team, it has 50% fewer fans, resulting in very low energy consumption, while also reducing noise. Its compact size means that we can also transport more units in the same truck!  This design achieves a 30% reduction in water consumption compared to OVHcloud’s earliest dry cooler design. A key innovation in the dry cooler is its advanced adiabatic cooling pads system, which cools incoming hot air before it passes through the heat exchangers. This high-precision water injection system is the first of its kind, and adjusts water application based on multiple sensors and extensive iterative calculations, including data center load, ambient temperature, and humidity levels.

Unlike traditional adiabatic systems, the pads’ system does not use a conventional recirculation loop. Instead, water is injected when needed onto the pads via a simple setup consisting of a solenoid valve and a flow meter, eliminating complex hydraulics such as pumps, filters, storage tanks, level sensors, and conductivity sensors. The system maintains water quality and physical/chemical properties through careful design, drastically simplifying operation and reducing maintenance needs.

The CDU continuously analyzes data from up to 36 sensors distributed across the CDU itself and the associated dry cooler. It also collects operational data from solenoid valves, pumps, and dry cooler fans across the infrastructure loop. All components are monitored and managed by the system’s central intelligence—the CDU’s control panel—providing a comprehensive understanding of the entire system’s behavior, from the data center interior to the external ambient environment, ensuring real-time performance oversight and precise thermal regulation.

Through this iterative and precise control of water injection, the system optimizes cooling performance and Water Usage Effectiveness (WUE), ensuring minimal water consumption without sacrificing thermal effectiveness.

Advanced System Analytics, Learning & AI Integration

The entire system is designed to continuously analyze the thermal, hydraulic, and aerodynamic behaviors of the various fluids along the cooling path. It uses daily operational data to learn and adapt its performance dynamically, optimizing cooling efficiency and reliability over time.

The CDU’s brain—the control panel—aggregates data from 36 sensors distributed across the CDU and dry cooler, as well as operational data from solenoid valves, pumps, and dry cooler fans within the infrastructure loop. It also collects critical rack-level information, including flow rates, temperatures, and IPMI data that reflect IT equipment behavior and performance. All this operational data is pushed to a centralized data lake for parallel analysis, which forms the foundation for the next step: integrating cutting-edge artificial intelligence (AI). This AI will leverage the continuously gathered data and learning processes to enhance predictive capabilities, optimize future operating points, and enable fully autonomous decision-making.

This combination of real-time learning and AI-powered analytics will provide advanced diagnostics, predictive maintenance, and proactive management — maximizing uptime, reducing costs, and driving ever-greater sustainability.

Iterative Control System Innovation

The iterative control system manages all aspects in real time, hands-free, continuously learning from sensor data and operational feedback. It applies algorithms to the pump speed on the CDU, the fans on the dry cooler and the solenoid valve controlling water injection on the adiabatic pads.

On the rack side, the system uses a PICV valve, flow meter, and two temperature sensors to adapt the exact hydraulic flow needed by each rack, considering IT load and incoming water conditions, iteratively optimizing thermal performance and energy efficiency.

On the CDU, the system analyzes combined hydraulic signals from all racks alongside ambient data center conditions, dynamically balancing flow and pressure across the entire data center infrastructure without human intervention.

Furthermore, OVHcloud’s cooling system integrates intelligent communication between cooling line-ups to enhance failure detection and simplify maintenance. This is achieved through embedded freeze-gaud and resilience-switch mechanisms that ensure continuous operation and system resilience. The freeze-gaud system is designed to protect the dry coolers in sub-zero ambient conditions by keeping water circulating through their heat exchangers. If the overall loop flow drops below a predefined threshold, the system automatically opens a normally closed bypass valve to maintain circulation—preventing freezing despite the use of pure water (without glycol) as the cooling medium. The resilience-switch system maintains redundancy by hydraulically linking multiple cooling lines. In the event of failure or overload on one line, normally open solenoid valves isolate the affected line, while bypass valves on neighboring lines open to redistribute water flow and maintain cooling performance. This dynamic and autonomous valve management ensures uninterrupted service and rapid fault response.

Drawing inspiration from autonomous control methodologies in leading-edge industries, the system predicts future behavior based on iterative calculations, dynamically adapting pump speed, fans speed and solenoid valves openings to converge rapidly on optimal operating points. It also adjusts performance based on external constraints such as noise limits, water availability, or energy costs — for example, consuming more energy to save water in water-stressed regions or balancing noise restrictions in urban deployments.

This unique, self-optimizing end-to-end control system maximizes energy efficiency, sustainability, and operational simplicity, extending pump life cycles and ensuring the most environmentally responsible data center cooling solution available today.

This vertically integrated, autonomous system — including smart racks, the advanced CDU, and the intelligent dry cooler — represents a world-first in end-to-end, intelligent, sustainable, communication-free, and data-driven data center cooling.

Why is this important?

This innovation is critical because it marks a decisive step toward radically more sustainable, efficient, and autonomous data center cooling — addressing the growing demands of digital infrastructure while reducing its environmental footprint.

By using fewer, smaller components, we are saving power, cutting transport costs and reducing carbon impact. Using fewer fans on the dry cooler means up to 50% lower energy consumption on the cooling cycle – and the new pad system means 30% lower water consumption in the cooling system. The system is fully autonomous, avoiding human error. A temperature gradient of 20K on the primary loop – four times higher than the industry average – means that flow rates can be lower and water efficiency is higher. The system doesn’t rely on Wi-Fi or cabling, and the predictive control constantly adapts to external conditions or situational goals, feeding into a data lake to help continuously optimize performance.

Today’s world is built on technology, and datacenters are a key part of that technology, but there is a pressing need to ensure we can maintain human progress without incurring a significant carbon footprint. Power and water efficiency is a key part of this equation in the datacenter industry, and our innovation in the Smart Datacenter continues our trajectory of supporting today’s needs without compromising the world of tomorrow.

Recognizing that data is now key to many industries, OVHcloud supports its clients in their data journey. Without compromising data sovereignty and trusted cloud. Nor performance or security. When building its open and trusted Cloud, OVHcloud ensured that customers, partners and cloud users alike can keep their freedom of choice on a secure space. Providing users with clear, transparent, and accessible information on regulations applying to their data. OVHcloud offers a trustworthy environment. A place where data, including the most sensitive one, is protected with the highest standards and transparent certifications. This unique proposition, backed by a resilient infrastructure, ensures that your data stays your data. All the while being immune to extra-territorial laws. We are confident that data sovereignty and trusted cloud can make a difference.

What is Trusted Cloud and how does it support data sovereignty?

Trusted Cloud provides the highest levels of security, privacy, and transparency for its users. It gives industries the tools and the confidence to leverage the cloud for their digital transformation, while preserving their values, interests, and competitiveness.

In other words, Trusted Cloud provides data protection and sovereignty, that encompass both cybersecurity and legal dimensions:

• It ensures that the data of its users is encrypted, and backed up. With this in mind, users keep track of their data usage and have full control and ownership over their data and the insights derived from it. Trusted cloud complies with the strictest regulations and certifications (SNC, C5…) and offers contractual guarantees to its users.
• It gives its users the visibility and the control over the operations of the cloud provider, such as where the data is stored, who can access it, and how it is processed. 

OVHcloud Trusted cloud solutions contribute to the development of a European cloud market that is diverse, dynamic, and resilient. For this reason, our solutions comply with various certifications, including the highest known today in protecting organizations, industries, and governments’ interest: SecNumCloud.

What is SecNumCloud qualification all about?

SecNumCloud is issued by the French National Cybersecurity Agency (ANSSI). Its goal is to guarantee that the cloud service meets the strictest requirements in terms of data protection, and data sovereignty. The SecNumCloud standard covers requirements relating to the service provider, its employees and the way in which services are provided. It guarantees the robustness of the solution in the face of the most common cyber-attacks, as well as the rigor and formalization of the service provider’s processes and methods. The qualification process is a long-term guarantee, ensuring a relationship of trust between service providers and ANSSI.

The requirements of SecNumCloud’s most recent qualification (SecNumCloud 3.2) guarantee the protection of cloud services from non-European law thanks to a combination of three types of measures:

– technical: watertightness test standards of information systems;

– operational: only the service provider can intervene on the resources supporting the service;

– legal: exclusive application of European law.

What is at stakes with SecNumCloud is the use of this qualification for both public administrations and private organizations. Domains – industry sectors – such as finance and banking, health and cybersecurity, anchor their expansion to their ability to protect cloud data and cloud services.

OVHcloud has recently received the SecNumCloud 3.2 qualification on three distinct datacenters located in France: Gravelines, Roubaix and Strasbourg. With an unparallel commitment to SecNumCloud, OVHcloud is a firm believer that the qualification can guarantee the future of business for industries and administrations. For which cloud is now a center piece and a driver for disruption. By providing them with a trusted cloud solution that enables to store, process, and share their sensitive and strategic data.

OVHcloud can support European public and private organizations:

– by offering them a range of cloud services SecNumCloud qualified (such as Hosted Private Cloud, which is a dedicated, scalable, and hybrid cloud solution);
– OVHcloud’s SecNumCloud services are hosted in its own data centers in France;
– OVHcloud ensures that no data is accessible, transferred, or processed outside the European Union.

Extended certifications to meet local regulations

While SecNumCloud is the highest standard of security and data protection available today, our Cloud is multilocal and meets a number of stringent international certifications, such as:

1. ISO/IEC 27001 (International): Widely adopted internationally, including in European countries, this standard focuses on information security management systems.

2. C5 (Cloud Computing Compliance Controls Catalog) certification (Germany): the certification is a recognized standard for cloud service providers that emphasizes information security and data protection. It is designed to ensure compliance with German and European data protection requirements. C5 incorporates elements from ISO 27001 but adds specific criteria tailored to cloud services.

3. ENS (National Security Scheme or Esquema Nacional de Seguridad) certification (Spain): it establishes a common framework of basic principles, requirements, and security measures for adequate protection of information processed and services provided by electronic means. OVHcloud has obtained the highest level of the ENS certification ENS High.

How OVHcloud is paving the way for trusted cloud in Europe

Trusted cloud is pivotal to our customers ensuring they can continue innovating through a secure, reliable, open, and sustainable cloud.

At a European level, OVHcloud meets the most demanding certifications in the countries/markets in which it operates.

Our Group supports the establishment of a future European Cloud Certification Scheme (EUCS), which would provide users with harmonized EU-wide certification and maintain the same high standards of transparency and data protection, including against extraterritorial legislation for the most sensitive data.

If you want to learn more about trusted cloud and how OVHcloud can help you achieve your digital ambitions, please visit our website.

We are convinced that trusted cloud is the future of cloud computing. That it is a strategic asset for the digital sovereignty and the competitiveness of the ecosystems in industries. We invite you to join us in this journey and to discover how OVHcloud can help you to achieve your digital ambitions with trust and confidence.

Links SecNumCloud Trusted Cloud Sovereignty

(1)SecNumCloud: Le visa de sécurité ANSSI

OVHcloud. https://www.ovhcloud.com/fr/enterprise/certification-conformity/secnumcloud/

Throughout the year, there are other ways we help the community with grants to access our infrastructure for testing or evaluation purposes. But that’s not all. Deploying at an unprecedented scale a number of open-source solutions we might find ourselves in the unique position to help the community through development and patching efforts. This recently happened within our storage team, and we thought sharing our experience would be worth a read.

OpenZFS and FreeBSD at OVHcloud

Among the Storage Product Unit, our mission is to deliver innovative file storage services based on different hardware and software stacks. We take the time to test and validate new stacks of technologies to deliver high performance and high availability storage with the utmost care for data protection all the while keeping costs reasonable. With a complete portfolio of storage solutions, we use different set of software foundations for different storage access modes including OpenZFS for file storage.

Based on our comprehensive testing, we chose FreeBSD for some of our offers like NAS-HA or Datastore NFS. One of the reasons is that FreeBSD is managed as a complete operating system with OpenZFS being a first-class citizen and natively integrated. It benefits from many years of experience across many teams ensuring quality and security.

FreeBSD’s release management goes through multiple steps from idea inception to public releases:

• Technical reviews by peers,
• Current branch to test,
• Stable branch for wider user base testing,
• Release candidate test,
• Normal release,

At the same time, patches for software are released to fix vulnerabilities and bugs.

The ports collection is well-designed and simple. While FreeBSD has binary packages, which are handled by the pkg package manager, it also has the ability to compile software from source, allowing user to select desired options of compilation.

FreeBSD also provides tools like poudriere. Poudriere is a utility for creating and testing FreeBSD packages. It makes it easy for users to build and set up their own binary package repository in which packages are built with their own options.

FreeBSD has over five hundred system variables that can be read and set using the sysctl utility. These system variables are used to apply some changes to a running FreeBSD system. This includes many advanced options of the TCP/IP stack and virtual memory system that can improve performance.

Our goal today is not to come up with an exhaustive list of technical advantages on why to use FreeBSD. It would probably require one to many full blog posts. Keep in mind jail, pf, linux binary compatibility and so on…

As we use FreeBSD, we are convinced that ZFS is a high-performance file system with replication, compression, encryption, and snapshots. If you want more details, our very own Frédéric Zind said it all during his tech talk at Very Tech Trip 2023 : 🇬🇧 / 🇫🇷

To illustrate what we are doing with OpenZFS and FreeBSD, let’s take an example with NAS-HA. This product is a file storage service (active/passive cluster illustrated below) built upon 2 nodes and a ZFS-based filesystem shared with NFS and/or CIFS. NAS-HA is a good example of how we integrated both FreeBSD and OpenZFS to build an open and secure storage service dedicated to versatile workloads (centralized storage for private or public cloud instances, bare metal servers…). With NAS-HA we are talking about several thousands of servers holding Petabytes of data.

Open-source contribution

To demonstrate how OVHcloud participates in open-source communities, we are sharing what we came to encounter in 2022 and how we fixed it.

Through our constant monitoring system, our operate team observed a condition where some filers had no more allocated memory due to a lack of free vnodes. As a reboot was necessary to unhang filers, the operate and engineering teams investigated the root cause. They found several leads but unfortunately couldn’t reproduce the incident. To improve the situation, the following was done:

• Double the base reservation of memory for OS (vfs.zfs.arc_max). Since our FreeBSD servers are booted into an in-memory OS without swap, we have to be careful that everything has sufficient resources and do not fight ZFS for memory,
• Create a buffer memory zone (vfs.zfs.arc.sys_free),
• Change the speed of release memory with no effect,
• Disable periodic checks because default checks were consuming vnodes (a vnode is a structure representing a filesystem entity like a file, directory, device node) and were unneeded for our use cases,

Even after applying these changes, some crashes were still happening. Then, we noticed that the incident seemed to be related to a non-holded snapshot during a restoration. There was a snapshot rotation while the snapshot was used during restoration, and it caused a deadlock. At this time, we still had no repro.

What we found:

• Huge spike in vnode usage,
• Spikes seemed to be correlated to the rotation and restoration of our backups (available through the .zfs directory on our NFS shares),

What we managed to:

• Limit the interaction of the .zfs directory to our essential operations,
• Manually set vnode limit to 10 million (via sysctl),

As a result, parallel rsync copies successfully finished. No more problems were observed during our quality assurance period (5 days), so we reached out to concerned communities.

Random servers were still experiencing incidents. The community started to report similar behaviors. In parallel, we noticed that if an NFS client browses the .zfs directory exposed by our servers all ZFS operations on snapshots would end up in a deadlock state afterwards. This happened only on OpenZFS 2.x (FreeBSD 13.x) and only a reboot would unfreeze the server. Problem reports have been escalated to OpenZFS (here) and Freebsd (here). FreeBSD’s users confirmed our testing with a procedure to reproduce the incident.

Thanks to OVHcloud’s FreeBSD developers, we were quickly pointed at a FreeBSD developer, Mark, in touch with ZFS matters and we let him know what we have done, found, and tested.

Leveraging the community’s feedback, Mark suggested patches and our engineering team tested each one of them. Communication was smooth and the root cause was finally identified: a Linux-specific race condition. Technically speaking the issue was described the following way: “zfsctl_snapdir_fid() sets fid_gen to 0 or 1 depending on whether the snapshot directory is mounted. On FreeBSD it fails, making snapshot dirs inaccessible via NFS.” (See commit on OpenZFS’s github)The patch was committed and merged in FreeBSD and OpenZFS’s repositories.

Why didn’t we catch this bug during our preprod tests? Because of mathematics. With fewer servers, there is lower probability to have concurrent access to .zfs directory. This test has since then been added to our test book.

Main takeaways

As a cloud actor, we are committed to open-source and many people at OVHcloud spend incredible amount of time to reproduce bugs, read code, test, and ultimately submit patches. Some of us even develop open-source projects (CDS, bastion by example are shining great examples) and/or during their free time.

At the Storage Product Unit, we are first and foremost geeks that love FreeBSD and OpenZFS. And well managing more than 10k servers daily is quite the experience.

Most importantly, we want to give back to open-source communities like Linux Foundation through the Ceph foundation, Cloud Native Computing Foundation, Python community, PostgreSQL community, FreeBSD, OpenZFS and more. If you want to learn more, you can check our open innovation program as well as OVHcloud GitHub repo.

Ever since 2003, OVHcloud has developed watercooling at scale for its servers in a free cooling approach that prohibits the use of power-hungry air conditioning thus reducing power costs and water usage. Over the years, our technology has been refined from generation to generation. All these years of work allowed us to see that one crucial but often overlooked parameter is the Delta T, a parameter that measures the difference in temperature for the water coming in and out of the data centre. From this single parameter comes greater efficiency particularly when it comes to Water Usage Effectiveness (WUE). A parameter that we continue refining with the most recent milestone being the Delta T=20K. Ready for the deep dive?

Overview of OVHcloud cooling DC principles

A typical OVHcloud data centre benefits from an indirect free cooling technique, also known as IFC, with the deployment at scale of a liquid cooling system. It helps in improving global data centre power efficiency with up to 50% energy savings compared to conventional mechanical cooling. With such a cooling strategy that leverages liquid cooling and air cooling, OVHcloud’s data centres aim at being highly efficient.

Our proprietary watercooling system recovers fatal heat generated by our servers through carefully crafted waterblocks offering direct contact with the chips. If we break down the system, there are four cooling bricks in the data centre:

• Dry Cooler (heat rejection system),
• Pumping Substation (PSS),
• Rack Rear Door Heat Exchanger (RDHX) and Cooling Distribution Unit (CDU),
• Server waterblock,

A typical water circuit in an OVHcloud data center goes like this: cold water is pumped to the rack from a pumping substation (PSS) before reaching the Rear Door Heat Exchanger (RDHX) installed at the rack level. Each rack comes with 3 RDHXs, each door having its own set of fans supporting air cooling capacity. Water then flows through the CDU or Cooling Distribution Units: placed inside the rack, CDU are comprised of pumps and plate heat exchangers (PHEXs). They distribute liquid to the waterblocks, effectively acting as a watercooling system, where the IT equipment heat will be transported back to the dry coolers located outside of the data centre for heat rejection.

Playing an important role in the cooling infrastructure, the dry coolers use external air to cool down the circulating liquid pumped from the data centre without any refrigeration. Depending on the climatic conditions, the most efficient evaporative cooling (EC) system to date is used to cool the outside air before it reaches the fins of the dry cooler.

Kelvin, how delta T is measured and why it’s a key parameter

In the research field, the usual norm is to express temperature difference in Kelvin while all temperature measurements are in °C, F or K. In this document, temperature is express in °C, while temperature difference is expressed in K. When it comes to our data centres, Delta T is the difference of water temperature between the inlet and outlet of a rack (in other words the in/out water temperature of the data center): the greater this difference is, the more efficient the data centre will be. Why? An increase in the temperature difference in the racks (dT) causes a decrease in the flow rate of water supplied per rack allowing the use of lower pumps capacity. Thus, an increase in the temperature difference permits a higher heat load per pumping unit and dry cooler for an equivalent PSS hydraulic capacity and heat rejection system.

From Delta T=5K to Delta T=20K

As we have seen, from a greater temperature difference comes more efficiency. It also means that the performance of the dry cooler is significantly improved. Indeed, water enters at higher temperature levels, while fan and evaporative cooling water consumption per kWh IT is reduced because a lower air flow rate is required. That later consequence allows the optimization of the ventilation capacity.

Since 2019, we upgraded to a new generation of cooling infrastructure for our most recent data centres capable of operating at a Delta T=20K instead of Delta T=5K. We rely on a lower number of dry coolers per data centre, yet more efficient ones, with smaller pumps and reduced piping. The result is a potential higher footprint (kW/m2) for the data centre, lower overall carbon footprint, better PPUE and an improved WUE, as well as reduced CAPEX and OPEX.

• Delta T=5K explained

In a traditional OVHcloud data centre with a Delta T=5K setting, water temperature is to be maintained at 25°C providing inlet air temperature at 30°C and inlet water temperature of the waterblock at 35°C. As shown in the figure below, two circuits in parallel are connected to the rack.

Through constant development, we improved the cooling design of our rear door heat exchanger (RDHX) to increase the inlet water temperature and reduce pinch between cold air and cold water: this way, the data center inlet water temperature is now set at 27°C against 25°C previously.

• Delta T=20K explained

Building up on the notable improvements we discussed, we were also able to setup a serial circuitry with PHEX and RDHX on the rack level in order to achieve a complete Delta T=20K environment as shown in the figure.

With water temperature maintained at 27°C, the inlet air temperature of the data centre remains at 30°C and inlet water temperature at 45°C for the waterblock. The data centre water outlet temperature is set at 47°C. On a sidenote, our robust design gives us confidence in the ability to operate with a higher temperature profile, still in a Delta T=20K setting, like 32-52°C.

What are the benefits of Delta T=20K VS Delta T=5K? And what are the advantages when applied to different data centre locations?

To measure the gains of OVHcloud’s IFC system operating with a Delta T=20K, we conducted an extensive series of benchmarks in various locations with the same Cooling Module configured first as Delta T=5K then as Delta T=20K. We should be able to highlight significant gains in terms of water consumption.

We elected for the following methodology. We collected and analyzed the weather data for the past 40 years, including temperature and relative humidity with a record every 30 minutes. For the Roubaix site for example, the warmest year happened to be 2019. Figure below shows the temperature frequency in hours extracted from climatic data recorded at Roubaix in 2019. The key hours are between 5 and 20 °C.

Based on our working principle, we can deduct the WUE in every country where we have a data centre. For your reference, in this scientific paper we estimated the theoretical WUE for some of our locations that offer the best possible representation of various climatic conditions: Roubaix (France), Mumbai (India), Singapore, VintHill (USA). For accuracy purposes we’re sharing below the estimated WUE for Roubaix:

This theoretical result is impressive. Yet, it’s important to have a better sense of what WUE can represent: we estimate that in an IFC with a Delta T=20K setting, 381 m³ of water are enough to cool a 600 kW DC versus 964 m³ in an IFC with Delta T=5K setting.

The table below is a summary of the most glaring changes between Delta T=5K and Delta T=20K:

Versus the competition?

To date, there are two different families of water liquid cooling techniques employed within data centres: close-coupled cooling and direct liquid cooling (see table). Watercooling solutions take advantage of the heat-carrying capacity of the water that is 3,3 times higher than that of air for a dramatic reduction in thermal resistance. With CPU and GPU being the most power-hungry parts in a server, liquid cooling via direct-to-chip contact contributes in returning warmer water from IT equipment than that of closely coupled cooling systems. Logically, the coolant temperature directly impacts energy savings and cooling efficiency participating in a significant decrease in the total DC site energy consumption by 14 to 20%¹. Please note, more information on the above can be obtained here.

What’s next?

As we celebrate the 20th anniversary of our watercooling technology this year, we continue to innovate. We have been working on designing a new breed of data centre employing climate friendly technologies when it comes to cooling. In a global context of surging energy prices and several water shortages in the 2022 summer, it becomes quite obvious as to why our previous technological bets pay off. With best-in-class PUE and WUE for our data centres, we continue to work tirelessly on improving our footprint within the ecosystem illustrating our deep commitment towards sustainability. Hopefully by now you have come to realize how important to the data centre is the Delta T parameter. From the infrastructure point of view to the tangible water efficiency, Delta T=20K contributes in adapting to various geographies and ultimately water conservation.

The water consumption of data centres has indeed a tremendous impact locally going that far as to affect river flow, groundwater levels and influence overall water resources for a whole territory. Thus, reducing WUE proves critical for any modern data centres to lower the overall impact on local communities.

The impressive saving of water illustrated by our WUE also contributes to a streamlined PUE for compliance with ISO/IEC 30134 standard. Last, but not least, the demonstrated advantages of Delta T=20K will prove to be a solid basis for the next generation of improvements our teams develop.

1: Performance analysis of new liquid cooling topology and its impact on data centres –  https://www.sciencedirect.com/science/article/abs/pii/S1359431122006779?via%3Dihub

Here at OVHcloud, we pioneered free cooling, and have used water cooling at scale to cool down our servers since 2003. This unique approach helped us to set high standards in PUE/WUE indexes for our global datacentres, all the while lowering our overall carbon impact.

With the evolution of IT equipment power, OVHcloud R&D worked on a new generation of datacentre cooling solutions. One of the most promising technologies today is immersion cooling, thanks to direct contact of the fluid with hardware components. It allows us to build datacentres all over the world maintaining low PUE/WUE indexes even in harsh climatic zones.

In this blog post, we will explain how we developed our own breed of immersion cooling to achieve several goals, from improving datacentre footprint and reducing power consumption, to adapting to new operating conditions and improved overall reliability.

OVHcloud Hybrid Immersion Liquid Cooling

Immersion Cooling is the practice of submerging electronic devices in a thermally and not electrically conductive liquid.

The typical immersion cooling solution usually employs pumps, heat sink structures, heat exchangers, condensers, sealed evaporative equipment etc. that either consume large amounts of energy to operate, require sealed casings, or occupy relatively large surface areas that limit the number of servers that can be implemented.

OVHcloud developed its own Hybrid Immersion Liquid Cooling technique. It is comprised of a direct to chip water cooling system and a passive natural single phase immersion cooling system containing two fluids:

• Water: cooling a heat sink through water blocks disposed on CPUs and GPUs with the same solution used in all OVHcloud servers and a proprietary serpentine convection coil connected to a pumping substation (PSS) and a dry cooler to evacuate heat outside the DC.
  
• Immersion Cooling (IC): the fluid is contained in a tank and cools all IT equipment in the server rather than just CPUs and GPUs; that fluid basically replaces the air circulating in OVHcloud servers thus enhancing the efficiency of any component not cooled by OVHcloud water cooling systems.

It comes with a new 3 floor server rack design, in a library format, which we can populate with up to 48 servers (1U) or 24 servers (2U), in a book format. Each server is submerged in its own tank for independent cooling, allowing large scale deployments. In addition, each server benefits from a specific monitoring system of all environmental factors ensuring safe and secure operation at the server level.

Benefits in terms of power consumption and efficiency

OVHcloud Hybrid Immersion Liquid Cooling has several advantages that makes it an efficient solution:

• The new passive rack design means there are neither pumps nor fans, resulting in zero cooling electrical consumption at the rack level
• High power racks can operate with datacentre inlet temperatures of up to 45 °C, allowing for different cooling loads at different climatic conditions
• Stay energy efficient using free cooling available on site
• The electrical consumption and CAPEX are further reduced through the elimination of the Evaporative Cooling system (no pumps) used on dry coolers for data centers located in areas where air ambient climatic temperatures are below 43 °C
• The difference between supplied and recovered temperature of DC water (DT) of 20K can be achieved through highly efficient CPU and GPU cold plates and a large thermal contact surface between the serpentine and the dielectric fluid. The patent pending serpentine design occupies a very small area within the server chassis maximizing space for components
• Global datacentre cooling infrastructure power consumption is reduced by at least 20.7% compared to the OVHcloud water cooling system
• The energy consumption of a server per year is reduced by at least 20% compared to air cooled servers and 7% versus water cooled servers
• Reduced power consumption means lower operating expenses (OPEX)

Finally, in the context of ever-increasing CPU/GPU TDP, the solution supports higher computing power density.

A new generation of datacentre

Our new Hybrid Immersion Liquid Cooling approach stands outs with several key advantages for datacentre operators:

• Actual footprint of up to 37U/m² can be guaranteed with no extra pumping and condensers systems. The design is scalable to two or three times its footprint if immersion cooling racks are installed in stacked maritime containers
• With no sealing required nor sophisticated heat exchangers and pumping circuits the capital expenditure (CAPEX) remains low
• With a datacentre outlet temperature of 65 °C, fatal heat is better captured, and affordable heat recovery systems can be envisioned
• More flexibility and freedom in design at OVHcloud, where the new proposed rack design requires access to the front side, unlike classic racks requiring access from both the front and rear sides.

Redefining datacentre indexes

With such a new game-changing technique, the usual datacentre performance indicators evolve dramatically.

Enhance PPUE^[1] to 1.004: PPUE or partial power usage effectiveness defines a certain portion of the overall PUE of a datacentre within a clearly defined boundary. Infrastructure cooling PPUE is as follows:

Reduce WUE to 0 for datacentre located in areas where ambient air climatic temperatures are below 43 °C: WUE or water usage effectiveness is a sustainability metric to measure the amount of water used by datacentres to cool IT equipment. Annual site water usage includes water used for humidification and water evaporated on-site for energy production or cooling of the datacentre and its support systems. It is defined as follow:

Environmental considerations

The Hybrid Immersion Liquid Cooling technique relies on a non-volatile dielectric hydrocarbon fluid. OVHcloud R&D tested and qualified the system with a variety of market fluids. While each fluid has different properties, we paid special attention to their compliance with the following considerations:

• Non-corrosive
• Ultra-low vaporization
• Nontoxic
• Non allergenic
• Biodegradability in 30 days
• Very high flash point
• Dielectric Strength up to 42 kV
• GWP^[2]=0
• ODP^[3]=0

Operational changes within the datacentre

The new 3 floor library format rack design is footprint optimized for a high density of servers and incidentally compute power per square meter (U/m²). It could be fitted with manual/automated guidance to extract servers, reducing the maintenance impact of one server on the overall rack.

The new Hybrid Immersion Liquid Cooling technique also improves reliability reducing failure rate by up to 60 % by eliminating risk of dust circulation. It contributes to a silent datacentre with the removal of any fan at rack and server levels. Finally, several components are no longer needed:

• No heat exchangers attached to the racks (3 heat exchangers are eliminated per rack),
• 36 fans usually installed per rack are eliminated
• All small fans inside the servers are eliminated
• Cooling modules comprised of pumps and plate heat exchangers are eliminated

What’s next?

The new OVHcloud Hybrid Immersion Liquid Cooling paves the way for unapologetically more compute power in modern datacentres. Its unique patent pending design is sustainable and preserves the opportunity to valorize fatal heat due to higher operating temperatures. Moreover, lower power and water consumption, as well as improved footprint can be achieved with the benefit of adapting to regions with harsh climatic conditions. We are incredibly excited by the very first use cases of Hybrid Immersion Liquid Cooling in the fields of banking, medical and scientific research, not forgetting gaming workloads. We can’t wait to share more with you in the future.

[1]: Partial PUE is used instead of PUE. It is expected that the global PUE will improve in a similar approach which includes the power lost in the energy distribution system [More details about the PPUE and PUE here]. In line with its transparency commitments, OVHcloud will communicate PUE values upon gathering significant data on the long run on a specific location basis.
[2]: GWP stands for Global Warming Potential. The fluid does not contribute to global warming in opposition to most of gas/liquid fluids used in cooling technologies.
[3]: ODP stands for Ozone Depletion Potential. The fluid does not contribute to ozone depletion in opposition to most of gas/liquid fluids used in cooling technologies.