At OVHcloud, we firmly believe that open innovation and close partnerships with the academic world are essential to building a more efficient, resilient, and transparent internet. In this spirit, we are pleased to contribute to research on the Border Gateway Protocol (BGP) – the core routing protocol that keeps the global internet running. Often described as the ‘glue’ of the internet, BGP is what allows thousands of independently operated networks to interconnect and reach one another. The OVHcloud network is one such example, operating one of the largest global backbones.

By delivering cloud infrastructure and services, we enable the bgproutes.io platform to evolve, scale, and provide better insights into the routing data shared across global networks. This partnership aims to help researchers analyse, view, and interpret BGP data from a wide range of global vantage points.

Partnering with academic researchers to drive progress in internet infrastructure

BGP routing plays a critical role in the stability of the global internet. However, its complexity and opacity can present significant challenges, e.g., routing incidents, unintentional and deliberate route hijacks, and the spread of inaccurate routing data, among others.

For many years, the research team at the University of Strasbourg have been developing innovative solutions to make BGP routing more accessible, transparent, and understandable for network engineers and academic researchers.

Their platform, bgproutes.io, serves a dual purpose:

1. collecting routing data from a large number of independent networks, and
2. providing users with powerful tools to analyse BGP data accurately with precision and in real time.

Supporting this initiative allows OVHcloud to further its commitment to promoting scientific collaboration. Through our compute and hosting capabilities, researchers can develop, test, and scale their platform in a secure, high-performance, and scalable cloud environment.

Accelerating network innovation through an open data platform

The bgproutes.io project aims to bring greater transparency to BGP routing by enabling:

• the large-scale collection and aggregation of BGP routes from global networks;
• dynamic, real-time visualisation of routing pathways;
• anomaly detection and behavioural analysis;
• open access to collected data to facilitate further research.

The platform relies on OVHcloud’s infrastructure to process larger amounts of data, enhance visualisation, and integrate new analytical tools, including machine learning capabilities and multi-source correlation engines.

This partnership drives innovation in internet routing by equipping researchers with the ideal environment to test new ideas and offer the global internet community a more powerful and accessible tool.

OVHcloud’s support is reinforced by the expertise of our network engineering teams and high-performance Bare Metal solutions. Combined, they offer the stability, scalability, and flexibility needed to push the boundaries of routing research.

OVHcloud’s commitment to research, open data, and a stronger internet

OVHcloud, through this partnership, reaffirms its long-standing commitment to academic research, open data, and a safer and more transparent internet.

Our support for the University of Strasbourg and the bgproutes.io project contributes to clearer global understanding and improved reliability of BGP routing, which is crucial for a stable internet.

Together, we can make the internet more open, secure, and innovative.

Introduction

The underlying infrastructure of OVHcloud’s Cloud services consists of datacentres connected by a global telecommunication network which carries data to and from end users.

The core network (backbone) features nodes (also known as Points of Presence – PoPs) and long-distance/metropolitan spans (also known as links) which connect the nodes.

The PoPs are located in colocation facilities hosting optical transmission systems (Dense Wavelength Division Multiplexers – DWDM), IP routers, switches and servers.

The links are based on optical fibre routes interconnecting the PoPs and the datacentres, following a topology designed to maintain traffic in the event of multiple span cuts (for resiliency purposes). Two operating models are used:

• Operating model 1: OVHcloud owns the dark fibre cable or rents a pair of dark fibre cables (through long-term Indefeasible Right of Use – IRU) and operates its own DWDM systems to activate wavelengths (10, 100, 400 Gbps point-to-point transmission signals) on top of it.
• Operating model 2: OVHcloud leases the wavelengths activated by long-distance telecommunication operators on their international network (carrier’s carrier market).

In this study, terrestrial and submarine transmission infrastructures are differentiated as their physical realities are dissimilar.

Scope of the study

Items covered by the study:

• POPs (colocation buildings and their technical environment)
• Fibre cables and their underlying infrastructure
• All active telecommunication equipment hosted in the PoPs as well as the line equipment hosted in amplification sites.

Items excluded by the model:

• OVHcloud datacentres network equipment (accounted for in OVHcloud datacentres’ impact inventory)
• ISP (Internet Service Providers) network as well as customer premises equipment.

Environmental impact of the PoPs

1. Electricity consumption (use impact)

OVHcloud measures the electrical consumption of the PoP’s equipment, and of the technical environment of the colocation facility in which it is hosted. All the ancillary systems are taken into account, therefore the PUE (Power Usage Effectiveness) of the sites is de facto included in the model.

The impact factors (per kWh of electricity) are retrieved from the Ecoinvent database.

• Network equipment (manufacturing / distribution / end of life impacts)

OVHcloud reviews all the equipment deployed inside each PoP, including checking their commissioning date (for amortisation purposes).

The equipment reference is used to retrieve the impact factors from the Negaoctet and Resilio v2024.6 databases (amortised over six years). Should the exact model not be found, a similar generic reference is chosen.

• Facilities technical environment (manufacturing / distribution / use/end of life impacts)

Based on the electricity consumption of the technical environment of the colocation facility, the impact factors of each PoP (per contracted kW per year per kWh of electricity) are retrieved from the Ademe PCR datacentre and cloud database.

Environmental impact of the terrestrial links

1. Optical fibre cable (manufacturing / distribution / end of life) impacts

Operating model 1: OVHcloud manages the transmission layer.

The optical fibre cable related impacts are calculated by allocating two strands out of a 288-strand cable. The allocation is then corrected to reflect the 25-year ramp-up before reaching 100% usage of the cable over a lifespan of 60 years. This leads to 0.88% of allocation.

The impact factors (per km) are retrieved from the Ecoinvent database.

Operating model 2: OVHcloud leases a wavelength to a carrier’s carrier.

The assumption is that the carrier follows the same optical fibre cable allocation rules (0.88%). In addition, the impact is pro-rated considering the ramping up of the carrier’s optical systems (leading to 4.32% of 0.88% of allocation):

• Maximum number of channels per DWDM systems: 48
• Maximum load rate of DWDM systems: 85%
• Six-year ramp up to reach maximum load rate of the DWDM system
• DWDM system lifespan: 8 years

In both operating models, 10% of the civil work (trench, ducts) impacts necessary to lay the optical fibre cables are allocated.

• Line optical systems (manufacturing / distribution / use/end of life impacts)

Operating model 1: OVHcloud manages the transmission layer, therefore amplifier types and locations are known. (De)Multiplexers are excluded as they are already accounted for in the environmental impact of the PoPs (see previous section).

Operating model 2: OVHcloud leases a wavelength to a carrier’s carrier. The assumptions are as follows:

• (De)Multiplexers and repeaters are chosen using standard equipment
• Typical distance between two regenerator sites: 500 km
• Typical distance between two repeater sites: 90 km
• Maximum number of channels per DWDM systems: 48
• Maximum load rate of DWDM systems: 85%
• Six-year ramp up to reach maximum load rate of the DWDM system
• DWDM system lifespan: 8 years

Once the optical equipment mapping has been done, the emissions factors for each link are then retrieved from the Ademe ISP – Negaoctet database.

Environmental impact of the submarine links

Submarine cable system (manufacturing / distribution / maintenance / end of life impacts) and electricity (use impact)

Operating model 2: OVHcloud leases a wavelength to a carrier’s carrier from PoP to PoP.

The climate change impact is retrieved from the 2025 Lisbon Suboptic study for both transatlantic and transpacific cable systems (per km and per year).

For the other impact factors (abiotic and water resources use), the submarine cable systems are modelled based on the following assumptions:

• (De)Multiplexers and repeaters are chosen using standard terrestrial equipment
• Cable is considered as a standard MV electrical cable
• Typical distance between two landing stations: 7000 km
• Typical distance between two repeaters: 80 km
• Maximum capacity: 200 Tbps for Transpacific / 350 Tbps for Transatlantic
• Maximum load rate of the DWDM systems: 100%
• Ten-year ramp up to reach Maximum load rate of the DWDM system
• Submarine cable system life span: 25 years

Results (per year)

Conclusion

The above presented methodology allows to assess accurately the environmental impact of our backbone with a multi-factorial approach covering GHG emissions, water consumption and abiotic ressources usage. On the carbon emissions side, the findings are that the previous methodology we used (based on a Renater research paper) in our GHG emissions reporting was overestimating OVHcloud backbone impact by a factor 2.

Our new data center in India offers three salient features:

Homogenization

Partners and customers in India can now subscribe to the same cloud services and infrastructure as those in regions outside of APAC. With local computing and storage capabilities, our new data center provides Indian businesses with improved support to meet changing data compliance needs. This can help simplify the management of global IT infrastructure, reduce costs, and streamline operations for multinational companies.

Industrial Integration¹

Being vertically integrated with full control of the value chain including server production, helps the Group to bring energy, water and carbon-efficient servers to the market faster, contributing to a more sustainable future for the industry. With this, we are empowered to provide more secure and sustainable solutions for our partners and customers.

Improved Network

Our new data center offers connectivity with low latency and bandwidth options starting from 1 Gbps with 25TB data traffic per server, which also peers with all major Indian internet service providers (ISPs), ensuring fast and reliable access to cloud services for businesses in the region.

In addition, we have a well-defined product roadmap that is carefully crafted, outlining the products and services tailored to meet the evolving needs of our partners and customers. With the upcoming product range to be deployed in the new India data center, we are committed to enabling our partners and customers to stay ahead in the fast-moving world of cloud computing.

Businesspeople often lose sleep over network outages. Downtime caused by unplanned outages results in the bottom line taking a hit and can cost huge sums of money. According to Gartner, the total average cost per minute of unplanned downtime runs at about $5,600.

Network resilience is becoming increasingly important for mission-critical operations as well as the increasing use of cloud services by all types of business. Data availability is becoming of key importance for everyone: from those running corporate IT departments in the cloud, to websites providing government services — even down to healthcare organisations performing surgery using extended reality (XR). 

Because of the growing dependence on network performance, we have instituted a five-year plan to move towards the delivery of a hyper-resilient network and reduce the risk of network threats that can cause business operations to screech to a halt. Because these threats come from all angles, including from human error, weather conditions and fibre cuts, as well as susceptibility to bugs and cyberattacks — we believe hyper-resilience is the path all network providers need to follow.

We are now three years into our plan and have been working hard to completely re-engineer every part of our network, from the optical and IP layers to the routes into our global data centres. The thinking behind this is to increase the trust in the performance of our network by increasing both reach and speed. 

Using continuous improvement to build hyper-resilience 

Our route to hyper-resilience is based on a five-pillar approach to improve the performance of all aspects of the network, including event management, incident management, change management and problem management. Broken down, these pillars support the overlay, changing the way customers consume network services and the underlay, which allows us to automatically configure each of the components that our customers use.

This is all underpinned by a philosophy of continuous improvement so we can make accurate assessments on the number of risks introduced and find effective ways to manage them.

Reducing risk means not introducing problems — something that often happens with human intervention. To facilitate centralised management of the network and to make it more flexible, we have introduced a software-defined networking approach. This allows for enhanced control and uses automation across both physical and virtual network environments.

To back this up, there are multiple routes taking fibre connections into our data centres to further reduce the risk of downtime. When fibre optic cables were intentionally sabotaged earlier this year, several cities in France experienced internet slowdowns and outages. With our network having multiple paths for data traffic, we were able to re-route traffic, so our customers didn’t notice.

A proactive and predictive approach to managing risk

Our risk management methodology is both proactive and predictive and is designed to grow and evolve — nothing ever remains static in the evolution of networks. For problem management, we can now solve any issues early on, using a data driven approach. This data shows us any recurring patterns that indicate possible points of failure. Rather than let the problem grow, it allows us to solve any issue immediately. We have also increased the amount of network filtering to reduce the risk of any kind of malicious activity. These preventative measures mean there is less risk of a bigger incident later on down the line — and we can ensure that there is as little impact on the customer as possible.

One of our principal goals is to reach 100 per cent automation to overcome not just human error, but also ensure that any failures can be detected and fixed as they occur. With the software we build to manage the network, we’ve introduced automation into code production. This greatly reduces errors and together with robust CI/CD processes, we can iron out any bugs and errors before any code goes into production.

Zero-touch provisioning is another way of reducing and eliminating risk. Our next-generation network can detect new devices and determine what their purpose in the network is, and configure, install and deploy the devices automatically. This level of automation also removes the headache of updating device firmware. Before, this was a time-consuming and disruptive process. Now we can update devices more easily without any network interruption. We’re also supporting this further with software that’s designed to be self-healing to detect, react to and correct issues on the fly.

Leveraging new services for customers 

By building a hyper-resilient network, we’re now able to focus on delivering a better service to customers and help them with their specific business needs. It also means we can offer more value-added services that customers can make use of in the cloud, such as private networks, and support any business looking to leverage infrastructure as code (IaS). Not only can we deliver services much faster and more easily, we can ensure they have resilience built in by default. 

While we recognise that threats can never be fully eliminated, we aim to make our own network as resilient as possible and see this journey as an ever-continuing one. Ultimately, it’s all about ensuring that customers can rely on their networks, generate value and increase their agility. With networks moving beyond being a commodity to being a valuable business enabler, it’s vital that hyper-resilient networks become the mantra for all.

Les pannes de réseau sont une source constante d’inquiétude pour les entreprises. Les temps d’arrêt causés par des pannes non planifiées se traduisent par des pertes financières et peuvent ainsi coûter très cher. Selon Gartner, leur coût total moyen par minute s’élève à environ 5 600 dollars.

Pour tous les types d’entreprise, la résilience du réseau devient de plus en plus importante pour les opérations critiques, ainsi que pour l’utilisation croissante des services Cloud. La disponibilité des données est essentielle pour tout le monde, des services informatiques d’entreprise dans le Cloud aux sites Web fournissant des services publics, en passant par les organismes de santé qui pratiquent des opérations chirurgicales en réalité étendue (XR).

En raison de cette dépendance croissante aux performances du réseau, nous avons élaboré un plan quinquennal visant à mettre en place un réseau hyper-résilient et à réduire le risque de menaces susceptibles de paralyser les opérations commerciales. Étant donné que ces dernières proviennent de tous les horizons, notamment de l’erreur humaine, des conditions météorologiques et des coupures de fibre, ainsi que de la vulnérabilité aux bogues et aux cyberattaques, nous pensons que l’hyper-résilience est la voie que tous les fournisseurs de réseaux doivent suivre.

La mise en œuvre de notre plan a commencé il y a trois ans. Nous avons travaillé d’arrache-pied pour réorganiser complètement chaque partie de notre réseau, des couches optiques et IP aux voies menant à nos centres de données mondiaux. L’objectif est de renforcer la confiance dans les performances de notre réseau en augmentant sa portée et sa vitesse.

L’amélioration continue au service de l’hyper-résilience

Notre cheminement vers l’hyper-résilience repose sur une approche basée sur cinq piliers visant à améliorer les performances de tous les aspects du réseau, notamment la gestion des événements, la gestion des incidents, la gestion des changements et la gestion des problèmes. Ces piliers soutiennent la couche supérieure, qui modifie la façon dont les clients utilisent les services réseau, et la couche inférieure, qui nous permet de configurer automatiquement chacun des composants utilisés par nos clients.

Tout cela est sous-tendu par une philosophie d’amélioration continue qui nous permet d’évaluer avec précision le nombre de risques introduits et de trouver des moyens efficaces de les gérer.

Réduire les risques signifie ne pas introduire de problèmes, ce qui arrive souvent lors d’interventions humaines. Pour faciliter la gestion centralisée du réseau et le rendre plus flexible, nous avons introduit une approche réseau définie par logiciel. Cela permet d’améliorer le contrôle et d’utiliser l’automatisation dans les environnements de réseaux physique comme virtuel.

Pour consolider cette approche, de multiples voies de connexion en fibre optique sont prévues dans nos centres de données afin de réduire encore davantage le risque d’interruption de service. Lorsque des câbles ont été intentionnellement sabotés au début de l’année, plusieurs villes françaises ont connu des ralentissements et des coupures d’internet. Comme notre réseau dispose de plusieurs routes pour le trafic de données, nous avons pu le réacheminer, et nos clients ne s’en sont ainsi pas rendu compte.

Une approche proactive et prédictive de la gestion des risques

Notre méthodologie de gestion des risques est à la fois proactive et prédictive. Elle est conçue pour croître et évoluer : rien ne reste jamais statique dans l’évolution des réseaux. En ce qui concerne la gestion des problèmes, nous pouvons désormais résoudre tout problème de manière anticipée, en utilisant une approche axée sur les données. Ces dernières nous montrent les schémas récurrents qui indiquent les points de défaillance possibles. Cela nous permet de résoudre le problème immédiatement plutôt que de le laisser s’aggraver. Nous avons également augmenté le filtrage sur le réseau afin de réduire le risque d’activités malveillantes. Ces mesures préventives réduisent le risque d’un incident plus important plus tard, et nous pouvons faire en sorte que l’impact sur le client soit le plus faible possible.

L’un de nos principaux objectifs est d’atteindre un niveau d’automatisation de 100 % afin d’éliminer non seulement l’erreur humaine, mais aussi de garantir que toute défaillance puisse être détectée et corrigée dès qu’elle se produit. Avec le logiciel que nous avons créé pour gérer le réseau, nous avons introduit l’automatisation dans la production du code. Cela réduit considérablement les erreurs et, avec des processus CI/CD robustes, nous pouvons éliminer les bogues et les erreurs avant que la mise en production.

Le provisionnement sans intervention est un autre moyen de réduire et d’éliminer les risques. Notre réseau de nouvelle génération peut détecter les nouveaux appareils et déterminer leur rôle dans le réseau, puis les configurer, les installer et les déployer automatiquement. Ce niveau d’automatisation élimine également le casse-tête de la mise à jour des micrologiciels des appareils. Auparavant, il s’agissait d’un processus long et perturbateur. Désormais, nous pouvons les mettre à jour plus facilement, sans aucune interruption du réseau. Nous allons plus loin encore dans ce domaine avec un logiciel conçu pour s’auto-réparer afin de détecter, de réagir et de corriger les problèmes à la volée.

Tirer parti de nouveaux services pour les clients

Grâce à la mise en place d’un réseau hyper-résilient, nous pouvons désormais nous concentrer sur la fourniture d’un meilleur service aux clients et les aider à répondre à leurs besoins spécifiques. Cela signifie également que nous pouvons offrir davantage de services à valeur ajoutée que les clients peuvent utiliser dans le Cloud, comme les réseaux privés, et soutenir toute entreprise cherchant à exploiter l’infrastructure en tant que code (IaS). Nous pouvons ainsi fournir des services beaucoup plus rapidement et plus facilement, et nous pouvons également nous assurer qu’ils sont résilients par défaut.

Même si les menaces ne pourront jamais être totalement éliminées, nous nous efforçons de rendre notre propre réseau aussi résilient que possible et nous considérons que cette démarche doit être poursuivie en continu. En fin de compte, il s’agit de faire en sorte que les clients puissent compter sur leurs réseaux, générer de la valeur et accroître leur agilité. Puisqu’ils ne sont plus un simple produit de base mais un outil commercial précieux, il est essentiel que les réseaux hyper-résilients deviennent la norme pour tous.

Here at OVHcloud, we pioneered free cooling, and have used water cooling at scale to cool down our servers since 2003. This unique approach helped us to set high standards in PUE/WUE indexes for our global datacentres, all the while lowering our overall carbon impact.

With the evolution of IT equipment power, OVHcloud R&D worked on a new generation of datacentre cooling solutions. One of the most promising technologies today is immersion cooling, thanks to direct contact of the fluid with hardware components. It allows us to build datacentres all over the world maintaining low PUE/WUE indexes even in harsh climatic zones.

In this blog post, we will explain how we developed our own breed of immersion cooling to achieve several goals, from improving datacentre footprint and reducing power consumption, to adapting to new operating conditions and improved overall reliability.

OVHcloud Hybrid Immersion Liquid Cooling

Immersion Cooling is the practice of submerging electronic devices in a thermally and not electrically conductive liquid.

The typical immersion cooling solution usually employs pumps, heat sink structures, heat exchangers, condensers, sealed evaporative equipment etc. that either consume large amounts of energy to operate, require sealed casings, or occupy relatively large surface areas that limit the number of servers that can be implemented.

OVHcloud developed its own Hybrid Immersion Liquid Cooling technique. It is comprised of a direct to chip water cooling system and a passive natural single phase immersion cooling system containing two fluids:

• Water: cooling a heat sink through water blocks disposed on CPUs and GPUs with the same solution used in all OVHcloud servers and a proprietary serpentine convection coil connected to a pumping substation (PSS) and a dry cooler to evacuate heat outside the DC.
  
• Immersion Cooling (IC): the fluid is contained in a tank and cools all IT equipment in the server rather than just CPUs and GPUs; that fluid basically replaces the air circulating in OVHcloud servers thus enhancing the efficiency of any component not cooled by OVHcloud water cooling systems.

It comes with a new 3 floor server rack design, in a library format, which we can populate with up to 48 servers (1U) or 24 servers (2U), in a book format. Each server is submerged in its own tank for independent cooling, allowing large scale deployments. In addition, each server benefits from a specific monitoring system of all environmental factors ensuring safe and secure operation at the server level.

Benefits in terms of power consumption and efficiency

OVHcloud Hybrid Immersion Liquid Cooling has several advantages that makes it an efficient solution:

• The new passive rack design means there are neither pumps nor fans, resulting in zero cooling electrical consumption at the rack level
• High power racks can operate with datacentre inlet temperatures of up to 45 °C, allowing for different cooling loads at different climatic conditions
• Stay energy efficient using free cooling available on site
• The electrical consumption and CAPEX are further reduced through the elimination of the Evaporative Cooling system (no pumps) used on dry coolers for data centers located in areas where air ambient climatic temperatures are below 43 °C
• The difference between supplied and recovered temperature of DC water (DT) of 20K can be achieved through highly efficient CPU and GPU cold plates and a large thermal contact surface between the serpentine and the dielectric fluid. The patent pending serpentine design occupies a very small area within the server chassis maximizing space for components
• Global datacentre cooling infrastructure power consumption is reduced by at least 20.7% compared to the OVHcloud water cooling system
• The energy consumption of a server per year is reduced by at least 20% compared to air cooled servers and 7% versus water cooled servers
• Reduced power consumption means lower operating expenses (OPEX)

Finally, in the context of ever-increasing CPU/GPU TDP, the solution supports higher computing power density.

A new generation of datacentre

Our new Hybrid Immersion Liquid Cooling approach stands outs with several key advantages for datacentre operators:

• Actual footprint of up to 37U/m² can be guaranteed with no extra pumping and condensers systems. The design is scalable to two or three times its footprint if immersion cooling racks are installed in stacked maritime containers
• With no sealing required nor sophisticated heat exchangers and pumping circuits the capital expenditure (CAPEX) remains low
• With a datacentre outlet temperature of 65 °C, fatal heat is better captured, and affordable heat recovery systems can be envisioned
• More flexibility and freedom in design at OVHcloud, where the new proposed rack design requires access to the front side, unlike classic racks requiring access from both the front and rear sides.

Redefining datacentre indexes

With such a new game-changing technique, the usual datacentre performance indicators evolve dramatically.

Enhance PPUE^[1] to 1.004: PPUE or partial power usage effectiveness defines a certain portion of the overall PUE of a datacentre within a clearly defined boundary. Infrastructure cooling PPUE is as follows:

Reduce WUE to 0 for datacentre located in areas where ambient air climatic temperatures are below 43 °C: WUE or water usage effectiveness is a sustainability metric to measure the amount of water used by datacentres to cool IT equipment. Annual site water usage includes water used for humidification and water evaporated on-site for energy production or cooling of the datacentre and its support systems. It is defined as follow:

Environmental considerations

The Hybrid Immersion Liquid Cooling technique relies on a non-volatile dielectric hydrocarbon fluid. OVHcloud R&D tested and qualified the system with a variety of market fluids. While each fluid has different properties, we paid special attention to their compliance with the following considerations:

• Non-corrosive
• Ultra-low vaporization
• Nontoxic
• Non allergenic
• Biodegradability in 30 days
• Very high flash point
• Dielectric Strength up to 42 kV
• GWP^[2]=0
• ODP^[3]=0

Operational changes within the datacentre

The new 3 floor library format rack design is footprint optimized for a high density of servers and incidentally compute power per square meter (U/m²). It could be fitted with manual/automated guidance to extract servers, reducing the maintenance impact of one server on the overall rack.

The new Hybrid Immersion Liquid Cooling technique also improves reliability reducing failure rate by up to 60 % by eliminating risk of dust circulation. It contributes to a silent datacentre with the removal of any fan at rack and server levels. Finally, several components are no longer needed:

• No heat exchangers attached to the racks (3 heat exchangers are eliminated per rack),
• 36 fans usually installed per rack are eliminated
• All small fans inside the servers are eliminated
• Cooling modules comprised of pumps and plate heat exchangers are eliminated

What’s next?

The new OVHcloud Hybrid Immersion Liquid Cooling paves the way for unapologetically more compute power in modern datacentres. Its unique patent pending design is sustainable and preserves the opportunity to valorize fatal heat due to higher operating temperatures. Moreover, lower power and water consumption, as well as improved footprint can be achieved with the benefit of adapting to regions with harsh climatic conditions. We are incredibly excited by the very first use cases of Hybrid Immersion Liquid Cooling in the fields of banking, medical and scientific research, not forgetting gaming workloads. We can’t wait to share more with you in the future.

[1]: Partial PUE is used instead of PUE. It is expected that the global PUE will improve in a similar approach which includes the power lost in the energy distribution system [More details about the PPUE and PUE here]. In line with its transparency commitments, OVHcloud will communicate PUE values upon gathering significant data on the long run on a specific location basis.
[2]: GWP stands for Global Warming Potential. The fluid does not contribute to global warming in opposition to most of gas/liquid fluids used in cooling technologies.
[3]: ODP stands for Ozone Depletion Potential. The fluid does not contribute to ozone depletion in opposition to most of gas/liquid fluids used in cooling technologies.

Unbridled growth in data storage and processing

One of the main objectives of the strategic plan that we completed over the past five years is to meet a strong proximity requirement expressed by our customers. Customers want us to guarantee maximum access to their infrastructure in their local markets, and to commit to complying with their legislative constraints, particularly those regarding data sovereignty. Today, we have 31 data centers across the globe. And we have two isolated entities, which means that we are one of the few cloud providers that can offer infrastructures that are not subject to the Cloud Act for our European, Canadian and Asian customers; and others that are subject to the Cloud Act for our American customers.

Our growth has followed the digitisation of the economy, as well as the digitisation of professions, which has only accentuated the demand for storage and data processing. Big data, and more recently edge computing and artificial intelligence, have increased the need to store and manipulate massive volumes of data in real time and in unprecedented ways. Since the beginning of the Covid-19 pandemic, the widespread adoption of new digital tools and the use of cloud solutions for leisure, business continuity and critical service management have greatly amplified this trend. Our data centers are more strategic now than ever before, as they maintain national activities in most of the countries where our infrastructures are deployed.

Responsible industry

A study by the International Energy Agency (IEA), published this year, has shown that despite the exponential growth in data volume, data center power consumption worldwide has not exploded. In fact, power consumption is stagnating. Contrary to conventional wisdom, this was no surprise to industry experts, and is the result of their joint efforts in terms of energy efficiency and environmental policy. The IEA has shown that growing demand does not necessarily lead to a deterioration of the environment:

This global stabilisation of the energy consumed by data centers is mainly explained by:

• More energy efficient servers and data centers
• Massive cloud adoption

The graph above provides a good summary of the various cumulative effects of next-generation servers and data centers, whether it be the energy required for a unit of compute, a TB of storage, or the capacity

required for workload. This doesn’t take into account the rapid improvement in data center PUE, from an average of more than 2 before 2010, to PUEs around 1.2-1.3 today for the most recent generations of data centers.

That being said, a net growth in compute/storage requirements should also theoretically lead to gross growth in consumption, which is more controlled, but gross growth nonetheless… Unless we manage to reduce consumption of already existing needs by accelerating migration to the cloud.

Massive cloud adoption should be accompanied by the migration of all small private installations to large, industrially managed data center. Companies that have small on-premises computing rooms are increasingly using our services to operate in a virtualised and shared way, through our huge global infrastructures. In the vast majority of cases, the non-critical size of these On-Premise computing rooms does not justify a massive investment to implement an energy efficiency policy like the one we deploy on a very large scale. As a result, all of these small in situ computing rooms added together are actually much more energy-intensive than their cloud counterparts. Today, most of these rooms, previously distributed geographically, are condensed in our data centers – which are already part of an eco-responsible approach – or in those of our competitors.

The graph below, detailing consumption by data center type, illustrates this dynamic well:

The consumption of cloud data centers and hyper-scale data centers is growing compared to the decline of traditional On-Premise, simply because they have replaced the consumption of these traditional data centers. There are several reasons for migrating to the cloud: for its features, flexibility, simplicity and speed of use, its price… but its environmental responsibility is a major one. This commitment is rooted in our DNA, and motivates the deployment of a responsible industrial model across all of our infrastructures.

Innovation as a key to success

The issue of our environmental footprint has never been more at the heart of our concerns, both among manufacturers and our customers. Global warming has brought about massive challenges to the way we cool our data centers , wherever they are in the world. All of our infrastructures must be able to withstand both increasingly high outdoor temperatures and the huge amount of heat they generate, without becoming more energy-intensive. To meet this challenge, we are accelerating industrial innovations, particularly those around our cooling system (one of the most energy-intensive stations). We aim to keep our installations at a moderate temperature, and to take our PUE (Power Usage Effectiveness) index as low as possible, which we know we can optimise up to 1.09.

Thanks to our liquid cooling technology, developed and industrialised since 2003, we are one of the pioneering companies that has paved the way for more efficient data centers. Contrary to popular belief, water cooling does not waste water since it operates within a closed circuit. Historically, this technology through its vertical industrial integration has allowed us to achieve economies of scale and to establish our positioning as Price Leader by Design. But our growth has never been achieved at the expense of the fair use of our resources, to the point where this has now become a strategic issue that meets our commitments in terms of environmental responsibility.

Our environmental policy also extends to the server level. Designing our own servers while managing the entire production chain is a guarantee for us that we can reuse components wisely and renew them

regularly. We determine their lifecycle, which corresponds to 2nd and 3rd life servers, then to recycling. This virtuous circle also allows us to gradually replace the old parks of each of the server ranges with less energy-intensive versions. We use these levers to improve our overall energy performance.

A mutually enriching ecosystem

At our electronic component suppliers, the evolution of each generation of components depends mainly on the finesse of its silicon engraving, which contributes to significantly reduce its size. This reduction in component size results in a reduction in the energy wastage surface resulting in reduced power consumption and heat release at the server level. It also means a decrease in the size of the server itself. At the industrial scale of our infrastructures, each new generation of components produced by our technological partners is often a huge development for us. By combining them with our own industrial innovations, we can store more servers with an equal surface, compared to the previous range of these same servers. This concentration also allows us to offer our customers higher computing power with equal consumption.

Our commitment to eco-responsible EcoCare is a quest that we pursue in collaboration with our ecosystem of technology partners. It is at the heart of our relationship with them, and is based on our operational and industrial excellence, combined with our great capacity for disruption.

A commitment to transparency

Twenty one years ago, OVHcloud was founded on the promise of making the data revolution a success for everyone, and we believe that the cloud can be part of the solution to bring about new, environmentally-friendly models.

To achieve this goal, we will continue our efforts to minimise our carbon footprint. We will continue to promote sustainable development through more efficient, less energy-intensive industrial designs. Thanks also to new, increasingly accurate energy performance indicators that will monitor all of our infrastructures and help drive the emergence of a more virtuous business model in the cloud industry.

How can we go even further?

Our obsession with energy efficiency and the frugality of our design brings value to us and to our customers, not only economically but also environmentally. But as we can see from the data center consumption curve, migration to the cloud cannot indefinitely be the factor controlling data center consumption. And gains in energy efficiency will become increasingly difficult to achieve. We therefore believe that we should not only optimise our infrastructures, but also look at the higher layers, the applications, analyze how the code is written and determine its impact in terms of energy consumption. Energy efficiency must also be developed at the software level. This is a great opportunity to inform our customers about their own impact, so they can make the right decisions about their own usage.

To find out more, please log in to the OVHcloud #EcosystemExperience keynote on 03 November 2020 by signing up here.

Or follow the dedicated sessions (by registration):

• The OVHcloud Carbon footprint assessment: rates, reduce and offset, with CLO2
  
• Data centers, Cloud and IT energy consumption: how to develop a sustainable cloud together, with IEA
  
• Energy matters in the Cloud, with INRIA
  

The OVHcloud network has 34 PoPs (Points of Presence); located in Europe, North America and Asia-Pacific. With a global capacity of around 21Tbps, the OVHcloud network can handle much more traffic than other providers.

At OVHcloud, network traffic is constantly growing to meet the needs of millions of internet users, who utilise our 31 data centers across the globe.

Over the past few years, OVHcloud have been working to improve the infrastructure of our worldwide backbone network.

Scaling up our network further

With almost two years of research and development, the OVHcloud network team has come up with a brand new infrastructure. Each datacenter is connected to multiple PoPs (Point of Presence). PoPs share traffic with various other OVHcloud datacenters as well as exchanging traffic with different providers (known as Peers).

The current architecture, known internally as ‘sticky router’, is based on a router for routing and a switch which plays the role of the power board. For a few years, it had worked quite well (and at a low cost), but the method eventually reached its limit in terms of bandwidth. We needed to find and design another system which could handle the increasing traffic. Our requirements were simple; we needed a low cost, low-power efficient, and scalable infrastructure.

Providing the best possible connectivity for our worldwide customers has been the company’s driving force since it was created by Octave Klaba. To do this, we wanted to be connected to local providers as far as possible, requiring multiple ports in 10Gbps or 100Gbps.

Re-thinking our PoP topologies, we considered new scalable technologies, including:

• Port capacity: would have to be based on 100Gbps and 400Gbps ports, and have a good cost efficiency. This would help eliminate any bottlenecks in the network. It also needed to maintain 10Gbps links for those providers who were not ready for 100Gbps ports.
• Easy to upgrade: The new architecture needed to be easy to upgrade, in terms of capacity. This is partly for the growth of the company, but also to maintain availability when network maintenance is required on the PoPs.
• Power: The team needed to find the best hardware for maximising power-consumption efficiency; especially in countries like Singapore where it’s expensive.
• Security: A key requirements would be to work with our security teams to find the best solution for protecting the network against threats (massive DDoS attacks).

After almost 1 year of research and tests, the design team came up with a brand new architecture; which was scalable, power efficient, easy to install and robust. The new architecture was named SMAUG, after the dragon in The Hobbit.

Overview of SMAUG

To be adaptable, the architecture has multiple capacity options, depending on how big the PoP is. This is because different amounts of traffic are exchanged at different datacenters. Each capacity option has its own specificity, with the objective of no bottlenecks.

Spine and Leaf infrastructure

SMAUG is a ‘Spine and Leaf’ infrastructure. This means the ‘spine’ (called SBB for SuperBackBone) aggregates the leaves and connects each datacenter. The ‘leaf’ devices (called PB for PeeringBox) are used for connecting providers and also internal services such as xDSL equipment or OVHcloud Connect.

SMAUG infrastructure also provides another way to connect a datacenter where there is a minimum of two PoPs, both in a different location. For example, in Singapore, our datacenter is connected to two PoPs, which have more than 30 kms between them – this meets the rule of not using the same power source for two PoPs.

To ensure redundancy, both the PoPs need to be connected to each other with a huge amount of capacity – in 100Gbps or 400Gbps, depending on the PoPs. The transmission team was also involved in developing a new infrastructure called ‘GALAXY’. GALAXY was based on a different constructor – combined with a simple-to-deploy, scalable, operational model, with less energy consumption.

The role of the leaf is pretty simple and is similar to the top of the rack in a datacenter. It has a huge amount of uplink capacity towards the spines and has the configuration for connecting BGP peers; such as transit providers, private interconnection (PNI) and Internet eXchange.

The spine role is more complex and has added functionality, including:

• Long-haul links: It has the connection of the links, based on 100Gbps, towards datacenter(s) and PoP(s).
• Routing: It has the entire routing table in order to take the best path towards the OVHcloud datacenters or towards the external networks.
• Protection: The VAC team has been involved in it by developing a new protecting tool (for more information: https://www.ovh.com/blog/using-fpgas-in-an-agile-development-workflow/) in order to help to protect the entire OVHcloud network from DDoS attacks.
• Mitigation: It also helps the detecting system used by the VAC infrastructure (https://www.ovh.co.uk/anti-ddos/)

Testing and deploying

After the design team and management were sure it was the best architecture for the OVHcloud network, we started testing different brands, providing all the functionality was implemented correctly. After all the tests had been completed in the laboratory, and we were confident that the solution was viable, we started the deployment of the infrastructure in Singapore. This region was one of the most important in terms of traffic growth. It was also easier because we already had the dark fibers for the links between the datacenter and the PoPs.

In January, we ordered all the devices and transceivers, then prepared the migration plan in order to schedule the whole deployment at the end of March. At the end of February, we prepared the configuration and tested the brand new devices. When everything was in order, we sent all of them to Singapore PoPs.

At the beginning we planned to do this migration mid-march 2020 and we were supposed to send our technicians from France to Singapore, but due to COVID-19 we had to change our plans. We had to find another solution and ask our local technicians working in our Singaporean datacenter to do the job. The migration plan was more complicated due to the new reorganization that needed to be in place for the pandemic.

After a long discussion between the management, the network team and the Singaporean OVHcloud technicians, it was decided to do the migration of the first PoP at the beginning of April and the second one at the end of April. The migration began by racking the brand new devices in two new racks, preparing the wiring for the migration, and doing some checks before the hotswaps.

Migrating to SMAUG

The pressure for this migration to be successful was high, as we did not want it to impact our customers. On the first night of the migration – once we had dried out the traffic from the first PoP – we asked the technicians to move all the long-haul links to the Singapore DC, and for Australia, France and USA, to new devices. After some tests, we put the new devices in production. The first step of the migration went well.

The next day wasn’t quite so smooth, as it was the first time we were putting our new Peering Box in production with our new border protection system, based on FPGA servers. After we removed the traffic from our peers, traffic was then leaving the OVHcloud network via the second PoP. Then we moved the fibers to the new Peering Box via a hot swap from our datacenter supplier. After we got everything plugged into the new equipment, we then began putting production back in slowly. We needed to do this in conjunction with our security team, in order to check that this new border protection system was working properly and not dropping legitimate traffic.

On the last day of migration, another technology had been put in place by our transmission team. The goal here was to add capacity between the Singapore datacenter and the PoP where we installed these new devices. After we isolated the traffic between both ends, we migrated the dark fiber to the new DWDM optical system (Galaxy) in order to add 400Gbps of capacity to the datacenter. As it was new, we had some trouble explaining how to fix some cabling issues in the system. After it was all fixed and ready, one by one, we put 4x100Gbps links in production.

After completing all these different steps we analysed and fixed some issues in order to make the second PoP faster with the same schedule.

Once both PoPs were in production we monitored how it was handling the traffic and the DDoS attacks. We also contacted our close customers to ensure that there were no issues.

In conclusion

This new infrastructure, SMAUG, brings significant improvements in power and space efficiencies as well as reducing complexity in the overall network design. It will also help OVHcloud to keep up with increasing applications and service demands.

Looking ahead, we believe the flexibility of SMAUG design will help OVHcloud to leverage faster network capacity and be ready for future technologies.

Thanks to the teams and industry partners that helped make this new infrastructure possible.

A promise is a promise, so several weeks ago we fulfilled it: your servers now have twice the bandwidth, for the same price!

We knew from the start that this upgrade would be feasible, as our 20Tbps network core can definitely cope with the extra load!  We work daily to make sure you enjoy using our network, which is one of the largest in the world among hosting providers.

Indeed, our network is constantly evolving, and our teams work tirelessly to optimise the capacity planning and anticipate the load generated by all our customers, spread across our 28 datacentres.

It’s also more than capable of managing the waves of DDoS attacks that arrive almost daily, sending millions of requests to hosted servers in an attempt to render them unavailable. These are absorbed by our in-house Anti-DDoS Protection, without any customer impact! As a reminder, we suffered one of the biggest attacks on record a few years ago, which generated traffic of more than 1Tbps, but was nonetheless absorbed by our infrastructure, without any impact on our customers.

To guarantee this additional public bandwidth, our Network and Bare Metal teams have worked closely together to be more and more LEAN when it comes to our infrastructures. As a result, thousands of active devices (routers, switches, servers etc.) have been updated in a completely transparent way!

The overall deployment process has taken some time, as we have done a rolling upgrade, taking a QoS and isolation approach to prevent possible traffic spikes. Product range by product range, datacentre by datacentre… The deployment itself was quick and painless, as it was fully automated. The potential bottleneck was making sure that everything worked as intended, which involved carefully monitoring our full server farm, as bandwidth doubling can have a huge impact, especially at OVH, where (let me mention it once again!) egress traffic is indeed unlimited!

Here’s a quick overview of the new bandwidth for each server range:

Even if the bandwidth doubling doesn’t yet cover the full extent of our ranges, or the So you Start and Kimsufi servers, we haven’t forgotten our customers who’re using those servers. We have also updated our bandwidth options to offer all our customers an even better service, at an even better price.

We aren’t going to stop there though! We will soon announce some nice new features on the network side of things.  And of course, lots of other innovations will arrive in the coming months. But those are other stories, which will be told in other blog posts… 😉

Actuellement, les 2 datacenters gravelinois GRA 1 et GRA 2 peuvent accueillir plus de 85 000 serveurs.

D’une surface totale de plus de 9 hectares dont 20 000 m² de bâtiments, le site gravelinois est en expansion. Pour répondre à la croissance et à la sécurisation, le site s’est doté cette année d’une ligne haute tension complémentaire. La puissance électrique s’élève désormais à 40 MW (voir la photo ci-dessous).

De nouvelles salles serveurs ont permis de répondre à la demande en très forte croissance sur ce site. Au cœur de l’Europe,  GRA 1 et 2 sont reliés directement à Paris, Londres et Bruxelles ce qui permet des latences très performantes avec ces villes dans lesquelles les usages numériques connaissent un développement exponentiel.

Pour assurer la haute disponibilité de l’ensemble du site, chaque alimentation électrique, en plus d’être redondée, est secourue par des onduleurs. Un groupe électrogène est systématiquement installé en parallèle, comme on peut le voir ci-dessous.

Les équipes R&D d’OVH continuent également d’innover de leur côté, notamment en vue d’améliorer la performance énergétique des datacentres. Dans ce contexte, Gravelines devrait prochainement disposer de nouvelles avancées technologiques sur lesquelles nous reviendrons. A suivre…