What’s new?

Over the past months, we continued to gather feedback from our customers and improve our offers. Now powering thousands of databases clusters, we focused on the major requests discussed directly with our customers, or from official Github roadmap and Discord community. From better flexibility to higher resiliency, let’s have a look about new features.

Additional storage space for flexibility

The first major improvement is about storage and flexibility. Before, each databases service was launched with a fixed amount of storage, depending of the nodes sizing. As a example, nodes with 16GB of RAM came with 400GB of storage. A recurring request of more flexibility, at creation but also once it’s running is now live with additional storage option.

During cluster creation, you can now opt for more usable storage space for your service. And once your cluster is up and running, you are free to increase this value without any downtime. So far you cannot decrease the storage space, but it’s prioritized for the incoming months.

You can directly verify minimum and maximum storage space from our website or control panel.

Secondary backup for the same price

We now provide a secondary backup location per service for the same price, for all Databases services except MongoDB (only one remote backup for now). One in the same physical location, and another one in a remote location (offsite backup).

Another recurring request is to provide a better fault-tolerance, for any situation, such as a rack failure, a datacenter failure or anything else. Since the beginning, we provide daily backups in the same geographical location. Being in the same location is very useful in case of localized issues. You avoid network latency, and improve restoration time. But remote backups were awaited. it’s now provided for free for any cluster, running or soon-to-be created. You can find your backup location directly in our API, and soon via OVHcloud Control Panel. More information in our official documentation!

Exact time machine with PITR

Point-in-time-recovery (PITR) is now available for all MySQL, PostgreSQL and MongoDB Enterprise services, directly via OVHcloud Control Panel or API. What does is bring?

One of the nicest feature from MySQL, PostgreSQL and MongoDB is Point-in-time-recovery, known as PITR. Imagine being able to travel back in your database operations, and restore your data at a really specific timestamp, such as “two days ago, 15h02 and 24 seconds” ?

PITR proved its efficiency over the years and saved the life of many databases admins. Imagine yourself deleting the wrong table at 15h02, and being able to restore the minute before instead of the backup from last day? This is not black magic, but only a combination of daily backups and logs files (such as WAL logs for PostgreSQL).

… And much more !

In case you missed it, we also released tons of improvements in the last six months, here we go !

• Latest DBMS version with PostgreSQL 15 and MongoDB 6.
• Automatic email alerts when your cluster is reaching storage limits (also, you can retrieve the disk usage with our API)
• Ability to terminate PostgreSQL/MySQL queries directly from control panel or API. Quite useful when a specific query is impacting your cluster.
• Much more control over MongoDB roles from control panel and API. More information.
• More storage by default for MongoDB offers. Don’t hesitate to check our website!

What’s next?

Our roadmap for the next months is public, and still connected to customer requests.

The first item is about storage performance. While our clusters fit globally our customer workloads, advanced ones can feel performance limitation on the storage part. We are currently implementing better storage, reaching 20’000 IOPS instead of 3’000. It’s already implemented for the same price in MongoDB cluster, and soon for everything else (roadmap card).

Second major item will be a better integration with Kubernetes clusters, located in OVHcloud or any other provider. Today, you can link them together, but it will be improved with a Kubernetes operator (roadmap card), simplifying the authentication between them.

That’s all for today ! If you have specific needs of just want to give it a try, don’t forget that all the services are billed per hourly, meaning that you can test your use case for a couple of euros. and please join our Discord community to share your feedback after !

Let’s be honest – Google Colab is solving many challenges. It enables millions of people around the world to learn and use Python and play live with hundreds of libraries for free. It’s quite often for data-oriented use-cases – but not only. To be fair, I discovered the power of notebooks through Google Colab a long time ago.

Notebook? You said notebook?

A notebook is a document which contains code (e.g. Python code), text and images that can be read by us, humans, but also executed inside the notebook. With Jupyter app, it can be launched inside your web browser, allowing you to explore and experiment easily and share your work with many others.

Think of notebooks as cooking recipes that you can follow live, step-by-step and see the result directly. It’s truly wonderful.

Now imagine notebooks that can be linked to remote power and storage to perform your use cases at scale. That’s it 😀

Solutions like Google Colab are nice, but limited

There are plenty of managed notebook solutions on the market today. The market is split in two – complicated solutions inside cloud providers (AWS Sagemaker, Google AI Platform Notebooks, Azure ML Notebooks…) and pure players trying to bring notebooks to the masses. The main actor in this field is Google Colab.

Historically, Google Colab is based on Jupyterlab, and forked from this awesome open source project few years ago (source: their FAQ).

Since mid-2021, they now provide 3 plans in 9 countries, from free to paid (Pro and Pro +).

After browsing their website and FAQ, I drafted this comparative table:

If you are a student or an individual, Google Colab is ideal for you, as most pricing plans contain the basic features you need for exploring this magic world of data. That’s cool.

But once you are working on more intensive projects, you may reach Colab limitations:

• Compute resources are not guaranteed: for example, you don’t know exactly how long and how much you will have GPU power or RAM memory. This is very critical when timing is important (and reproducibility)
• You cannot chose which GPU model will be used: it might be a good one, or an old one like K80
• No background execution except in Pro+: you cannot close your internet browser, or it will automatically stop your work
• Maximum 24 hours time of execution: if you’re running intensive trainings, it’s a serious limitation
• Not official JupyterLab version: the live code editor is based on Jupyterlab, but not the exact open source one. You have some features missing
• Unavailable in multiple countries: quoting their FAQ, “For now, both Colab Pro and Pro+ are only available in the following countries: United States, Canada, Japan, Brazil, Germany, France, India, United Kingdom, and Thailand”
• Requires acceptance of Google ToS: when you use Google Colab, you need to fully accept the Google terms of services and privacy policy. Read them carefully 😃

Seeing their pricing, it’s fully understandable to put some limits.

But what can you do if you want more?

Good news everyone! You have some (simple) alternatives

We are a European company, and if I’m correct, the only one to provide managed notebooks in the cloud with GPU power at scale. We released OVHcloud AI Notebooks a few months ago and it’s really exciting to solve people’s challenges. Based on the current usage, it’s clearly a success (thumbs up to the team behind this new product!).

AI Notebooks = European legislation, guaranteed resources, back-end execution, native Jupyterlab or VSCode editor, no maximum running time, available everywhere… and yet it’s also simple to use.

Go back to the comparative table. You’ll see that we solve many blockers 😃.

To explain further, I’ve made a short video where I start in Google Colab and migrate my work in OVHcloud AI Notebooks. I took my time, explained everything and it lasts 8 minutes. If i wanted to automate it, it should take +-10 seconds.

Want to give it a try? Fearing it’s too expensive?

Our pricing is quite simple: you don’t pay per month, you pay what you consume.

1 CPU is 0,03€ per hour, 1 NVIDIA V100S GPU is 1,75€ per hour.

If you use a notebook with 2 CPUs during 24 hours, it will cost 1,44 euro (0,03 x 2 x 24h).

It’s more expensive compared to Google Colab plans, however it’s not exactly the same product. OVHcloud AI Notebooks offers managed notebooks in the cloud, but with less limitations (our real competitors are AWS Sagemaker or Google AI Platforms).

And we support startups and research! If you are interested, reach our startup program where the whole OVHcloud Public Cloud ecosystem is included (AI tools, K8s, storage, …) up to 100’000 euros.

We also do some philanthropy for schools, open source projects… contact us!

Thanks for reading 😃

A large choice of “vanilla flavor” DBMS

The market is brimming with database engines, we will offer you this large choice over the coming months. Our Public Cloud Databases are of course fully managed and compatible with the full OVHcloud portfolio including bare metal for instance.

Did you know that delegating the setup and daily operations of your database cluster could help your improve your performance by 15% to 20%? That’s a pretty good argument to move your database to the cloud if it wasn’t already the case!

You might still wonder how these DBaaS differ from existing market offerings?

• Aside from MongoDB, all other engines are open-source so that you’ve got no risk of vendor lock-in and can adapt your technology strategy in the long run. We partner with the leader of open-source DBaaS, Aiven, to deliver this full portfolio.
• What’s more, we deliver the official, full and updated experience or “vanilla flavor” of each DBMS, and not a fork in our cloud environment. It means that you benefit from all the features, updates and community innovation.
• They are hosted in our trusted Cloud-Act free cloud infrastructure – in Europe and Canada today, and soon in other parts of the world – to ensure you full data privacy and infrastructure compliance. 
• Last, but not least, after the free beta, their pricing is fully transparent, predictable and much cheaper than competition. Why don’t you check our price list by yourself? What you see is what you pay! All costs including outgoing traffic are included. Don’t forget to multiply the price by the number of nodes!

You voted, we delivered

We know you have been waiting for these managed databases in the public cloud for some time, so we asked you which were your favorite ones and delivered them first. After 3 weeks of usage, here is our beta testers’ podium:

1. Almost half of our first beta testers use PostgreSQL. It’s the most advanced, extensible and highly secure relational database that you use for mission critical business applications. 
2. Another big third of our users are on MySQL, one of the most popular transactional engines for eCommerce platforms, but also an ideal choice for many web applications. 
3. Redis is the third favorite database with almost 15% adoption. It is frequently used for website caching and personalization, chat and messaging apps but also in the gaming industry for leaderboards.
4. The last few percent of users are on Apache Kafka , the must have distributed data streaming platform. Its horizontal scalability is useful  for IoT data ingestion, real-time analytics as well for event-driven applications.

But what do Kafka, MySQL, PostgreSQL, OpenSearch and Redis do?

Long story short: too many uses cases for one blog post! But let’s dive a bit into the main concepts and key differentiators.

Imagine that you are literally an e-commerce website… At least try to imagine 😀  It may be a little silly, but a good way to understand.

It’s almost Christmas, plenty of customers are creating accounts, ordering gifts, putting their credit cards. As a website, you cannot lose their information, you want to store it precisely. Here come relational databases to the rescue!

Relational databases with MySQL and PostgreSQL

Relational databases are widely adopted worldwide, with pretty simple tables, with columns and rows. Each table can be linked to others, with relationships, it’s called a schema.

The most famous ones are MySQL, PostgreSQL, MariaDB and SQL server.

The idea is always the same, tables and relations, with a goal to guarantee data validity despite errors, power failures.

Imagine that you, the website, receive an order for a new gift; you ask for a payment of 50 euros. The customer is paying, but just for 2 seconds your database is flapping and unreachable.

As a website, do you want approximate data? Do you want to be almost certain that the customer has payed? Certainly not! You want to be 100% confident about the transaction in your databases.

Relational Databases provide ACID guarantees (for Atomicity, Consistency, Isolation and Durability) that make them the best solution to deal with this kind of use case.

MySQL and MariaDB are often used for simple website and web application, such as WordPress.

PostgreSQL is more production grade-oriented, with full ACID compliance, advanced security and high availability (HA mechanism, and various extensions for various use case (geographical data, metrics, …).

Also, PostgreSQL is the only database system in the list owned by no company. It’s owned by the PostgreSQL community, you and me. Unlike MySQL that’s the property of Oracle for example.

Let’s come back to our e-commerce scenario!

It’s Christmas time for our website, your are now able to receive and store correctly your products, the orders, the bills… neat!

But wait… Now more and more customers are coming to you, asking ALWAYS for the same price and description of this wonderful pair of socks. Worst, they put a few products in the basket, but don’t buy them.

You are slowed down by storing and giving information from your local storage (images). In the same way, pushing in database information that you would not need to keep (basket) is frustrating. Last but not least, tons of requests in your relational databases (“SELECT Articles in discounted_prices…”) are always the same, giving the same results…

Grrr! It’s really painful to store this kind of ephemeral information. What if you could store and retrieve this data somewhere very, very fast?

Hello Redis™*! your best friend for caching data

The most handy database engine to answer your need is Redis™*.

Redis has two main characteristics: first, it stores data in volatile memory (RAM), not in classic storage such ad SSD drives. Second, it’s not a relational database, but a key-value store. You can store various type of data, such as HTML code, images, user sessions, gaming leaderboards, chat sessions, …

Each time you have to keep in mind that this data “can” be lost and you will store/retrieve it with huge performance.

Most popular use cases are website performance improvement, databases caching, gaming leaderboard, chat tools,.. Twitter, Snapshat, instagram, AirBnB, Pinterest.. they all use Redis!

The most handy database engine to answer your need is Redis.

Redis has two main characteristics: first, it stores data in volatile memory (RAM), not in classic storage such ad SSD drives. Second, it’s not a relational database, but a key-value store. You can store various type of data, such as HTML code, images, user sessions, gaming leaderboards, chat sessions, …

Each time you have to keep in mind that this data “can” be lost and you will store/retrieve it with huge performance.

Most popular use cases are website performance improvement, databases caching, gaming leaderboard, chat tools,.. Twitter, Snapshat, instagram, AirBnB, Pinterest.. they all use Redis!

Kafka, to avoid kafkaian situations

Apache Kafka, is the world’s most famous streaming tool. Imagine a highly-available and highly efficient platform, allowing you to connect to dozen data sources, “snif” the data in real time with near zero latency and push it somewhere else. That’s Kafka which unlike Redis is used a bit everywhere.

You don’t store information in Kafka, you connect Kafka to information sources. Then, of course, this data can be reused.

Main use cases are messaging between applications (it acts as a buffer), real time analytics and real time data pipelines, aggregation of metrics and logs (imagine a company with 1.5 million website ands 250’000 servers to monitor, for example ).

OpenSearch to keep en eye on everything

OpenSearch is an open source fork of Elasticsearch, allowing you to ingest data, such as all the logs of your website and easily analyze them.

Perfect for us to follow all the 404 errors for example! Wonderful dashboard in minutes, in real time!

Here is an example made with OpenSearch sample data:

The A-team of data ecosystem

As a website, you can think of relational databases, Redis, Opensearch and Kafka as the A-team. They are not similar at all; each of them has a specific role to play in the adventure.

At the end, I love it when a plan comes together!

Learn more about these use cases with Aiven’s Cofounder and CTO, Heikki Nousiainen, at Ecosystem Experience!

*Redis is a trademark of Redis Ltd. Any rights therein are reserved to Redis Ltd. Any use by OVH SAS is for referential purposes only and does not indicate any sponsorship, endorsement or affiliation between Redis and OVH SAS.

After hundreds of cluster delivered for free, dozens of pull requests, tons of constructive discussions with customers and partners,… we are live!

We are really pleased to announce our Public Cloud Databases for MongoDB offers on General Availability 😀

And today is a good day! MySQL, PostgreSQL, Kafka, … are now also available for FREE in Beta!
We will talk about it in another blog post, that’s another story 😀

About MongoDB offers, what does it mean? How does it work? Why you should use it? Let’s find out together!

What do we solve?

Installing MongoDB is an easy operation if you have basic computer administration knowledge. But that’s the easy part.

Operating databases on a daily basis is way more challenging. Databases are quite often critical for your use case, and taking care of them ends up being a full time job. Instead of deploying and maintaining your databases yourself, what if a dedicated team took care of the setup and initial state-of-the-art configuration, securization, and maintenance in operational conditions?

That’s quite appealing. It’s not new in the “Cloud world”, many competitors built Databases as a Services solutions, I guess you’ve heard about the most famous ones, aka Amazon RDS or Google Cloud SQL.

But this is new for OVHcloud in our Public Cloud ecosystem, and a very important step!

We are now able to solve a major customer challenge: fast deployment of database clusters, with scalability, security and resiliency.

Let’s deep dive into the detailed features!

What’s in the box for MongoDB offers?

Official MongoDB software and license

MongoDB is recognized worldwide as the best document-based database engine. This software is available in two versions:

• MongoDB Community: the most common MongoDB version, under Server Side Public Licence (SSPL). To put it simply, it’s free unless you use it as a “Program as a Service”;
• MongoDB Enterprise: same core but advanced features for security, backups, auditing tools, BI connectors, MongoDB consulting… Perfect for advanced use-cases.

At OVHcloud, we made a strong partnership with MongoDB, allowing us to provide both Community and Enterprise licenses “as a Service”. That’s indeed a very good news for customers since we are able to provide “vanilla”, genuine version of MongoDB.

It ensures that you get the full MongoDB experience, and are able to move in / out OVHcloud if required. It’s not a fork, like AWS DocumentDB (see the MongoDB compatibility test) or Azure CosmosDB. Hence, you benefit from the latest MongoDB features as well as the official MongoDB documentation, and drivers work natively.

As I am writing, offers with Community License are in General Availability, and Enterprise will come later.

Easy to start, easy to use, easy to scale

Let’s see how simple it is!

Through our control panel, you can launch a MongoDB Database instance in one minute. Select a region, a version, a service plan, and voila! After a few minutes, your cluster is up and ready, allowing you to manage security rules, users, backups, logs and metrics.

It’s also easy to scale; you can upgrade to a higher service plan, add up to 8 nodes per cluster, and pay what you use- true for every public cloud product -on a hourly basis.

Of course, you can  delete your services in a few clicks, allowing you to try the products for few bucks.

Managed by OVHcloud

Again, our aim is to simplify your life and cloud adoption. What we do for you:

• Install MongoDB clusters, based on your choices (versions, nodes, and co);
• Operate the subjacent Linux system (patches, hotfixe, upgrades, …);
• Operate the DBMS software: provide minor and major DBMS versions;
• Operate the subjacent hardware;

What you remain responsible for:

• your data,
• your structures,
• your code
• performance monitoring…

In short, what’s “in” the database instance. We cannot replace you on this 😀

High-Availability and automatic backups

We built 3 service plans: Essential, Business and Enterprise.

All the plans come with automated daily backups, encrypted and stored ouside the datacenter of the cluster.

Starting with Business plans, High-Availabilty is provided natively with 3 nodes by default (Replica Set), maximum 8 nodes. Each node includes a copy of your data to keep it safe. In case of a node failure, your cluster is still up and ready.

Currently we do not provide clusters replicated across multiple regions but we are working on it.

Logs and Metrics Dashboards

To simplify your obvervability tasks, we provide live logs and main metrics in our control panel.

Our mid-term goal is to expose metrics and log to third-party tools.

Last but not least: simple billing! Cloud without headaches

We really focused on simplifiying your cloud adoption, and one of the biggest drawbacks for customers is to predict and control the cost of their cloud infrastructure.

Take a look at our pricing page, we’ve done our best to keep it simple: no hidden cost, everything is predictive and included:

• Databases services with included storage;
• Guaranteed resource performance over time (no burst, no IOPS to buy)
• Data transfer In/Out
• Daily Backups
• logs and metrics

Zero risk of overspending!

Plenty of use-cases

MongoDB is a document-based database, meaning that you can store heterogeneous data in it, not in a classic SQL way (tables, columns, lines) but in a “flat” way like a JSON file.

Many use-cases benefits from this data storage and queries, such as IoT, Real-time data analytics, Single View and much more. The official MongoDB website details the most important ones.

To get a managed MongoDB, what are the other options in cloud providers?

Let’s have a quick look at competitors. If you are looking for a managed MongoDB cluster under European legislation only, OVHcloud and CleverCloud are the only providers offering it.

Choosing a provider will require more than one table, but here is a quick overview. If you are not aware, MongoDB also provides managed services, it’s called MongoDB Atlas. really impressive offers from a feature perspective. You can host them in AWS, Google Cloud or Hosting. Since it’s not a cloud provider offer, I decided to not compare this last one.

The end! Useful links

I really hope that offers and features are clearer now, and like us that you are over hyper about the data future a OVHclooud. I you need more, here

I really hope that offers and features are clearer now, and that you are as enthusiastic as we are about the data future at OVHcloud!

Should you want to learn more, here we are:

French website

English Website

Official documentation

Discuss with us on our Discord servwr!

We’ve just started providing managed databases as a service in our Public Cloud offer, starting with the wonderful MongoDB engine.

Many others will follow in the coming months. This means so much to us and also for you, our fellow customers. Let me tell you what’s going on.

OVHcloud and databases, a long love-story

If you ask any company around the world about their most critical services in their architecture, there is no doubt that databases will be at the epicenter, almost as critical as the coffee machine 😀. Your data is usually at the heart of your use cases, your websites and applications, your company business.

What’s not so obvious is pushing your data in safe databases. Installing MySQL, for example, is one thing. However, doing it state-of-the-art with strong security, resiliency, and performance is another. It takes knowledge and patience.

That’s why today every cloud provider is building managed databases services with one goal: to free customers from the hard stuff – setup, security, high-availability, monitoring, backups, updates, …

Since we host more than 3 million websites with more than 1.5 million databases behind them, we have solid knowledge and experience of databases (curious ? more details here !). Same for our internal databases clusters, it’s a love story. But up until now, nothing had been provided to our customers in our Public Cloud.

Databases: a missing brick in our Public Cloud ecosystem, now resolved!

If you’re not familiar with the public cloud, I’ll sum it up quickly: you start a new project, add a payment method, then you’re able to launch multiple and various services (such as virtual machines, storage, big data and machine learning platforms) … and at the end of the month, you only pay for what you consumed (per hour or per minute).

It’s called “Public” because some parts, like network equipment or physical servers, is shared with other customers (unlike a Private Cloud). It’s secure. No one has access except for you. But isolation is mostly made with software.

So, back on topic, we launched our OVHcloud Public Cloud over 6 years ago, and have improved our inner ecosystem brick by brick, month by month, year by year. We started with virtual machines, block storage, object storage, cold storage, then added containers (Kubernetes clusters, private registries, ), then network (load balancers, ..) then big data and AI/machine learning, but one major brick was missing: managed databases.

To be fully transparent, we were thinking about it for a very long time, around 4 years if I’m correct. We felt like Indiana Jones trying to avoid traps during the Holy Grail quest. A long and treacherous path. But why so long ?

Well, data volumetry is growing faster than ever, usages are multiplying exponentially, same for the solutions to store and use them. About 10-15 years ago it was about classic domination, like Oracle, MS SQL server, PostgreSQL and MySQL and voila. Four years ago things were way more divided and complex, as Redis, MongoDb, Cassandra, ElasticSearch were already in high demand.

We feel kind of like spectators of a moving landscape, it’s moving so fast that it’s hard to catch up. Don’t blink or you’ll miss something! 😉

Four years ago, we wanted to provide, not only 1 or 2 databases engines, but the most appreciated by our customers, the most requested and loved by communities, and it’s quite complex. Each of them has their own method to security, to perform backups, to setup high resiliency, to manage users and roles, to perform updates, requirement of various hardware, … a brain teaser when you want to industrialise it!

To catch up with the market, we needed to find the best companies outside of OVHcloud to help us on this quest.

We found MongoDB Inc, providing not only a databases engine, but also high quality tools and support to help customers (and cloud providers like OVHcloud) to deploy, maintain and monitor large MongoDB infrastructures. From day one, it worked fast and (furiously) resilient by design.

Introducing MongoDB offers this week, and right now it’s free!

Yeah! Yay! Woohoo! We are thrilled to announce managed databases services in our public cloud, now with MongoDB! That’s really good news.

In just a few clicks (or 1 API call) you’ll have a managed MongoDB cluster with a Community or Enterprise licence, delivered in under 5 minutes. We are one of the few cloud providers worldwide to deliver official MongoDB software, not a fork, as a service.

We even created three service plans to aide you on your journey: Essential, Business and Critical. Full details on the website!

And you know what? It’s free during the Beta phase (100% free, excluding official support or SLA, not qualified for production). Just connect to our OVHcloud control panel and launch Essential or Business MongoDB clusters. Enterprises ones will be available shortly after, in July.

As an OVHcloud customer, you’ll be able to focus on your core business, and rely on us for all your database infrastructure and maintenance needs.

We will take care of the rest like setup, installation, security and then maintain these services operationally.

Don’t get me wrong: you’re still responsible for your own data and your queries. It’s not magic. But we’ll take care of the infrastructure part, which is oh so nice. We added extra features such as daily backup (archived in another geographical region), high availability (MongoDB replica set), and public or private network (vRack). We’ll have more features soon too! For example, observability tools are currently underway and in development.

But what about other database engines?

MySql, PostgreSQL, Redis, Cassandra, M3db, Kafka, ElasticSearch,  … How appealing does this list look!? 😀

Even more good news! We’ve also teamed up with a pure player in the databases field as a technological partner for Public Cloud Databases and streaming tools. This means that MongoDB is not only a serious first step, but in a few short months we will add even more engines!

Their core-business is doing databases and they handle it like no one else. We will be able to catch up and provide to you, our customers, a much bigger and better Public Cloud portfolio. They will simplify industrialization and all the engine-related specifics.

Do not worry! We will come back here at the end of June to share our tips and tricks and lovely insight.

We really hope that you’ll enjoy this new adventure with OVHcloud. Try for yourself and please share your feedback on OVHcloud Discord #database-MongoDB!

Whether the machine learning workload involves fraud detection, forecasts, chatbots, computer vision or NLP, it will need frequent access to computing power for training and fine-tuning.

GPUs have proven to be a game-changer for deep learning. If you’re wondering why, you can find out more by reading our blog post about GPU architectures. A few years ago, manufacturers such as NVIDIA began to develop specific ranges for cloud datacentres. You may be familiar with the NVIDIA TITAN RTX for gaming — and in our datacentres, we use NVIDIA A100, V100, Tesla and DGX GPUs for enterprise-grade workloads.

In short, GPUs are perfect for tasks that can be solved or improved by AI, and require a lot of processing power.
They offer optimal compute, and are widely used in deep learning. A growing number of companies are using AI, and GPUs seem to be the best choice for them.

However, when dealing with pools of GPUs, the back-end architecture can be really tricky.

So how do we use them to benefit a company with minimal hassle and headaches? On-premise or in the cloud?

These are good questions that I’m keen to discuss here, from both a business and technical perspective.

Dealing with GPU pools… The struggle is real.

For anyone who has had to deploy and manage more than 1 GPU for a data-AI team, I’m sure this topic will bring tears to your eyes, and make your voice tremble. Yes, it is indeed complicated.

I can talk about it on our blog, because our team of data scientists here at OVHcloud had to deal with the exact same annoying issues. Thankfully, we solved all of them — stay tuned!

GPU sharing is hard. Even if one GPU is better than none, in most cases it will not be sufficient, and a GPU pool will be far more effective. From a tech perspective, dealing with a GPU pool — or worse, allowing your team to use this pool simultaneously — is very tricky. The market is really mature for CPU sharing (via hypervisors), but by design, a GPU has to be attached to a VM or container. This means that quite often, it needs to be “booked” for a specific workload. To get around this issue, you’ll need to provide a scale-out with orchestration, so that you can dynamically assign GPUs to jobs over time. Whenever you tell yourself “I want to launch this task with 4 GPUs for 2 days“, you should simply be able to ask, and the back-end should work its magic for you.

Setting up and maintaining an architecture is time-consuming.  So you’ve deployed servers with GPU, updated and upgraded your Linux distros, installed your main AI packages, CUDA drivers, and now you want to move on to something else. But wait — a new TensorFlow version has been released, and you also have a security patch to apply. What you initially thought to be a single task is now taking up 4-5 hours of your time per week.

Diagnosing is quite complex. If, for whatever reason, something isn’t working as it should — good luck. You barely know who is doing what, and you can’t track jobs or usage unless you connect to the platform yourself and set up monitoring tools. Remember to grab your snorkel set, because you’ll need to deep-dive.

Bottlenecks are almost inevitable. Imagine setting up a pool of GPUs based on your current AI project workloads. Your infrastructure is not really designed to scale automatically, and as soon as the AI workloads increase, your jobs have to be scheduled while the GPU fleet is being updated constantly. A backlog starts to accumulate, and a bottleneck is created as a result.

Providing tools for teams to work collaboratively on code is mandatory. Usually, your team will need to share their data experimentations — and the best way to do this for now is with JupyterLab Notebooks (we love them) or VSCode. But you’ll need to keep in mind that this is more software to set up and maintain.

Securing data access is essential. The required data must be easily accessible, and sensitive data must be covered by security guarantees.

Cost control is difficult. Even worse, for one reason or another (who said holidays?), you might need to stop almost all your GPU servers for a week or two — but to do this, you would need to wait for any ongoing jobs to be completed.

All jokes aside, while we may be passionate about tech and hardware, we have other things to do. Data engineers cannot achieve their full potential and talent in maintenance-based or billing-based tasks.

Kubeflow to the rescue?

Kubernetes 1.0 was launched 5 years ago. Whatever your opinion is on it, in five years they have become the de facto standard for container orchestration in enterprise environments.

Data scientists use containers for portability, agility, and community — but Kubernetes was made to orchestrate services, not data experimentations.

Kubernetes alone is not tailored for a data team. It presents too much complexity, with the sole benefit of solving the orchestration issue.

We need something that not only improves orchestration, but also code contribution, tests and deployments.

Luckily, Kubeflow appeared 2 years ago, and was open-sourced by Google at the time. Its main promise is to simplify complex ML workflows, for example data processing => data labeling => training => serving, and complete it with notebooks.

I do really love the promise, and the way they simplify ML pipelines. Kubeflow can be run over K8s clusters on-premise or in the cloud, and can also be set up on a single VM or even on a workstation (Linux/Mac/Windows).

Students can easily have their own ML environment. However, for the most advanced uses, a workstation or a single VM might be out of the question, and you would need a K8s cluster with Kubeflow installed on top of that. You’ll have a nice UI for starting notebooks and creating ML pipelines (processing/training/inference), but still zero GPU support by default.

Your GPU support will depend on your setup. It may differ if you host it on GCP, AWS, Azure, OVHcloud, on-premise, MicroK8s, or anything else.

For example, on AWS EKS, you need to declare GPU pools in your Kubeflow manifest:

# Official doc: https://www.kubeflow.org/docs/aws/customizing-aws/

# NodeGroup holds all configuration attributes that are specific to a node group
# You can have several node groups in your cluster.
nodeGroups:
  - name: eks-gpu
    instanceType: p2.xlarge
    availabilityZones: ["us-west-2b"]
    desiredCapacity: 2
    minSize: 0
    maxSize: 2
    volumeSize: 30
    ssh:
      allow: true
      publicKeyPath: '~/.ssh/id_rsa.pub'

On GCP GKE, you will need to run this command to export a GPU pool:

# Official doc: https://www.kubeflow.org/docs/gke/customizing-gke/#common-customizations
 
export GPU_POOL_NAME=<name of the new GPU pool>
 
gcloud container node-pools create ${GPU_POOL_NAME} \
--accelerator type=nvidia-tesla-k80,count=1 \
--zone us-central1-a --cluster ${KF_NAME} \
--num-nodes=1 --machine-type=n1-standard-4 --min-nodes=0 --max-nodes=5 --enable-autoscaling

You will then need to install NVIDIA drivers on all the GPU nodes. NVIDIA maintains a deamonset, which enables you to install them easily:

# Official doc: https://www.kubeflow.org/docs/gke/customizing-gke/#common-customizations
 
kubectl apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/container-engine-accelerators/master/nvidia-driver-installer/cos/daemonset-preloaded.yaml

Once you have done this, you will be able to create GPU pools (don’t forget to check your quotas before — with a basic account, you are restricted by default, and you will need to contact their support).

Okay, but do things get easier from here?

As we say in France, especially in Normandy, yes but no.

Yes, Kubeflow does resolve some of the challenges we’ve mentioned — but some of the biggest challenges are yet to come, and they will take up a lot of your daily routine. Many manual operations will still require you to dig into specific K8s documentation, or guides published by cloud providers.

Below is a summary of Kubeflow vs GPU pool challenges.

Forget infrastructure, welcome to GPU platforms made for AI

You can now find various third-party solutions on the market that go one step further. Instead of dealing with the architecture and the Kubernetes cluster, what if you simply focused on your machine learning or deep learning code?

There are well-known solutions such as Paperspace Gradient — or smaller ones, like Run:AI — and we’re pleased to offer another option on the market: AI Training. We’re using this post as a self-promotion opportunity (it’s our blog after all), but the logic remains the same for competitors.

What are the concepts behind it?

No infrastructure to manage

You don’t need to set up and manage a K8s cluster, or a Kubeflow cluster.

You don’t need to declare GPU pools in your manifest.

You don’t need to install NVIDIA drivers on the nodes.

With GPU Platforms like OVHcloud AI Training, your neural network training is as simple as this:

# Upload data directly to Object Storage
ovhai data upload myBucket@GRA train.zip
 
# Launch a job with 4 GPUs on a Pytorch environment, with Object Storage bucket directly linked to it
ovhai job run \
     --gpu 4 \
    --volume myBucket@GRA:/data:RW \
    ovhcom/ai-training-pytorch:1.6.0

This line of code will provide you with a JupyterLab Notebook directly plugged to a pool of 4x NVIDIA GPUs, with the Pytorch environment installed. This is all you need to do, and the entire process takes around 15 seconds.

Parallel computing — a great advantage

One of the most significant benefits is that since the infrastructure is not on your premises, you can count on the provider to scale it.

So you can run dozens of jobs simultaneously. A classic use case is to fine-tune all of your models once a week or once a month, with 1 line of bash script:

# Start a basic loop
for model in my_models_listing
do
 
# Launch a job with 4 GPUs on a Pytorch environment, with Object Storage bucket directly linked to it
echo "starting training of $model"
ovhai job run \
--gpu 3 \
--volume myBucket@GRA:/data:RW \
my_docker_repository/$model
 
done

If you have 10 models, it will launch 10x 3 GPUs in few seconds, and it will stop them once the job is complete, from sequential to parallel work.

Collaboration out of the box

All of these platforms natively include notebooks, directly plugged to GPU power. With OVHcloud AI Training, we also provide pre-installed environments for TensorFlow, Hugging Face, Pytorch, MXnet, Fast.AI — and others will be added to this list soon.

Data set access made easy

I haven’t tested all the GPU platforms on the market, but usually they provide some useful ways to access data. We aim to provide the best work environment for data science teams, so we are also offering an easy way for them to access their data — by enabling them to attach object storage containers during the job launch.

Cost control for users

Third-party GPU platforms quite often provide clear pricing. This is the case for Paperspace, but not for Run:AI (I was unable to find their price list). This is also the case for OVHcloud AI Training.

• GPU power: You pay £1.58/hour/NVIDIA V100s GPU
• Storage: Standard price of OVHcloud Object Storage (compliant with AWS S3 protocol)
• Notebooks: Included
• Observability tools: Logs and metrics included
• Subscription: No, it’s pay-as-you-go, per minute

And there we go — cost and budget estimation is now simple. Try it out for yourself!

Mission complete?

Below is a summary addressing the major challenges to resolve when dealing with GPU pool sharing. It’s a big yes!

Conclusion

If we go back to the main challenges faced by a company that requires shared GPU pools, we can say without a doubt that Kubernetes is a market-standard for AI pipeline orchestration.

An on-premise K8s cluster with Kubeflow is really interesting if the data cannot be processed into the cloud (e.g. banking, hospitals, any kind of sensitive data) or if your team has flat (and lower-level) GPU requirements. You can invest in a few GPUs and manage the fleet yourself with software on top. But if you need more power, very soon the cloud will become the only viable option. Hardware investments, hardware obsolescence, electricity usage and scaling will give you some headaches.

Then, depending on the situation, Kubeflow in the cloud might be really useful. It delivers powerful pipeline features, notebooks, and enables users to manage virtual GPU pools.

But if you want to avoid infrastructure tasks, control your spending, and focus on your added value and code, you might consider GPU platforms as your first choice.

However, there is no such thing as magic — and without knowing exactly what you want, even the best platform won’t be able to meet your needs. Yet some start-ups, not listed here, can offer a combination of platforms and expertise to help you in your project, infrastructures and use cases.

Thank you for reading, and don’t forget that we also offer inference at scale with ML Serving. This is the next logical step after training.

Want to find out more?

• Solution page: https://www.ovhcloud.com/en-gb/public-cloud/ai-training/
• Public documentation: https://docs.ovh.com/gb/en/ai-training/
• Community: community.ovh.com/en/