Web hosting – How do our databases work?

In our series of articles about migrating web hosting infrastructure from Paris to Gravelines (the first of which is here), we neglected to mention the databases. This is however a very important part, with 800,000 databases to be migrated!

In these new series of posts, we will tell you about our experience with web hosting databases.

How to handle 800k databases?

Our previous posts explained why we have to migrate databases along with their websites. You also know that we set up a network bridge for customers using non-referenced databases.

Yet migrating databases also brings challenges. We will describe the most important ones.  But first, a little reminder of our different architectures.

OVH offers two types of databases:

• Shared databases (which we call SharedSQL)

• Private databases (which we call, you guessed it, PrivateSQL)
  

What is a SharedSQL?

As the name suggests, it is a database on a shared server, which contains many other bases, and which shares its hardware resources, including RAM.

Of course, not everyone has access to everyone’s data! Each database is defined with a single user able to access it.

This technique allows us to exploit up to 2500 databases on a single server. This is much cheaper than offering a physical server per customer. This is how we can offer low cost access to several databases within our hosting offers.

What is a PrivateSQL?

Unlike SharedSQL, PrivateSQL are databases with guaranteed resources, including RAM. In PrivateSQL, users also have many more rights and greater customisation capabilities.

Does each user have their own server? Not really! Several years ago we used Docker technology to containerise our databases, we already discussed it in this post: https://www.ovh.com/en/blog/docker-administration-databases-a-flying-ideas/. With a PrivateSQL, it’s not just the database space that is private, the RAM assigned to the service is also guaranteed. This means that in no matter the circumstance, the performance is consistent.

Seven differences!

When considering migration, we had to study the difference in architectures between Paris and Gravelines.

Regarding PrivateSQL, it was simple: there is no difference, we had already aligned the architectures when we containerised them. Migrating them would be a piece of cake: we just had to move a container from point A to point B.

At least on paper, in real life it was not so simple, but we will talk about it next time.

Here is a simplified diagram of a PrivateSQL

In contrast, SharedSQL databases were not really aligned. When we set up the Gravelines datacentre in 2016, we took advantage of Docker’s flexibility to review our database technology, and therefore adapt our former solution.

Small comparative schema of a SharedSQL in Paris and Gravelines:

SharedSQL at Paris P19 and Gravelines:

On the one hand (in Paris), we have servers with a single database management system (MySQL) hosting 2500 databases on one machine.

On the other (in Gravelines), we have added a level: Docker containers have the same database system (MySQL) hosting up to 250 databases. On each machine, we host 10 containers.

This new system greatly simplifies maintenance, reducing the impact on database performance as it occurs.

Here is what it provides:

	Paris	Gravelines	Explanation
Number of clients impacted by an incident on an instance of MySQL	2500	250	When MySQL is on a container, the impact is lower than when it is directly on the server.
Total number of customers per server			Equivalent. In Paris, the 2500 databases operate on the same MySQL process. In Gravelines, they share 10 processes, each in their container. But the number per physical machine remains the same.
Easy server migration in the event of a hardware failure			Docker containers provide us with ease of migration and disaster recovery between two machines. Thus, we can easily and quickly move a container from one server to another without data loss.
Startup Speed			Starting a container is about the same as starting a process.
Version upgrade			The transition from one version to another is simplified with the Docker image system. We just need to change the image of the container to the desired version, and restart the container.
Rollback in case of problem during an update			We just have to restart the container on its previous image.
Robust and long-lasting implementation			Running databases under Docker is not simple (see https://www.ovh.com/blog/docker-administrating-databases-a- crazy-idea /), but we have a few years’ experience.

Of course, running a database in a Docker container seemed like a crazy idea. But after many years of production, this technology simplifies everyday life in the Gravelines datacentre.

So when we had to choose between replicating the Paris infrastructure, or changing infrastructure during migration, we chose to change technology.

And all of a sudden we have another migration challenge.

In our next article, we will detail how to move a huge database in a few minutes, while changing a large part of the software and the infrastructure behind it!