At OVHcloud our motto is “Innovation for freedom”. OVHcloud is powered by passionate people trying to stay at the state of the art of technologies. But many things slow innovation: handling legacy is one of them.
One way to tackle that is to define and follow good life cycle policies.
What is a life cycle policy
A life cycle policy is a set of guidelines, written documents, describing the different steps in a product’s life, including its end of life.
Why you should have a life cycle policy
When you read the news, it is not uncommon to hear from here and there that company XYZ got hacked through an old software, outdated for ages.
Having a good life cycle policy guarantees you never have to deal with outdated software. It defines when you must upgrade your software and when you must stop using an old version and migrate to a new one.
Benefits are multiple:
- You don’t have to rely on unmaintained software
- You reduce attack perimeter, as newer versions have security patches applied
- You benefit from new versions’ features, allowing you to accelerate innovation
Of course there are downsides. The biggest one is having to stick with it. A life cycle policy is a written document and as such has value only if people read it and apply its recommendations.
Why you should automate it
A wise man once told “A good sysadmin is a sysadmin doing nothing”. Not because he is lazy, nor because he is bad at his job and got his credentials revoked, but because he automates all painful tasks, to focus on tasks with added value.
It also removes one major cause of errors: the human factor. An automated job will never mistype a command line or click on the `destroy` button, instead of the `upgrade` one.
Last but not least, it is a good way to make sure your carefully crafted document is not redirected to `/dev/null` in the following months.
Bonus: you also free a lot of time to automate new things !
How we did it
Our policy on cloud databases (available here: https://docs.ovh.com/gb/en/clouddb/managed-db-life-cycle-policy/) forces us to warn our customers at least 30 days before a major upgrade by e-mail. This email contains the day the upgrade will take place on.
We also have an internal mechanism allowing our customers to upgrade by themselves, using our API.
Combining these two bricks were all we had to do, with minor adjustments:
- When sending a email, we keep track of the planned date of the operation.
- An automated job is triggered every day, checking what operations are due this day and launching them.
Almost everyday, our job realises there is no major upgrade planned and goes back to sleep. But if a single operation is found, this job calls our API, triggering this upgrade. No human needed.
This process is very simple, yet very effective. We already had all the building blocks ready, from sending an email to API calls for major upgrades. All we had to do was to link all these operations together in a nice workflow. This allow us a very fine tuning of how many operation we want being done everyday, down to the instance.
Want to try the process for the first time ? Easy, trigger it on one of our test instance.
Is this process reliable and scalable ? Enough to upgrade a batch of 10 000 instances in 30 days with no human intervention !
One last word on an automated life-cycle policy
At Web Cloud Databases, with thousands of instances, 4 different DBMS and a total of 12 major versions, automation is the key.
In May 2021, for this process’ firsts steps in this world, we upgraded 6K instances in a week. The only human interaction needed was to launch a script to inform all affected customers.
Figures of this use-case ? 6000+ instances updated, 0 website down, 1 human needed for 5 minutes.
Member of the database team since 2017