The delivery process is the set of steps - from git commit to production - that takes place to deliver your service to your customers. Drawing on agile values, Continuous Integration and Continuous Delivery (CI/CD ) are practices that aim to automate this process as much as possible.
The Continuous Delivery Team at OVH has one fundamental mission: to help the OVH developers industrialise and automate their delivery processes. The CD team is here to advocate CI/CD best practices and maintain our ecosystem tools, with the maximum focus on as-a-service solutions.
The centre of this ecosystem is a tool called CDS, developed in-house at OVH.
CDS is an open-source software solution that can be found at
https://github.com/ovh/cds,
with documentation at
https://ovh.github.io/cds
.
CDS is the third generation of CI/CD tools at OVH, following two previous solutions, that were based on Bash, Jenkins, Gitlab and Bamboo. It is the end-result of 12 years' experience in the field of CI/CD. Familiar with most of the standard tools of the industry, we found that none completely matched our expectations regarding the four key aspects we identified. That is what CDS tries to solve. These four aspects are:
Elastic
CDS resources/workers are
launched on demand
, to guarantee low waiting times for users, with no over-consumption of idle resources.
Extensible
In CDS, any kind of action (Kubernetes and OpenStack deployments, pushing to Kafka, testing for CVEs…) can be captured in
high-level plugins
, to be used as
building blocks
by users. These plugins are straightforward to write and use, so it's easy to meet the most exotic needs in an effective and stress-free way.
Flexible, but easy
CDS can run
complex workflows
, with all sorts of intermediary steps, including build, test, deploy 1/10/100, manual or automatic gates, rollback, conditional branches… These workflows can be
stored as code
in the git repository. CDS provides basic
workflow templates
for the Core team's most common scenarios, in order to ease the adoption process. This way, building a functional CI/CD chain from nothing can be quick and easy.
Self-service
Finally, a key aspect is the idea of
self-service
. Once a CDS project is created by users, they are completely autonomous within that space, with the freedom to manage pipelines, delegate access rights etc. All users are free to customise their space as they see fit, and build on what is provided out-of-the-box. Personalising workflow templates, plugins, running build and tests on custom VM flavors or custom hardware… all this can be done without any intervention from the CDS administrators.
CI/CD in 2018 - Five million workers!
- About 5.7M workers started and deleted on demand.
- 3.7M containers
- 2M Virtual Machines
How is it possible?
One of the initial CDS objectives at OVH was to build and deploy 150 applications as a container in less than seven minutes. This has been a reality since 2015. So what's the secret? Auto-Scale on Demand! With this approach, you can have hundreds of worker models that CDS will launch via hatcheries whenever necessary. [caption id="attachment_14542" align="aligncenter" width="512"]
CDS Hatchery[/caption] A hatchery is like an incubator: it gives birth to the CDS workers and maintains the power of life and death over them. [caption id="attachment_14546" align="aligncenter" width="512"]
CDS Hatcheries - Worker @Scale[/caption] Each hatchery is dedicated to an orchestrator. Furthermore, one CDS Instance can create workers over many cloud platforms: - The
Kubernetes
hatchery starts workers in pods - The
OpenStack
hatchery starts virtual machines - The
Swarm
hatchery starts docker containers - The
Marathon
hatchery starts docker containers - The
VSphere
hatchery start virtual machines - The
local
hatchery starts process on a host [caption id="attachment_14548" align="aligncenter" width="512"]
CDS Hatcheries[/caption]
What's next?
This is all just a
preview of CDS
... we have lots more to tell you about! The CI/CD tool offers a wide range of features that we will explore in depth in our
upcoming articles
. We promise, before 2019 is done, you will not look at your CI/CD tool the same way again...