Before delving into a full explanation of CDS workflows, let’s review some of the key concepts behind pipelines and jobs. Those concepts are drawn from the reference book, 8 Principles of Continuous Delivery

The basic element: “The job” 

A job is composed of steps, which will be run sequentially. A job is executed in a dedicated workspace (i.e. filesystem). A new workspace is assigned for each new run of a job.

A standard build job looks like this: 

You can use “built-in” actions, such as checkoutApplication, script, jUnit, artifact upload/download.

• The checkoutApplication action clones your Git repository
• The Script action executes your build command as “make build”
• The artifactUpload action uploads previously-built binaries
• The jUnit action parses a given Junit-formatted XML file to extract its test results

A pipeline: How to orchestrate your jobs with stages

With CDS, a pipeline is not a job flow. A pipeline is a sequence of stages, each of which contains one or more jobs.

A  Stage  is a set of jobs that will be run in parallel. Stages are executed sequentially, if the previous stage is successful. 

Let’s take a real-life use case: the pipeline that built CDS. This pipeline has four stages: 

• The “Build Minimal” stage is launched for all Git branches. The main goal of this stage is to compile the Linux version of CDS binaries. 
• The “Build other os/arch” stage is only launched on the master branch. This stage compiles all  binaries supported by the os/arch: linux, openbsd, freebsd, darwin, windows – 386, amd64 and arm.  
• The “Package” stage is launched for all Git branches. This stage prepares the docker image and Debian package.
• Finally, the “Publish” stage is launched, whatever the Git branch. 

Most tasks are executed in parallel, whenever possible. This results in very fast feedback, so we will quickly know if the compilation is OK or not. 

CDS Workflows: How to orchestrate your pipelines

The workflow concept is a key feature, and widely considered a native, manageable and feature-rich entity in CDS. A CDS Workflow allows you to chain pipelines with manual or automatic gates, using conditional branching. A workflow can be stored as code, designed on CDS UI, or both, depending on what best suits you. 

Let’s take an example. One workflow for building and deploying three micro-services:  

• Build each micro-service 
• Deploy them in preproduction 
• Run integration tests on preproduction environment 
• Deploy them in production, then re-run integration tests in production

For the building part, there is only one pipeline to manage, which is used three times in the workflow with a different application/environment context each time. This is called the “pipeline context”. 

Any conditional branching against the workflow (e.g.“automatic deployment on the staging environment, only if the current Git branch is master”) can be executed through a “run conditional” set on the pipeline. 

Let’s look at a  a real use case. This is the workflow that builds, tests and deploys CDS in production at OVH (yes, CDS builds and deploys itself!):

1. For each Git commit, the workflow is triggered 
2. The UI is packaged, all binaries are prepared, and the docker images are built. The “UT” job launches the unit tests. The job “IT” job installs CDS in an ephemeral environment and launches the integration tests on it. Part 2 is automatically triggered on all Git commits.  
3. Part 3 deploys CDS on our preproduction environment, then launches the integration tests on it. It is started automatically when the current branch is the master branch. 
4. Last but not least, part 4 deploys CDS on our production environment. 

If there is a failure on a pipeline, it may look like this:

The same kind of workflow is used for building and deploying the Prescience Project (https://labs.ovh.com/machine-learning-platform):

But of course, you’re not limited to the most complex tasks with CDS Workflows! These two examples demonstrate the fact that workflows allow to build and deploy a coherent set of micro-services. If you have simpler needs, your workflows are, of course, simpler.

Pipeline reusability allows you to easily maintain the technical parts of the build, tests and deployments, even if you have hundreds of applications. If hundreds of applications share the same kind of workflows, you can leverage the maintainability of the workflow templates. We will talk more about this in a future post.

Much more than “Pipeline as Code”… “Workflow as Code”

There is no compromise with CDS. Some users prefer to draw the workflows by the web UI, others prefer to write yaml code. CDS lets you do both! 

There are two ways to store workflows: either in the CDS database or on your Git repository with your source code. We call this “Workflow as Code”.

This makes it possible to have a workflow on a given branch, and then develop it on another branch. CDS will instantiate the workflow on the fly, based on the yaml code present on the current branch.

CDS is OVH open-source software, and can be found on https://github.com/ovh/cds, with documentation on https://ovh.github.io/cds. 

Previous Posts: 

• CDS Introduction: https://www.ovh.com/fr/blog/how-does-ovh-manage-the-ci-cd-at-scale/  
• DataBuzzWord Podcast (French): https://www.ovh.com/fr/blog/understanding-ci-cd-for-big-data-and-machine-learning/   

The Continuous Delivery Team at OVH has one fundamental mission: to help the OVH developers industrialise and automate their delivery processes. The CD team is here to advocate CI/CD best practices and maintain our ecosystem tools, with the maximum focus on as-a-service solutions.

The centre of this ecosystem is a tool called CDS, developed in-house at OVH.
CDS is an open-source software solution that can be found at https://github.com/ovh/cds, with documentation at https://ovh.github.io/cds.

CDS is the third generation of CI/CD tools at OVH, following two previous solutions, that were based on Bash, Jenkins, Gitlab and Bamboo. It is the end-result of 12 years’ experience in the field of CI/CD. Familiar with most of the standard tools of the industry, we found that none completely matched our expectations regarding the four key aspects we identified. That is what CDS tries to solve.

These four aspects are:

Elastic

CDS resources/workers are launched on demand, to guarantee low waiting times for users, with no over-consumption of idle resources.

Extensible

In CDS, any kind of action (Kubernetes and OpenStack deployments, pushing to Kafka, testing for CVEs…) can be captured in high-level plugins, to be used as building blocks by users. These plugins are straightforward to write and use, so it’s easy to meet the most exotic needs in an effective and stress-free way.

Flexible, but easy

CDS can run complex workflows, with all sorts of intermediary steps, including build, test, deploy 1/10/100, manual or automatic gates, rollback, conditional branches… These workflows can be stored as code in the git repository. CDS provides basic workflow templates for the Core team’s most common scenarios, in order to ease the adoption process. This way, building a functional CI/CD chain from nothing can be quick and easy.

Self-service

Finally, a key aspect is the idea of self-service. Once a CDS project is created by users, they are completely autonomous within that space, with the freedom to manage pipelines, delegate access rights etc. All users are free to customise their space as they see fit, and build on what is provided out-of-the-box. Personalising workflow templates, plugins, running build and tests on custom VM flavors or custom hardware… all this can be done without any intervention from the CDS administrators.

CI/CD in 2018 – Five million workers!

• About 5.7M workers started and deleted on demand.
• 3.7M containers
• 2M Virtual Machines

How is it possible?

One of the initial CDS objectives at OVH was to build and deploy 150 applications as a container in less than seven minutes. This has been a reality since 2015. So what’s the secret? Auto-Scale on Demand!

With this approach, you can have hundreds of worker models that CDS will launch via hatcheries whenever necessary.

A hatchery is like an incubator: it gives birth to the CDS workers and maintains the power of life and death over them.

Each hatchery is dedicated to an orchestrator. Furthermore, one CDS Instance can create workers over many cloud platforms:
– The Kubernetes hatchery starts workers in pods
– The OpenStack hatchery starts virtual machines
– The Swarm hatchery starts docker containers
– The Marathon hatchery starts docker containers
– The VSphere hatchery start virtual machines
– The local hatchery starts process on a host

What’s next?

This is all just a preview of CDS… we have lots more to tell you about! The CI/CD tool offers a wide range of features that we will explore in depth in our upcoming articles. We promise, before 2019 is done, you will not look at your CI/CD tool the same way again…

Together, we explored the topic of continuous deployment in the context of machine learning and big data. We also discussed continuous deployment for environments like Kubernetes, Docker, OpenStack and VMware VSphere.

If you missed it, or would like to review everything that was discussed, you can listen to it again here. We hope to return soon, to continue sharing our passion for testing, integration and continuous deployment.

Although the podcast was recorded in French, starting from tomorrow, we’ll be delving further into the key points of our discussion in a series of articles on this blog.

Find CDS on GitHub:

• https://github.com/ovh/cds

…. and follow us on Twitter:

• https://twitter.com/yesnault
• https://twitter.com/francoissamin

Come chat about these subjects with us on our Gitter channel: https://gitter.im/ovh-cds/