• The birth of agile telemetry at OVHcloud – Part I
• Agile telemetry at OVHCloud – Part II

In the final post relating to agile telemetry, we focus on the micro-vision. In any project, keeping micro and macro vision separate is vital for effective communication.

Macro and Micro vision

For development teams, micro-vision is crucial. While a macro perspective is important for meeting goals such as delivery deadlines, overseeing the micro details of a development ensures a product is good quality. Here is how we managed to achieve this important goal:

Micro-Vision

This part of our Dashboard displays three important pieces of information relating to the progress of our sprints: the expected, the unexpected and the reason for the unexpected.

(1) This is the progress of the current sprint and its percentage of completion. The percentage of completion is simply computed:

(2) This highlights the number of times the development team has gone out of its sprint to fix something and the completion times. The attached graph shows the peak days when the team was mobilized.

To report this type of information:

• Teams add in the “Label” field on JIRA the title: impediment.
• They log the time spent in the “log work” field on JIRA.

The accumulated information is then visible in our dashboard. This helps us to understand why a sprint does not progress. When teams go through weeks of troubleshooting, we can explain why.

NB: It is essential to separate the sprint from your impediments. These two pieces of information allow you to compare the “expected” and the “unexpected”. This allows us to propose 2 types of graphs.

(3) The last part highlights why the teams intervened when they came out of their sprint. This precious data gives us a clear vision of the situation. If a team spends 90% of its time dealing with a bug, or troubleshooting in support, it informs decisions making processes for future projects.

Example : We provide 30 hours of bug correction time during the sprints; should we focus on creating something new, or reducing the technical debt?

Conclusion : Today, Agile telemetry is used by 19 teams at OVHCloud. It facilitates and allows us to measure the performance and progress of the teams remotely, and in real time. It also provides us with the up-to-date information we need to conduct our projects. It is this methodology that has won Innovations Awards for OVHCloud employees.

If you want to dive into Agile Telemetry you can do it now!! Simply grab our open source dashboard and our bot Jerem.

• Establish an efficient, simple and universally-accepted methodology to support the development teams.
• Set out the different visions necessary for the good handling of a project.
• Define the micro vision, for development teams.
• Define the macro vision, for sponsors.
• Industrialise this innovative concept for other teams at OVHcloud.

As with every new challenge, there was an element of complexity. OVHcloud is an international company, and our teams are distributed all around the world. The tool therefore needed to be easily readable and simple to maintain, and the indicators had to update automatically, as our teams work in a changing context where there are often unplanned events

We needed to create a tool that would allow us to remotely measure the performance and progress of teams in real time, and update the information needed to manage a project. This is our telemetry.

“Before engaging a development team, create your construction plans… draw!” With this idea in mind, I started to sketch a few drafts on a notebook, then on software.

This first sketch highlighted what I wanted to show my teams. We could track our progress on every subject we were working on (the Epics we talked about in part I). We were able to see if the roadmap was achievable, based on the rhythm of the team, and measure the team’s performance. Finally, we had a tool that reconciled the micro and macro visions.

But how do you make this board come alive?

At OVHcloud we use a ticketing tool: JIRA. This tool helps us in our everyday project organisation. However, it doesn’t show the information we need. Nevertheless, we can make it talk…

Based on this concept, we and the teams have built a metrics table with the GRAFANA tool.

This is our Agile telemetry

In this second blog post, we will be focusing on the macro vision. The next post will look at the micro vision part. 😉

Macro vision

This part of the table gives us a MACRO view of the progression throughout the quarter, and the associated charts.

1^st data block: Quarter KPIs

• (A) Predictability. Are you able to deliver all projects/requests? With this indicator, top management and development teams can monitor their capacity regarding deadlines. We are able to monitor this data with the following formula:

• (B) Quarter progression. Shows the completed parts of the quarter.

• (C) Team velocity. The number of story point completed per day. With this information, we are able to built our future sprints, and maintain the team’s rhythm.

• (D) Average impediment. This information is critical to building the next sprint and estimating the security coefficient.

Example

Let’s say we have a seven-person team, with a capacity of 350 hours per sprint (7 people x 5 hours/day x 10 days). If our team logged 58 hours as impediments during a sprint, we could use this information to tailor the security coefficient for the next sprint.

On average, the team use 17% of their time to resolve impediments during a sprint. With this information, we are able to make a security coefficient.

2^nd data block: Sum of Roadmap Epics

This table collects the expected epics. It shows the number of story points evaluated by the teams, the number of points they have already completed, the number of tasks “not estimated”, and the rate of progress. With this table, we can give figures concerning a progression. 

3^rd data block: Quarter Completion

1. Epic completion (1): During each week, we can watch the roadmap’s progression. Sometimes, we will see the completion roadmap decrease. It’s normal, because we have added a story point in our plan. (4)
2. Impediment count (2): This is the number of times the teams came out of the sprint due to impediments.
3. Impediment duration (3): The total time spent on impediments.
4. Delta story point planned (4): The number of points entering or leaving the roadmap, allowing us to see any variations on the charts.

4^th data block: Draw me a graph!

• A.  This figure represents the progress curve of the roadmap. It is created using the sum of the completed epics.
• B.  Our burnup roadmap.

This graph represents two curves:

• a – The curve of the elements quantified and estimated by the teams.
• b – This is the limit, calculated using the velocity of the team on the number of days in the quarter. We then add the team’s safety coefficient.

• c – The curve of the tasks completed by the teams.

Conclusion

This dashboard brings together all the information that we have managed to pull together, thanks to the amazing contributions of the teams. By believing in the values and trusting the method, the group already has a maturity that they have achieved themselves.

For our top management, this board shows all the key information they need. When the predictability is green, all is well. When something is wrong, they are able to take the right decision, whether this involves changing priorities on the roadmap, or hiring new people to help teams successfully complete it (for example).

In our next post, I will explain the micro vision in detail. Until then, keep Agile !

My first action, as a newcomer, was to ask the team to express themselves. I wanted to understand how the group worked, as listening is one of the first steps in the agile construction process. 

When dealing with significant changes, it’s necessary to do some change management and accompany people throughout the process. All too often, the word “agile” is used in a way that sacrifices common sense, becoming the scapegoat when unforeseen events or delays in delivery occur. Reassuring the team and explaining them that agility wasn’t going to make their lives harder, or add an overhead to the daily tasks, was a key objective. We achieved this in several ways:

• Organising several presentations and methodology training sessions
• Holding physical meetings with the teams at Paris and Lyon 
• Explaining how the method would work during a real sprint
• Collectively drafting a guide to our implementation of the SCRUM method 

Once we had validated the fundamentals together, we moved on to the methodology. At this stage, we could begin to add value to the project by using agility.

Value in our agility

After a few sprints using the agile method, our group found some stability, i.e. we had reached a good “velocity”, as we call it in agile practitioner slang. Velocity is an important value, and a determining factor in our ongoing journey towards agile telemetry. 

To quickly (re)define it, velocity is the number of “effort points” the team is able to accomplish during a sprint. This key indicator allowed us to assess how much work our team could deliver over a given period of time.  

Velocity and effort points in our Managed Kubernetes project

Let’s look at a simplified schema of how this method worked in our Managed Kubernetes project: 

Here, we have three main steps. For each of them, we collected its quantitative values, to get a consolidated view of each step’s velocity, in addition to a global view of the whole project. With these velocities, we got a good view of the project’s advancement, and some visibility on the deadlines.

Example :

This agile model allowed us to achieve our goal and deliver the Managed Kubernetes project to our customers on time, and this success subsequently led us to share these good practices with all teams in the Product Unit.

But a second challenge presented itself, as maintaining this process required a lot of discipline, with numerous opportunities for mistakes to be made. And so our next step was to create a tool capable of reducing the risk of error, while maintaining full visibility of projects at both the macro and micro levels. This is what we will present in detail in a future blog post: Agile telemetry by OVHcloud.