Cloud Hosting

All the sessions from Transform 2021 are available on-demand now. Watch now.

This post was written by Rob Zuber, CTO of CircleCI

Ive been leading software teams for more than 20 years and one thing Ive learned about metrics is that leaders tend to put too much emphasis on engineering metrics alone, without considering the bigger picture.

After speaking to a range of engineering industry leaders, and poring over millions of jobs processed from software teams worldwide, we found that the most insightful and relevant metrics fall into three categories:

Engineering velocity metrics measure the speed and efficiency of software delivery pipelines its the metric category that managers typically pay the most attention to. While Ill explain why its not the only important category to track, velocity metrics are critical in helping teams identify slowdowns and find ways to optimize their overall performance.

Some of the most common velocity metrics include:

Moses Mendoza, former Head of Engineering at data processing and review platform Zapproved, uses throughput to understand the pace of his teams work.

Throughput helps us identify and understand speed but the throughput of a system is also bound by its primary constraint, Mendoza said. Throughput will show you what the slowest issue is in a chain of events, but it wont show you how to fix it to speed up your work.

Graeme Harvey, an engineering manager on my team, emphasizes that its important for all engineers to customize throughput measurement to their individual team.

Because our team practices pair-programming, measuring throughput isnt something that can be tied to an individuals productivity, Harvey said.

When it comes to throughput, his engineers optimize for the team rather than the individual. Pair-programming and helping each other might feel like its impeding the progress of an individual but in actuality, it refocuses energy on whats most important for the team and ultimately the business.

While throughput is a valuable metric that helps you track output, there is no one-size-fits-all way to measure it. Measuring throughput accurately requires you to evaluate the structure of your team and how you work.

According to Alex Bilmes, former VP of Growth at software configuration tool Puppet, there are two ways to measure change lead time. One way to measure change lead time is to look at how long it takes to get an idea out and for the idea to go full cycle. The other way is to look at deployment lead time, which measures how long it takes to get to production after a developer has pushed the change to production.

Full change lead time will point out issues in communication and understanding, as well as the depth of your backlog. Deployment lead time is more likely to show the quality of your pipelines and tooling.

Sprint velocity measures the amount of work a team can tackle during a single sprint and can be used for planning and measuring team performance.

Tom Forlini, CTO at video conferencing platform Livestorm, dives even deeper when measuring velocity, focusing on three smaller metrics:

Livestorm engineers work on two-week sprints and have 50 story points per sprint, Forlini said. We track the number of issues done vs. planned because it gives us a good indication of the sprint planning quality between Product and Tech.

Then, his team looks at the percentage of issues by type. When a sprint contains only new feature issues, we know from the start that it might be quite a challenging sprint to tackle, Forlini added. Ideally, you should balance the type of issues by sprint as much as possible.

Morale metrics are probably the most overlooked metric category in engineering. They tell you how engineers feel about the quality of their work and their job happiness, which is a major retention factor. Keeping retention high means keeping morale high.

Some common morale metrics include:

At Zapproved, Mendoza tracked morale in order to monitor employee retention. We measured morale at work using surveys, having conversations, and asking managers to dive deeper in one-on-one meetings to find out how employees felt.

If responses to a survey are overwhelmingly positive, youll want to know what is working and how to replicate that positive work environment. Similarly, if responses are negative, its helpful to find out directly from your team why they feel that way and what you can do to fix the problem.

Mendoza at Zapproved measured confidence by reviewing every sprint in conjunction with that teams manager and their scrum master. As we measured code quality confidence over two or three sprints, if we saw code quality tanking, it meant something was wrong with how the teams planned their individual investment with the work, says Mendoza.

The engineering managers that I lead also measure work by confidence.

Focusing on confidence over coverage as a metric requires that the emphasis isnt on code coverage, Harvey added. Its critical to break the reliance on having 80% or 90% code coverage and then shipping it only to find out the code is broken. Test coverage is a partial proxy for code confidence. If you know 95% of your code is fully tested, versus 20%, then youre going to feel pretty confident that if your tests pass, your code is legitimate.

Harveys team focuses on delivering small iterations quickly. This provides the confidence that the team is building something of quality, nothing is broken, and theyve made the right choices in building tools for the dashboard.

Everything an engineer does should propel the company forward. Thats why its also essential to track business metrics.

Some common business metrics include:

Tracking business metrics is how your team accommodates for user growth effectively. According to Yixin Zhu, formerly of Uber, while its essential to look at engineering execution metrics, its also important to be dialed into the businesss goals and to measure the company growth.

As Uber grew exponentially, tracking business metrics was incredibly important in order for Zhus engineering team to succeed. When youre talking about doubling every six months, you have to be tracking that to know what you need to build, what degradations to expect, how many data centers you need, how many boxes, etc., Zhu said.

In short, engineers have to keep an eye on real-time business metrics to project and plan accurately. You have to be proactive, Zhu added.

Here are some tips to help you get the most value out of your engineering efforts:

Rob Zuber (@z00b) is the CTO of CircleCI. He leads a team of 250+ engineers who are distributed around the globe. He is a three-time founder and five-time CTO, currently living in Oakland, CA with his family.

Go here to see the original:

Use these metrics to get the most out of your engineering team - VentureBeat