Compuware | Mainframe DevOps
October 9, 2018 DevOps

The DevOps Balancing Act of Measurement and Reward

Overview: Learn why simply measuring things won’t give you the outcomes you need. You also have to consider why behaviors are currently being rewarded across your teams and how that aligns with what you measure.


You can’t improve what you don’t measure; but “What you measure and reward is what you’re going to get,” Gordon Bethune, the acclaimed CEO who transformed Continental Airlines, said.

To his first point, as your organization moves through its DevOps journey, transformation leaders should be identifying quality, velocity and efficiency KPIs and metrics for teams to continuously improve against. We lay out several of these metrics in our new eBook, “Metrics Worth Measuring: Charting Your Mainframe DevOps KPIs,” which is the third eBook in our “Achieving DevOps Guidebook Series.”

Read the eBook

But to Bethune’s point, simply measuring things won’t give you the outcomes you need. You also have to consider why behaviors are currently being rewarded across your teams and how that aligns with what you measure. This requires yet another mindset shift under DevOps.

Compuware Product Manager Mark Schettenhelm wrote about this in a column for Enterprise Executive:

“How an organization responds to the accomplishments or failures of its people sets the standard for how work should be done. If you can’t change that, you can’t change your culture, and it won’t matter what piece of technology [or KPIs] you introduce.”

A great example Mark gives is how mainframe teams have traditionally rewarded developers who understand complex applications, yet they do nothing to make those applications easier for others to understand. Instead, we should be lauding the behavior of those who “remove dead code while making a change, who break apart complex logic” so others who don’t understand complex applications can.

Measuring the right things matters; but the behaviors you reward matters too. When your teams aren’t performing as well as they should be against the KPIs you set in place, look to reasons for why you reward or correct them; the answers will lie there.

Leverage KPI Combos

You need to consider that, under DevOps, many KPIs aren’t specific to development or operations, but are the responsibility of both areas.

Your pre-DevOps methodology for measuring KPIs probably exposed conflicting goals between teams. You would measure development velocity and measure production quality separately, inevitably putting the goals of both at odds. Under DevOps, you might still measure individual KPIs in some cases, but behaviors should be rewarded to encourage the achievement of mutual goals.

Create your own user feedback survey

One example might be to measure the number of software releases for a given period and factor that against the number of defects not detected in testing. Perhaps your goal is for six releases per quarter, but you also want to limit your production exposure to no more than five defects per release.

In a pre-DevOps scenario, perhaps you would see developers increase code drops, but testing would fall behind or be left until the end, inevitably diminishing quality. Under DevOps, your dev and test teams would need to meet or beat that goal together to achieve success.

Particularly in the testing phases, you want testing engineers to work efficiently, but without sacrificing completeness. This means rating them on their cycle time balanced out by the number of bugs that made it to production.

Setting Targets and Goals

If you are starting from scratch with DevOps and have a large gap between where you are and where you want to be, consider creating a series of goals, each increasingly challenging. You might want 10 code drops per quarter, but if you’re moving from a waterfall methodology with one massive release per year, maybe you should start with two code drops per quarter. As the team develops a way to do this while keeping production stable, you can set the next target.

Goals deemed unachievable will be incredibly demotivating. Remember to pair incremental targets with celebration. Recognition of shared success will help build the behaviors DevOps needs to thrive.

Making continuous improvements and increasingly setting the bar higher sounds challenging. But with a service like Compuware zAdviser, you can automate a lot of the measurement work.

zAdviser uses machine learning to find correlations between developer behaviors and KPIs based on DevOps toolchain data and Compuware product-usage data. And it’s free to Compuware customers on current maintenance.

Compuware zAdviser | DevOps | KPIs | Elastic | Kibana

Compuware zAdviser’s highly intuitive and interactive dashboards are built on Elastic’s Elastic Cloud service and Kibana

DevOps is essential to organizational success in a digital age, but it’s not just a methodology you implement. DevOps is a journey that requires a mindset shift in what you measure and in behaviors you reward to help your teams meet critical KPIs that impact business outcomes.

IT professionals will find personal and professional rewards beyond those you give them when their silos are broken down and collaboration in the name of achieving mutual goals powers their day.