Podcast Series: Customer Success Investigators - Compuware

Customer Success Investigators Case 1: Remote Workforce Performance Metrics

Customer Success Investigators Case 1: Remote Workforce Performance Metrics

Send Us Feedback

Listen on Itunes

Listen to Stitcher

Craig: For our first session, we’ve got Atul Bhovan and Bill Mackey from our business to come and tell us how they’ve investigated new stories this week and also David Rizzo as to how he’s keeping the cogs whirring at Compuware as we’re in this lockdown period. So, grab yourself a snack and sit down for the first Compuware CSI session.

Bill, nice to have you along today. Could you give us a bit of an introduction as to your role and how long you’ve been at Compuware?

Bill: Yeah, thanks Craig, and I appreciate you having me. So, I’ve been at Compuware almost 30 years, it’s actually 30 years in June. Long time, and I started off in the file and data management group as an SE; managed that group for a while, and I went into what’s kind of known as an SME role and finally became a product manager about five or six years ago.

So, I’ve been, product management since then and I’m currently the product owner of zAdviser.

Craig: Well, Bill, I’d like to just put that on pause a second, as we say hello to Atul, because I want to pick you up on that, but Atul Bhovan, please say hello, introduce yourself to our listeners.

Atul: Hi, everyone, Atul Bhovan, as Craig mentioned. So, I’ve been with Compuware for over 20 years, now. I’ve been working with various different customers around Europe and South Africa and it’s all been, really, around modernizing the mainframe platform and working practices. So, I’ve learned a lot and I’ve been able to share a lot with them, over time, as well. So, it’s an ongoing and exciting journey. So, I’m happy to be here and have a conversation with both Bill and Craig. So, Craig, thanks for having me.

Craig: No problem, great to have you both along. So, Bill, we’re going to be speaking to David Rizzo very soon. But I’d like to just put a bit of a marker down as to how zAdviser has evolved, and what it is and how we’re using that to help measure and manage our customers’ environments. Could you give us a bit of a potted history on zAdviser and how it emerged?

Bill: Great, yeah, it’s a very interesting story. So, Compuware has had a program known as the VIP program for about 17 years, “VIP” stood for Value Improvement Program, and it served to show our customers the value they were getting from the Compuware tools. So, that’s been around for a long time. It started off as an Excel spreadsheet and then it moved to a BIRT reporting tool. Finally, we decided to get into the BI space and we went to Qlik and this was all done kind of outside the internal Compuware, product management, engineering staff by a group of folks over there, in the UK who worked on this Qlik application, and that group was headed by Elizabeth and a bunch of folks from over there, and they did a wonderful job building out the application, and it became a very powerful tool for our customers and more and more customers kept coming on board, sending us more and more data and eventually basically it, the Qlik tool, was kind of overrun with the amount of volume, and amount of data and it was not performing well enough for us to give it to our customers.

So, it had some problems that we couldn’t overcome. So, at some point we decided, and this was probably five or six years ago, to bring the VIP program in-house and start fixing some of the things with the engineering team, and put a product management team behind it and take it away from the external part of Compuware and bring it internal, where we had more control over it.

So, once we started doing that, we started interviewing the field and we found out all of the things that were that needed to be fixed and around the same time… This is kind of where the DevOps craze kinda started, and this was more of a DevOps on the distributed side of our house and there was something called KPIs, was becoming the range. Well, [Compuware CEO] Chris O’Malley recognized this and decided that we really needed to take the VIP program in that direction as well and become the DevOps company for the mainframe, and really focus on mainframe KPIs and it just made sense to take our VIP program and turn into something more. That’s kind of how zAdviser was born. Chris O’Malley wanted us to figure out how to help our mainframe customers know what good looks like and become better-performing development organizations.

Chris liked to tell the story about the Icelandic baseball team. And all they did was practice because they were the only team in the country. They practice, practice, practice. So, he likes to say they had no real comparison to see how they knew they were getting better because they never played anybody. So, the thought was, we could get a lot of information from a lot of companies, aggregate that information and then give the aggregated information back to the whole mainframe community so that they started to get a feel for what good looks like.

Well, if we had a feel for how long that’s taken for the entire industry, we could start to compare ourselves. And when we start to measure ourselves, we can start to improve ourselves. So, if I know that my team’s taking 10 days to fix a bug and then find out that on average the mainframe community is taking eight days, well, maybe there’s patterns of behaviors that I’m not doing that I could do to improve. And that’s what zAdviser is going to do for our customers.

So now, with zAdviser, not only do we have the usage data that we’ve always had and we get KPIs from that, but we’ve also incorporated SCM data from tools like ISPW, ChangeMan, Endevor as well as ITSM data—tools like JIRA, ServiceNow and Rally. When we combine all that data, we can come up with some really, really nice KPIs, and we’ve got about 10 to 12 of those KPIs that are critical for our mainframe customers to, to measure so that they can improve on. So, that’s kind of a history.

Craig: Thanks Bill. Let’s hear from the man, himself. With no further ado, I’d like to introduce David Rizzo. David, good afternoon.

David: Good afternoon, Craig.

Craig: David just for our listeners, we would very much like for you to do a bit of a background check and introduction as to you and your role.

David: Sure, it’s great to be here to join you. So, I lead the engineering organization for Compuware, which means that I am responsible for maintenance, new development for the complete set of Compuware products in the Compuware portfolio.

Craig: So, have you been around for long?

David: I have been with Compuware… as we prepare for our 23rd quarterly release, I just completed 23 years with Compuware, ironically, last week.

Craig: Well, I do hope that wasn’t one release a year, David. That’s not really where we’re going, right?

David: No.

Craig: Yeah, good, good. And whereabouts in the world are you hunkered down, David?

David: I’m in the Detroit area, based out of our Detroit office, and live not too far from there.

Craig: So, you’ve been with Compuware for many years, and obviously we’ve got our HQ in Detroit. Have you ever seen such a situation where you’ve been away from the office in such a way?

David: Certainly not. I think myself like, most everyone, has not. We’re finishing up our seventh week of everybody working outside of the office and the longest I’ve ever been away from the office was 10 business days over the course of 23 years, and that only happened twice. It’s a little odd not to be in the office and see everyone, but we are making the accommodations.

Craig: Good, good. Well, that’s what we want to talk to you about today, and let’s get into some of these questions. And as we would like to understand how Compuware has maintained its cadence and given that it’s had to execute on it business continuity plans in response to COVID-19, its still on track to deliver its 23rd quarterly release. I hasten to add, quarterly, not annual release. That’s an impressive number by anybody’s standards. So, how have you managed to make that transition to a remote workforce, while maintaining delivery cadence?

David: Well, one of the things that we have always found that has contributed to our success is our ability to collaborate, and how we do collaboration and have done it for several years, is by use of tools and technology. So, even though we are co-located in Detroit and everyone has been in the office prior to this situation, we’ve always used the tools to be able to track and share information with people across the organization, so we use things like JIRA, where we’re tracking everything that we do. We communicate, we put information in there and we’ve done that. So, the way that we’ve been collaborating for years has set us up that although it’s much better face-to-face communication, we have tools like Teams, and the online video and audio capabilities, so we can still have those engagements. We can share screens to work with each other, and we have tools and process and procedure in place for transparency across the organization that lends itself to being able to share that information, no matter where you are. So, because of how we have worked and how we have set ourselves up to be able to do that, our work location, we have not suffered setbacks because of that. It’s actually, it prepared us to be able to do that.

Craig: That’s great. And so, maximizing the use of those tools and of course measuring the output from your team becomes I guess, a disparate task, but ultimately measuring their activity and tracking towards the release is going to be really important. So, why is measurement so important, during this and indeed, any other development cycle?

David: Well, certainly, it’s been said many times, and we all know, that if you can’t measure it, you can’t improve it. And something at Compuware that we look at is continuous improvement, and we use metrics very heavily and have for several years to look at how the development process is working and how much we’re accomplishing, and how we can do that to maximize our quality efficiency and velocity so that we can exceed our customer expectations and meet the need to the market. So, it’s always important to be able to measure and look at how things are going. Generally, again, when you have a co-located office, it’s very good to be able to see, and you feel that you can see what’s happening and you see how conversations are going and people that are talking and interacting, and you feel that, you get that sense of how things are. When you don’t have that, now you have to rely a little bit more on numbers and a little bit more on metrics to make sure those side conversations are happening and those things are going on by talking to people and we carry those conversations on and see what people are doing. But also, it’s good to have metrics to see how much they’re really accomplishing, how much is happening to get that comfort that things are moving along.

Craig: Sure, and you can’t achieve 23 successful quarterly releases without having some key metrics that you look into to help drive that, as you mentioned efficiency, quality, and velocity. So, would you be able to share with us your top three metrics that you consider the ones that really kind of give you insight into how your development teams doing?

David: Certainly. The metrics that we use and we’ve again, looked at for a long time, one of the top metrics we look at is our MTTR, mean time to resolution, for issues… external customer bugs. So, how long does it take us to resolve a bug? And so that gives you an indication of how we’re satisfying our customer, needs, but also how we’re working. So, that continues to be a key metric for us. From an efficiency standpoint we look at, are we working on the right things are we doing the things that we need to be doing?

And so, we look at how our work distribution is working, are we working on new enhancements, are we working on technical debt, are we working on bugs? We have to do all of those, so we look at how those things are… the distribution of the work to make sure that is even across those types of work. So that’s what we look at. And also, when we talk about velocity, we just generally look at how many issues are we resolving, how many things are we moving forward with, and that gives us a sense of, we know historically, where we’re at—we know what a team’s velocity generally should be, based on availability, and we can compare that, so we know that we’re working on the right things at the right speed, and with the right quality and meeting our customer expectations.

Craig: Cool, that’s great insight. And collecting those metrics is only part of the challenge, so making sense of those metrics and taking action on them really is what gets you the value. Can you help us understand some of the ways you use that telemetry to actually take action?

David: Again, one of the good things for us is we’ve been doing this for quite a while, now. So, these aren’t new numbers that are coming at us so when we look at the numbers we’re able to extrapolate some good information. And when you talk about productivity, which is a concern always, are we working on the right things and doing enough of the right things so we can take these numbers and look at the balance of the work we’re doing. We know how long it’s taken us so we look at the length of time from when we start something to when it’s done gives us an indication of how productive we’re being and how things are moving forward and advancing. So, we can look at each number and then we can drill into… when we see maybe our MTTR has changed. Has it gone up or gone down? In both ways we always look, but if the MTTR has gone up, we can then dig into our JIRA issues and look at… Was there something specific or was a specific issue or do we see just overall, things are taking longer, and drill into those. In some cases, you can find, because we can drill down through the numbers to look and see, perhaps, you’ll be able to recognize an individual that’s struggling on something, having challenges and may be reluctant to reach out. And so, by using those numbers, you can proactively reach out to an individual who may be struggling in a way to adjust to the new way of working, and you can help them and the team can work together to make everybody productive. So, it gives you a way to look at how things are going from high-level business and then you can drill down into individuals and be proactive. We identify a potential issue, which is everybody gets up in the morning and wants to work hard and deliver and some people are challenged based on the situations that they’re experiencing.

Craig: And I guess in these tough times, there’s a new way of incentivizing and I guess a new way of monitoring these people. So, I guess there’s a carrot-and-stick kind of mentality. But ultimately, that has to be ways of engaging with the community.

As you are our first guest, David, you’re going to kick us off with this one. But I’m going to be asking each week our guests for recommended reading on the topic and when I say recommended reading, it doesn’t have to be a 30-chapter book, it could be a pamphlet, could be something that you’ve help create, even. But have you got any suggested reading for our community to help with metrics and understanding the output of metrics?

David: Yeah, depending on where you’re at with metrics, if you’re new to metrics and you haven’t had them in place before… Again, we’ve been using these and we’ve had the advantage of being able to do them. One of the things that I recommend, one of the books, is actually Mik Kersten’s book, Project to Product, which talks about workflows and value streams and understanding what to expect, and I think it’s a good… It’s a good book to read to make sure that you understand what the metrics will be telling you. Because it’s one thing to have the metrics, but really understanding what you should be looking at, because numbers by themselves can be misleading… in a good way, in a bad way if you don’t understand the system with which they fit in and how you want to move those numbers, and what they should look like. So, understanding how your workflow is set up, and what you should expect and again, as we are in changing times and might be working in different ways, making sure you understand what the expectation should be before you start actually looking at those deep dive numbers.

Craig: Anything from Compuware that you recommend?

David: We have a lot from Compuware. Certainly our zAdviser product and is out there that has metrics on it, and we have a lot of good information of how to look at measuring and we do a lot of blog posts and certainly Compuware.com, you can find a lot of good information based on where you’re at in your journey and what your individual needs might be.

Craig: We’ll post some of those in the episode notes so that people can get to them really quickly. But finally, then, David, I think one of the things we would all like to know is how you’re keeping morale up with your team. I presume your developer community is spread all around the US?

David:We are basically based out of Detroit, so our developers are pretty geographically condensed. But today being five miles apart seems like being a world away.

Craig: Absolutely, so, you’ll have similar taste in sport that obviously isn’t going on, you’ll have similar taste in local cuisine, which obviously isn’t going on so, so how do you keep the moral up, how do you keep people smiling behind their desks? Because it’s very easy to go stir-crazy in this time.

David: It absolutely is, and it’s very challenging. As we do, a couple of things we’ve done is you talk about sports, we have groups of people that Friday go out to lunch normally in normal times, and the lunch conversation is what are the sporting events that going on, what are the activities going on for the weekend, what family events are you going to? And none of that’s happening. So, the first couple weeks we had the virtual happy hours, and the virtual lunches, and we didn’t have anything to talk about we were trying to figure it out.

And people have come up with some fun different things, but one of the things that… One of the teams did that caught on a little bit is having their pets on camera as the focus of the camera during a meeting rather than themselves. So that people can see, get introduced to their pets, and do something different rather than just a standard looking at each other, or things like that. So, it gets people in, it gives some insight into people’s personal side.

Craig: It’s a nice touch, but I do recall a presenter as saying it should never work with the kids and animals right? As long as it keeps those guys smiling. And, of course, turning out code for our customers, that’s the key thing, right?

David: That’s right. It keeps them happy and keeps them moving. And in some of our Teams meetings, we’ve had guest appearances by their children unexpectedly.

Craig: Invited and not invited, I’m sure. But David, thank you very much for your time. I think it’s been a great insight to how we in keep the cogs going here. A compare. Thank you very much for your time.

David: Thank you.

Craig: So, there we have David Rizzo, keeping service levels and spirits high. Atul, I’d really like your thoughts on David’s key metrics there.

Atul: Thanks, Craig. One of the key things that David mentioned was around MTTR—the ability to identify individuals in the team who might be struggling and they may be a little too shy to ask questions or to ask for help. And we’ve all been there, we don’t want to be the person asking the stupid questions or silly questions. But it’s important that the teams can be proactive in helping those people. And I think this is quite often a cultural issue as well, because we do find as we’re working remotely, we want to make sure we maintain a good level within our team without being seen to be somebody who doesn’t quite understand. So, things like putting in place better ways of educating, training people, documentation and mentoring—those are some key outputs I think we can get from that metric. And I think we’ve all been there, in our careers in the past. So that was, I think, one of the biggest things that came out for me as a great example to share today.

Bill: Yeah, I agree. And he made, David made some great points in there. But one of the measurements, I’ve seen using zAdviser was where we can actually look at patterns of behavior of different developers. I saw a visualization where we took a very high performing 20-plus year developer, we had a five-year developer and then a new person and we kind of showed the patterns of behavior for fixing a similar bug type and it was fascinating to see the differences in the patterns of their behaviors and how much re-work they had to do, how much debugging they had to do and the time it took to do it. So, when we saw those patterns of behavior, we were able to make some improvements to the newer person and make them a better developer, so it was kind of a very interesting to see that kind of information.

Craig: And also, we heard there from David around innovation and how that’s kind of critical. Bill give us your view on that, and how this is an important metric for us to monitor from a KPI perspective.

Bill: Yeah, David spoke to that about work distribution and just to expand on it a little bit… So, there’s a lot of give and take between the engineering teams and product management when it comes to deciding what work gets done. Compuware strives to give new innovation to our customers’ quarterly, and as an aside, we’re about to do that for our 23rd consecutive quarter which we’re very proud of… And the way that works is product management decides on priority and then engineering actually does the work. But sometimes things happen during a quarter. For example, product management might approach David Rizzo in engineering and say to him, “We’ve got these two customer needs that have to get put in mid-sprint, we’ve got to get this done. It’s all new innovation work.”

And David then has to talk with Sam and say, “Okay we can shift our focus away from maybe doing bug fixing and maintenance and put some of those resources on this new innovation.” And we know from looking at zAdviser that our sweet spot for doing innovation is to we need to have about 75 to 80% of our work staff working on new innovation and somewhere in that 25% range working on bug fixes and maintenance, and that works really well for us. We get the bugs fixed, we get the maintenance done, and we get the new innovation out. But when we have a disruption there where we’ve got to get some new innovation in that we didn’t necessarily plan for, which happens occasionally, then we have to shift resources and when we do that, there’s a cost. So, David and Sam can look at the hard numbers and say okay, if we go to 100 percent innovation for the next two weeks, to get these new things in, this is what’s going to happen the following two weeks, when we’ve got to make up for that work that we didn’t get done. And they can see that our drop in innovation might go down to 50% because we have to put a lot more resources on maintenance and bug fixes. So, having those metrics, having those hard numbers allows David and Sam to make some very strategic decisions.

Craig: Thanks Bill, that’s awesome. And so, Atul I think what we’d like to do each week, is work out how zAdviser is helping some of our customers. So, from an investigator’s perspective, could you give us a real-life example as to how zAdviser is helping some of our customers?

Atul: Yes, this is quite exciting for me, actually. We’ve been having quite a few discussions with some of my customers, both in the UK and around Europe as well. And one the biggest things that when I speak to systems programmers about is around how we can identify the product releases that have been installed in the various different environments. And what this gives them is the ability to see if there are old releases of products lying around, or old libraries, then, they can address that because for auditors and for compliance teams, this is quite a big thing because it does identify bottlenecks and potential areas for data breaches in the future as well, so it’s very important they address that.

But then, other aspects when I speak to production teams, they are identifying abends that are occurring and these are invisible abends that things are breaking or falling over behind the scenes. And for a consumer, or example, if I’m an external consumer, I don’t always want to report it to the helpdesk. So, for example, we identified at one instance 18,000 abends of the same type occurring over fixed period of time. And the fact that this customer was able to address that meant that they weren’t going to be giving their customers a bad experience and potentially losing customers as well. So, I think that was a big win for them.

And then moving on, just to a couple more examples, one was around the Compuware product usage, understanding how well the various different capabilities of the tools are being utilized. That really does give them the ability to identify ways of being proactive in providing training, providing documentation and other forms of support for their team to make sure they are as productive as possible. And in terms of productivity, that is a major area for anybody who is in charge of ensuring that capabilities of their teams are at their highest. Especially now, they’re focused on innovation, so being able to see productivity metrics and specifically, once we’ve done the value stream mapping exercise with them, which is a whole other topic, really, I think you find where the constraints are and are we addressing those constraints enough to make sure that our productivity levels are going up all the time?

So, we’ve had some great conversations, and I sure we’ll have many more as well. And I do invite our customers to contact us and let’s explore some of these topics together. It’s a really exciting area.

Craig: That’s great, Atul, I really appreciate the feedback and the insight there. And we also heard from David around, he suggested reading and his blogs. Bill, Atul, you want to kick that around a little bit?

Bill: Yeah, I’ll kinda kick that off. David mentioned Project to Product. All of product management at Compuware and all of engineering have read this book and we’ve found it invaluable. So, besides that book, the other book that I’ve really enjoyed is The Phoenix Project by Gene Kim, which is more of an easy read, it’s more of a story, but it really kind of highlights the whole DevOps process, which I thought was great reading so….

Atul: Bill, let me ask you, when you’re reading The Phoenix Project do you feel a bit like that’s part of your story, you’ve been through that, yourself?

Bill: Exactly! You can almost put yourself in some of those positions either as a manager or the person doing the work. We sometimes we get bogged down and there’s that one person you’ve always got to get their time to help you, or you’re inefficient. And The Phoenix Project just highlighted that perfectly.

Atul: Exactly, exactly. I hear that so much from many of the people I speak to, as well, about that. But I can just comment a bit on the Project to Product, as well? So, I think Mik and his team have done a great job in researching the subject matter and I can tell you for any customer of ours who’s actually going through a transformation, and there’s so much change, both cultural and technical and process-related, there are so many lessons to learn from the book itself, they share quite clearly what to expect in their journey. So, it’s a great heads-up in that sense, but also it does help us to avoid any pitfalls that may be coming our way, as well. So, again, it’s a great resource and there are some great reading materials out there. I’m sure your future guests, Craig, will be talking about those in the future as well. So, that for sharing that because that was really exciting.

Craig: So, on top of the books guys, David mentioned the blogs that we produce ourselves, and get pretty good press around.

Atul: I really do think that the blog posts that we provide are a treasure trove. They are very easy reading, first of all, and they’ve been written by very smart people and they’ve been very generous to actually share their experiences with us. And their experiences come from both internal within Compuware and from the field, so it is seen as a valuable gift, and I would advise everyone to take a little time to explore some of the posts we’ve got there. There are also related material links in the blog posts, but also they should feel free to reach out to the authors, as well. They’re all available on LinkedIn. Feel free to connect with them and they would be happy to share more ideas as well. So, it’s a great resource and please go and use that.

Bill: Yeah, I agree. Compuware professionals are putting up blog posts weekly. One of the most recent ones that I thought was excellent was one done by David Kennedy. So, if you get a chance, I highly recommend going at the LinkedIn looking up David Kennedy and reading the blog post that he just recently did. I think you’ll find it fascinating.

Craig: Will do, and we’ll get some of those blog posts posted into the show notes. And finally, gents, before we move on, the comments there from David around having animals do their team calls. Have you had any incidents of crazy things happening during these multiple calls that were having?

Atul: If I could go first. This is so embarrassing. So, in our team meetings, we do often bring with us various different animals, but in the form of puppets, for example. So, they may be animals from the Muppet Show, they may be animals from the Sesame Street, characters from there. And Craig, I know you will take part in these kind of debates, as well. So, do you have an example of what creature you shared within your team?

Craig: Well, well, having children myself, obviously, I’ve got lots of things lying around in the house we could bring to the table. So, I’m often accompanied by Elmo just to spice the things up a little bit. But we’ve had a little bit of fun with that. Bill, have you had Elmo having a conversation with you at all?

Bill: Maybe not Elmo. We’ve definitely had family members coming in and out of our images behind us with our cameras on… So, you see that pretty much on a daily basis, and it kind of makes it fun.

Craig: Yeah, so, we’re growing the Compuware family by virtue of all being locked down at home, so it’s a great way of recruiting, right?

Bill: Yeah. It’s also interesting sharing bandwidth with about five different people in your house when you’re trying to do a webinar or a webcast like this.

Craig: Bandwidth has become a premium. But guys, you sounded fantastic. Thank you for your time, much appreciated. If you could now go back down, into your lockdown caves and get back to delivering to our wonderful customers that would be great. But thanks Atul, thanks Bill, for joining us today.

Bill: Thanks, Craig.

Atul: Thank you, Craig.

Craig: So, there you have it, episode one of Compuware CSI. Thanks to Atul, Bill and of course David and for you, the listeners, to participate in our little journey. We hope to see you next week where we’ve got a special guest coming in from industry and we’ll talk a little bit more about our journey through COVID-19 and this lockdown period. Thank you, my name is Craig Hartwell. Good night.