It’s Time for Mainframe Teams to Adopt Automated Unit Testing
Overview: For the third episode of our series “10 Steps to True Mainframe Agility” on The Modern Mainframe podcast, we spoke with Compuware testing expert Glenn Everitt about the third step, Adopt Automated Unit Testing. Here are some highlights from the conversation.
You’ve taken the first step to define your current and desired state for your mainframe team. You’ve begun modernizing your mainframe development environment by adapting your culture and implementing an IDE. If you’re following Compuware’s 10-step model for achieving mainframe agility, the third phase of your journey will be to adopt automated unit testing.
As Compuware testing expert Glenn Everitt explained in the third episode of our series “10 Steps to True Mainframe Agility” on The Modern Mainframe podcast, “I think it’s one of the most important steps, because it’s where people spend almost the largest percentage of their time.”
“I was at a customer site last week. They said they spent 60% of their time just testing,” Glenn said.
That’s because unit testing on the mainframe is still a manual process for most organizations—ratcheting through a spreadsheet of unit tests, setting up test environments and data, verifying the data, prioritizing the data, and so on.
“Anything you can do to automate that step really makes a huge difference in how quickly you can get something delivered,” Glenn said.
Why Adopt Automating Unit Testing?
We’ve seen this “huge difference” made for several customers who chose to adopt automated unit testing.
One customer explained to Glenn during a visit how much time they had recently saved by having tests ready to go when some changes came out, which forced them to re-do all of their testing: “I could change a couple of data values and some of my data’s input stub, and I was able to test these new cases that I needed to validate really quickly,” the customer said.
Without this automation, the customer said they would have had to spend a month reworking their tests. Through automation, they reduced it to one day.
Another customer, a large UK bank, is using automated unit testing to successfully “shift left” so that testing takes place in parallel to coding, not at the end of a sprint where there’s less time to ensure quality. Ultimately, replacing manual processes with automated unit testing has helped the bank reduce execution time from two weeks to five minutes.
During our conversation, Glenn gave some key reasons for why companies should adopt automated unit testing to increase development quality, velocity and efficiency.
One of the key things your tool should help with when you adopt automated unit testing is easing the management of test data. For example, Glenn says your automated testing tool should create data stubs that allow you to stub out calls to I/O routines, and it should make it easy to automatically capture data and set up tests for you.
“It really eliminates a lot of that tedious setup work, particularly when you re-execute that test. When you re-execute that test and you’re reusing the test data that you’ve already captured, you don’t have to set it up every time. So, again, there are multiple levels of time-saving you get,” he said.
Test Suites, Test Cases
The tool you use to automate unit testing should also be capable of organizing test scenarios into test suites. With this, you can ensure your tests include all the test cases you want to execute.
“You can group these things into logical blocks that make sense to you and to how you want to organize your tests … There’s a lot of value in being able to guarantee that you actually have a consistent set of test cases you’re running,” Glenn said.
Glenn says having a tool that helps you run a consistent set of test cases is also critical to helping you compare code coverage to ensure you’re testing as much as you think you are.
“It’s one thing to go through and automate unit testing, but if you really have no idea of the percentage of code that you’re actually validating with that, you really haven’t reduced the risk of delivering that code,” he said.
“If you have tests that test the same thing a hundred times, that’s not particularly useful. If you have one test that tests 100 things, that’s a much a better approach.”
Tools Aren’t Enough
One of the other important points Glenn made during our conversation on The Modern Mainframe is that putting a tool in place to support automated unit testing isn’t enough. Ultimately, you need to adjust your processes and, most importantly, help your mainframe culture adapt if you want to reap the benefits of modern technology and automation.
Listen to my interview with Glenn for a better understanding of this. To read more about Step Three and the other nine steps to achieving true mainframe agility, download our eBook “10 Steps to True Mainframe Agility.” Otherwise, keep an eye out for future episodes in this series where we’ll explore the other steps with Compuware experts.