Two Revealing Studies on Mainframe Testing Practices
Overview: Traditional, manual mainframe testing methods–still prevalent in many large organizations–are draining resources, reducing code quality, and delaying the delivery of critical functionality. By contrast, a leading UK bank has brilliantly bucked this trend by automating mainframe testing–and is seeing big dividends. Read the research reports that shed light on these two divergent realities.
The world’s biggest and most successful enterprises continue to run their mission-critical workloads on the mainframe. According to the BMC 2019 Mainframe Survey, 59 percent of executives believe the mainframe will grow and attract new workloads.
Companies remain committed to the platform due to its unmatched digital strengths in reliability, performance, security and transactional efficiency, as well as its decades of proven intellectual property in the form of fine-tuned business logic and data.
But mainframe development faces strong headwinds.
Increasing workloads and a growing shortage of skilled professionals are increasing the need to ensure code has been thoroughly tested, which can put it at odds with relentless customer demands for better, faster–now. The good news is the shift to mainframe Agile and DevOps has improved the pace of software innovations delivery, but deep gaps remain.
For example, testing within mainframe development is still largely manual, and together with the pressure of shortened lead times, is delaying software delivery and threatening quality. Further, test automation coupled with a “shift-left” approach—where developers write unit tests at the same time as they write source code—is not widely embraced for mainframe application development.
Global Survey of Testing Practices
Compuware conducted a recent independent global study of application development leaders to examine in more detail the processes that organizations have in place to support testing on the mainframe, while exploring the challenges they face in simultaneously increasing quality, velocity and efficiency. A resulting report shows that unsurprisingly, traditional manual testing swallows up skilled developers’ time and in turn puts pressure on teams to take shortcuts. Ultimately, it shows that the current manual approach to testing is often difficult, costly, and time-consuming, and highlights the need to increase the use of automation to alleviate these pressures.
Key findings of the global survey of application development managers revealed:
• 77% find it increasingly difficult to simultaneously increase quality, velocity and efficiency to meet innovation goals when developing and testing mainframe application code
• 53% say the time required to conduct thorough testing is the biggest barrier to integrating the mainframe into Agile and DevOps
• 82% say that unless they can automate more test cases, they won’t be able to meet their business’ need for speed, and innovation and customer experiences will suffer
You can read all the findings in the whitepaper titled, “Test Automation: The Key to Increasing Velocity, Quality and Efficiency.”
Forrester Consulting Total Economic Impact Study
Compuware Topaz for Total Test enables developers to adopt a “shift-left” approach to testing to improve the quality, velocity and efficiency of mainframe software development and delivery. It acts as a force multiplier to the efforts of developers, allowing them to automate the creation and execution of unit, functional, integration and regression testing on the mainframe, while empowering even novice developers to validate COBOL code changes with the same speed and confidence as they can with other code.
Compuware commissioned Forrester Consulting to conduct a Total Economic Impact (TEI) study on Topaz for Total Test and examine the ROI enterprises may realize by deploying Topaz for Total Test. Forrester interviewed one team at a leading UK bank and found the bank saved over $20 million and netted a 467% return on investment.
Additionally, using Topaz for Total Test’s automated testing, the bank:
- Increased story point production from 300/release to more than 1,000
- Decreased bug leakage into production by 83%
- Reduced the time to set up unit tests by 80%
- Accelerated time to market for two critical projects
- Shortened employee ramp time to productivity
- Re-aligned staff toward new product development
Topaz for Total Test has contributed to the organization’s ability to achieve faster time to market, bringing new innovation to its customers more quickly and efficiently, while retaining confidence in the quality of the code being released.
The Criticality of Automated Testing
Testing remains a critical function of development. Where traditional, manual testing swallows up skilled developers’ time and in turn puts pressure on teams to take shortcuts, test automation speeds up the development process, helps reduce human error, improves quality and increases output and productivity. Automated testing also enables key workers to focus on more value-generating activities, such as user experience design and adding more business functionality. Enterprises that neglect their testing processes will fail to accelerate innovation to the detriment of their business.