Chapter 13: Automated Delivery II

Chapter 13: Automated Delivery II

Send Us Feedback

Listen on Itunes

Listen to Stitcher

Matt: Hello there. I’m Matt DeLaere, welcoming you to the Modern Mainframe podcast and Chapter 13 of our series, Building a Better Software Delivery Platform, in which Compuware Executive DevOps Solution Architect Rick Slade shares his years of experience to help you build a modern software delivery system. So, let’s say hi to Rick. Hey, Rick.

Rick: Matt, how are you? Good, good, doing well. And I hope you are.

Matt: Yeah, I am, thanks. So, this week we’re continuing our discussion of automated software delivery, focusing on automated testing. We talked about modern testing back in Chapter 10 with product managers Steen Brahe, who was making a return engagement. So, Rick, I’ll let you set the stage for today’s discussion and introduce our guest.

Rick: Thank you very much, Matt. And this is the second part of a two-part series on automation. We thought that testing was important enough to have its own segment. Last week with Dave Rizzo, we talked about automating the build and the deployment processes within the software delivery cycle, and testing is such a big, big part of that. I think it’s the most important part of that software delivery cycle. Testing is so critical to get right, especially on the mainframe platform where we are the keepers of our systems of record, those back-office systems that are so critical to our business. The ability to automate the work required to test those applications to make sure that they’re operating properly is just critical to the software delivery system, so I was glad that Steen accepted an invitation and come talk about testing. Steen, I’ll talk about his role, but he’s responsible for our testing product at Compuware, and we’re going to talk today about automating different facets of testing at a unit level, at a functional level, virtualized testing, non-virtualized testing. Testing, and automation of that work is so important.

So, with that we’ll get right into it. Steen, thank you for joining us again today. Maybe you could take us a few minutes, talk about, again, your role here at Compuware, and a little bit about testing automation in the marketplace in general, and then we’ll get into some questions.

Steen: Yeah, hey Rick. And thanks for getting me back. It’s really a pleasure to be here again and talk about test automation, which I’m really committed to. So, my role in Compuware is that I’m the Product Manager for Compuware’s mainframe testing automation tools, which covers Topaz for Total Test and Hiperstation.

Test Automation Today

And if we should look at what test automation on the mainframe in the marketplace today looks like, I must say at least the last 6 years, the increased interest in test automation on the mainframe has been extremely positive. Today, most mainframe companies show great interest in starting to do test automation, and most don’t have any automated testing today. Actually, a recent survey we have with 400 senior IT leaders at mainframe companies, only 7% of them, they automate their mainframe program testing. So, there’s a huge potential and a huge need on doing the test automation. And the interest is just enormous, all this approach to go into DevOps and to modernize your development environment, to development experience on the mainframe, testing is just a huge part of that. And as you say, Rick, testing, you think it’s the most important part of a modern DevOps process? I agree. Testing is where you can remove most waste in your development process.

Rick: Yeah, I totally agree, Steen. I know that for years and years—and I been in software development for almost 40 years now—and the testing for the longest time was no more than what I call traditional black box testing where you made your changes to your code and you executed that code and then you analyzed the results of that run, and that was essentially your testing. And there’s so much more that we can do and there’s so much more that we can be to help us be more efficient in that work; it consumes a lot of time. On an earlier podcast, I think I told some folks I’d run or led the application modernization practice at HP for several years, and the two biggest work activities was impact analysis on the front end of a change and testing on the back end of that change. With testing consuming the most amount of time. So, it really is critical, and the ability to automate of that work can make such a big difference.

Steen: Yeah, I completely agree.

What is Automated Testing?

Rick: Yeah. Steen, tell me a little bit about… I know you told me a little bit about automated testing, what is it from your perspective? When you use the term “automated testing,” talk to me or to the audience a little bit about what it is and the difference you think it can make in the software delivery system, based on your experience.

Steen: It’s a really good question because automated testing, you can actually look at it in several stages. So, many companies, when they go into wanting to do automated testing, they think automated testing is about the complete CI/CD pipeline, were everything is just automated. And you don’t have to do anything yourself, but you can get automation in several stages.

The first part of automation is really you as a developer, you have been used to use Xpediter or debugger to step through your code to do your unit testing, so you write a test driver, you execute the program, you manually step through it. Automating that part to have every possible test case that you as a developer can run, that is the first part of automation because you have automated the manual work that you as a developer or a tester would have to do. You’re still executing it manually, but it is automated for the testing part, so that’s the first part of this automation, and that’s actually the most important part, and many companies don’t realize that that is actually where you get the most benefit by automating the manual developer work.

Then you want to do as much automation as you want to. So, the next step after doing the automation of one this case and you start to automate more and more test cases, making a complete regression suite with all your test cases, then you also want to start automating the execution of the test cases, so you don’t have to remember to do that. And that is when we get into the continuous integration/continuous delivery concept where you have a pipeline, an automated pipeline that can also automatically execute your test cases and verify if it was a pass or a failure and in this way, stop an automated deployment and get back to the developer and say they have a problem. That is the next stage in automation, and that is also extremely valuable, but you cannot do that before developers have learned to automate their own testing. And that is the first step.

Repeatability is Key

Rick: Yeah, you used the word “repeatable” and that struck a chord with me. I think that is so true, Steen. I think one of the keys to automated testing and more importantly, what you just described, automated test case development is critical in having the ability to automate that creation or that test case, where you’ve got something that I, as a developer, can use over and over and over again for the support of programs or applications that maybe I’m responsible for.. But you also create a test case that’s usable by other developers. Programmer A makes a change to program one. Six months later, another requirement or request is made, programmer B can use that same test case to make sure that his or her changes are not corrupting that baseline code. And so, the ability to leverage that even, from a manual execution stage, the ability to reuse that test case, will save an organization so much time over a period of weeks, years, months, whatever.

Steen: Yeah, can I also just comment a couple of things there, because you point on something interesting. One is when you have one developer creates a test case, another developer two months later comes in and needs to maintain that program and needs to understand it, the best documentation you can have of the program, that is not a Word document. A Word document, that is something developers hate to write, and as soon as you have written it, it is out of date.

But test cases, they are the most precise specification and documentation of what a programs does. So, reading a test case can actually, as a developer, get you to understand what the program is doing in a certain condition, so that’s it really also a huge benefit of doing or creating automated tests.

I actually forgot to say a thing about the automation when you asked about that, but you pointed that out yourself. Because you also said the possibility to automatically create a test case, and I forgot to talk about that. And that’s actually specifically about the mainframe, it can be hard to create your test cases because mainframe programs developed over decades, tend to be monoliths that are using a lot of data. So, you can have linkage fields or record layouts of input data to a program with hundreds or thousands of fields. Setting up data to execute a program manually, that can be nearly impossible to set all of these fields correctly. So, having the capability to execute your program live, or for real, and then record a test case and actually take that data that was provided programmatic are to your program and save that to execute that test case later.

That is also an essential part of a really good testing tool, and that is what Topaz for Total test can do. It can let a developer execute a program and record a test case, including all the data that it’s using. And that saves a huge amount of time in setting up the test case. That is actually what we call “true test automation”—not only automated execution of the test case, but automated generation and creation of the test case. That is real and something completely new, or least a couple of years ago that we introduced this. But that is also a game changer in speeding up test automation for mainframe customers.

Rick: It absolutely is. I can remember not on the mainframe, but on the non-mainframe development side, the amount of time required to set up these test cases. It could be very labor-intensive, and you’re exactly right. For these very large mainframe systems, the creation of that test data or those test cases consumes a significant amount of time.

I know a lot of shops that I talk to and spend time with, are now requiring as part of, or the definition of “complete,” not only working code, but also working test cases are part of their deliverables in the changes that they are required to make. And so, being able to automate the development of those test cases will have a huge impact on cycle time in that software delivery cycle. So, it’s just good stuff. I’m glad you’re focusing on that because most people focus on… When we think about automated testing, we think about it from an execution standpoint, being able to run those test cases and your products with Total Test and Hiperstation gives us the ability to do that. But just as importantly, the ability to produce those test cases automatically with the machine saves significant time in that overall software development cycle as well.

How Automated Testing is Better Than Traditional Testing

I’d like to hear your thoughts, Steen, on why modern automated testing is better than the traditional testing. I talk to a lot of folks and traditional testing works, we’ve done a good job on the mainframe with regards to testing the code that we build and the code that we’re maintaining. The problem is the time it takes to do it. So, I’d like to hear your thoughts about why modern testing techniques, leveraging automation, is better than the traditional testing methods.

Steen: Yeah, I can talk about that. And I think I would actually use a case of a customer, or actually several customers that I have discussed with. So, what we normally do when we start to engage with customers that are interested in test automation, is that we start to make a value stream mapping of their development process and the testing process to understand how they do testing today. And a quite typical way that customers today, mainframe companies today are doing testing on the mainframe is, developers, they’re doing manual unit testing. So, they have a program that changed, they have to test it, so they debug the code by stepping through it line by line and making sure that it is working correctly. And they have to create a test driver to execute the program.

Then, later in the stages, when they want to do batch testing in their testing environments, typically they will set up JCL to submit jobs that will run the batch and then they will have to use manual steps to compare data sets, to look into DB2, to make some extracts of DB2 to compare data. So, it’s a lot of manual work to write all this code, the JCL, and the steps to get the data and the verification is typically manual. They’re looking into a comparison of 2 DB2 extracts, they’re looking into the differences between two files, so it’s all manual, they don’t have any testing framework to help them automate it and to help them understand if it is a pass or fail, that is a manual process. So, again, it is the manual work and all the work they have to build around the mainframe in order to create a testing framework, that is what mainframe companies have been doing for decades: developing their own testing frameworks, and that is just so much waste of time.
Well, it was not two decades ago, or just one decade ago, because you didn’t have any tools. But with a tool like Topaz for Total Test you have the framework to automate all this kind of work. You don’t have to manually sit in a debug and step through your code, you can do what we call virtualized testing. You can record the test case, you can execute, you can virtualize, you can control. Like you can do with a debugger, you can change variables, you can change all the output or input to the program from external side, you can change that, and then automate a case.

In what we call non-virtualized testing, when you’re testing in real batch on real CICs region, you can set up a test scenario they can actually submit that batch job. So, you can set up the JCL that you’ve been used to, you can automate the execution, you can automatically compare datasets against each other, you can verify data in DB2, all in an automated fashion. And have the test scenario pass or fail automatically based on the comparison results. So, it’s all about automation both you can set up the automated test scenario and you can set up the automated verification—was this a successful or non-successful test case?

So, we are back to, again, to remove all the waste and having the right modern tools to actually set up to these cases and automate. So, it’s modern tooling and it’s about removing waste on manual work and make the developers happy. No developers or testers like to do repetitive work and providing modern tools that helps them in automating what they actually think is boring work so they can actually do what is fun, and fun for a developer is to write code and writing code is what brings business value and business activity. So, that is why this automation is just so much better over traditional testing.

Rick: I think for the traditional mainframe developer, you think about a lot of the work that we’re doing with regard to applications and code on the mainframe, a lot of it is making modifications to existing code. A lot of times we’re making, or people are making changes to code that they had no knowledge or experience of in the past. And so, having an inventory of test cases that can be leveraged by a developer who may or may not have experience with a particular code base, being able to use that to test their changes to make sure that their changes are not corrupting that baseline. To be able to use that over and over and over again has such tremendous benefits not only from a time standpoint, but from a quality standpoint. The quality of the code that we’re able to produce leveraging tools like we’re talking about is going to be enhanced significantly. And so, not only are you gaining productivity times with regard to actual cycle times in the execution of those tests, but you’re reducing the time required for problem resolutions, because if I can go in and I can leverage these tests to produce high-quality output, I’m going to spend less time in problem determination and defect resolution downstream. So, the value is two-fold from a productivity standpoint and from a quality standpoint.

Virtualized vs. Non-Virtualized Testing

I want to move into… You just mentioned virtualized and non-virtualized. I know you and I talked about this on the last podcast, but I think it’s worth mentioning again, the differences between virtualized and non-virtualized testing, and then I’d like for you to describe the differences between those, and then we’ll talk about the automation of virtualized and non-virtualized testing. So, can you spend a few minutes and refresh our memories on virtualized versus non-virtualized testing?

Steen: Yeah, absolutely. So, originally, we talked about different testing types: unit testing, functional testing, integration testing, system testing, regression testing and so on, but what we realized was that it made a lot of confusions because different customers are using different concepts and have different personas doing different test types. So, it caused a lot of confusion when we basically talked about unit and functional testing. And what we actually found out was, What is it that Topaz for Total Test, our testing product does? Well, it focuses on testing the program. So, you want to test a program—that is what you are concerned about. If you are a developer or a tester, it is the program testing that is interesting and not what you are calling the test type. And how can you test a program? Well, with Topaz for Total Test, you can test the program in two different ways. You can test it virtualized. And virtualized, that is, when you execute a test case, you can stub out all the surroundings of the program. So, this is something that the developer will do in order to be independent of a CICS region and IMS transaction engine, independent of data sets on the mainframe, independent of sub-modules that you don’t have control over, independent of DB2. So, you as a developer, you can control in a certain test case, what should DB2 return for rows, what should this CICS API call return. So, you are complete in control, you are stubbing out the complete world and stubbing out, that is also called virtualization. We are visualizing the world, making it possible to be in complete control as a developer. That is virtualized testing.

Non-virtualized testing, that is when you want to test the same program, but you want to test it in a real environment with real data. That is, it could be a batch job running against a real DB2 sub-system, it could be a real CICS transaction or real case module accessing also a real DB2 sub-system that you want to test. So, you assure that it actually works in the real environment, not just in the virtualized environment.

And that’s two different ways of testing a program. What personas are doing it? If it is a developer, developers, I believe, the developers want to have both these capabilities in their toolbox. Depending on the problem their solving, sometimes virtualized testing is the right tool. Sometimes non-virtualized testing is the right tool.

Then we have the QA persons. They will only do the non-virtualized testing. They are the ones that are using a test environment or QA environment and running the program in the real environment, testing that the specification of a change is as expected. So, the testers will be the users of the non-virtualized testing while developers can use both. So we don’t… And the problem has been, when we have talked about functional testing, some companiess are using that concept for only testers, and some companies are not using it at all. So, virtualized, non-virtualized testing is just easier to understand, it is a program that we’re testing.

Integrating with Other Tools

Rick: I totally agree with that, and I think the application architecture plays a role in which technique to leverage as well. But I certainly see both in clients that I’m talking to. With regard to virtualized testing and non-virtualized testing, can both of these techniques be automated from within a CI/CD orchestration engine like Jenkins. Do I have the ability to include as part of my build the execution of virtualized test cases as well as non-virtualized test cases? Can I automate that work in a tool like Jenkins?

Steen: Yes, you can definitely do that. And it would be a good idea to include both. And what you typically would do, so what you could do if you have… Let’s say you have a development environment and you have a test environment, or QA environment, let’s call it the QA environment. So, as a developer, you have made a change in your development environment and now you want to promote your change from development to QA. So, you can do that and will… ISPW, Compuware’s mainframe version control system and build system for mainframe applications, it can actually trigger a Jenkins pipeling to run when a developer starts to promote, for instance, from development to QA.

And what you want to do in such a pipeline—you can of course do anything you want—but one thing you could do with automated testing is when that pipeline runs, it can check out test cases that you have shared in a version control system like Git, so it can check out both the virtualized and non-virtualized test cases, so you have them in the pipeline. The first step would normally be to execute the virtualized test cases, because then you’re testing against the node modules that are in these temporary datasets in the QA environment to verify that the compiled code in the QA environment works as expected.

After that step, you typically have a deploy step that will take the node modules and deploy to the production datasets in the QA environment, and that could be in the batch environment, in the CICS environment. And after the code has been deployed, you can then have a step that would execute the non-virtualized tests, and that will then execute the tests against the load modules that have now been deployed and are using the real subsystems like DBS and actually testing in the real CICS region. And this way you get both testing done in one pipeline. If any of the tests fail it can return back to the developer and tell them that they have broken the pipeline.

So, it’s also about quick feedback. As soon as the developer has promoted that change, they don’t have to remember to do this kind of testing, all the testing will be done automatically.

Rick: Gotcha. How hard, Steen, is it to implement this type of test execution from within a pipeline, a Jenkins pipe?

Steen: So, if you know about Jenkins and you know how to use command line interfaces of products, which is something that every CI/CD engineer knows about, then it’s not difficult because with Topaz for Total Test, it comes with a command line interface, a CLI as it is also called. And for Jenkins, we also have a Jenkins plugin. So, it’s actually really easy. And it’s really using modern techniques like Git to get the test cases in using the CLI, to execute the test cases, and the test cases will be executed exactly the same way as if you’re using Topaz, the Compuware IDE, where you are using Topaz for Total Test as a developer. The execution of the test cases would be exactly the same, it will be the same reports that are generated. So, as a developer, when that pipeline has been set up, you can go in, into the pipeline, into the concrete build and see the results. And if there is a problem, you typically will go back to Topaz, check out the test cases from Git, so you get the same test cases as the pipeline did, reproduce the problem in your IDE, fix the problem. Either, if it was a test case that was the problem, you can commit them back to Git; if it was the problem in the code, you can make the change, promote again, and you will see the pipeline is running successfully next time.

So, it’s not hard. Of course, there is something that you need to learn in order to set up Jenkins, but if you have that expertise already, then it’s not difficult to set up the Total Test part.

Rick: I’ve got customers that I’m working with Steen, and again, these pipelines can be set up essentially any way that you as an organization or developer want to set them up. It can be set up in single pipelines, you can create multiple pipelines based on separation of duties. I’ve got a couple of banks that I’m working with where they’ve actually got multiple pipelines that they’re leveraging because of compliance issues, separation of duty policy requirements within their organization, where just as you described earlier, developers may be using what they call a commit or a dev pipeline. And within that pipeline, they’ll leverage virtualized testing cases to support the work that they’re doing. But when that work is complete and turned over to testers within that Agile team or under a more traditional organizational structure to a QA group, then the QA group will have their own pipeline for executing the work that they are responsible for, and typically in those pipelines, we see the use of more non-virtualized test cases. So, you can set these things up any way you want within the pipes. And again, as you described with Jenkins, at least, we’ve built plug-ins to make that process or the leveraging of those test cases very easy to implement and incorporate in your automated pipes.

Incremental Implementation

One last question I wanted to talk to you about, and then I’ll turn it back over to Matt. What’s the best way, Steen, you know organizations, especially mainframe development organizations, have been testing for a long, long time using traditional methods, and again, I’m not going to say that they’re bad because the result is we got applications that have been running 30, 40 years. So, again, the problem is time. So, what’s the best way… What have you seen as a way to implement or to transition to more modern testing approaches like that we’re talking about today, for those shops that have been utilizing traditional methods for sometimes 30, 40 years? Is there a best way or best practice to implement some of these changes?

Steen: Definitely. I’ve seen both successful approaches and non-successful approaches, and the way to success, that is to take an Agile approach in adopting automated testing. And I believe this is nothing, this has nothing to do with test automation. All changes, I believe, when we’re talking about adopting new tools, it is… You should approach it with an Agile approach. And that is start small and pick a dedicated team that is interested in test automation, pick a dedicated team that gets the tooling enhanced, gets some experiences. The team itself should also start small, it’s just a simple… pick one of the simplest programs, start to learn the tool, see how they can create test cases, how they can automate the testing, and when they got the tool and understand how to create these automated test cases, they can start to expand to more complex programs. And as they learn and adopt this in their process, in their development process, they can also start to document this and describe the benefits. And when the team has been successful of adopting the tool, then you can start to expand in the company, because then you have some advocates internally that knows about the specific architecture and special development processes in the company and how they have been able to adopt this tool in their team. They can start to educate the rest of the organization or typically the methods and tool department can take that knowledge, bring them on and tell other departments on how they can get started and they can actually see their success.

So, it’s really about starting small and learn, and then expand the implementation. And start also to build what we could call a test automation knowledge center. So, you have a group in the bank or in the company… Sorry, we have a lot of banks as customers, but… So, in any company, you can have this knowledge center where you can start to spread the knowledge and the new teams that gets on, they have a place to ask questions and learn from. So, these are some of the things that we can see that is the way to success.

So, in these advices, we actually created a best practices guide that is online on our DevOps website, where this best practices got in adopting Topaz for Total Test, both in an organization and as an individual, and there is some good advice there on how to get started.

Rick: Yeah, that’s awesome stuff. I’ve seen and looked at that document. It’s a great document. We’ll see if we can’t create a link in our Good Coding page as well. Good stuff.

I couldn’t agree more. Music to my ears when you say that, because I think that leveraging Agile in the management of a software delivery system… Those that have been listening to the podcast for a while now will know that that’s such an important point that I try to make. I know back in the first podcast, we talked about that, or second podcast. And I believe that that’s the way to implement a modernization strategy, incrementalism. Learning from small successes will actually shorten your transition or implementation time, and again, it’s applicable to any technology that you’re trying to, or process quite honestly, that you’re trying to implement or incorporate change into your processes. So, I totally agree.

I see from time-wise we’re about to wrap up. Matt, any questions that you might have for Steen before we wrap it up?

Changing the Culture

Matt: I actually do have one. Steen, you mentioned earlier the survey, and one of the numbers that really jumped out at me on there, or two of the numbers, is that I think it was around 90% of respondents said that they saw automated testing as kind of a key to future success, but only 7% are currently using it. So, what’s keeping people from using it? Are there roadblocks that they’re running into? Why is there such a discrepancy in people who see the need and people who are actually doing it?

Steen: Yeah, and you’re totally right, it’s really interesting. Everybody agrees that this is the single most important thing, yet only so few are doing it. And I think the issue is exactly what Rick just described. You have development organizations that have been doing the same development, the same testing in decades. So, change is not easy when you have been doing the same thing so many years, both from the people in the organization, but also the organization itself. They have all kind of processes and methods around what they’re doing. That it is the hard part, the tooling part is the small part. It is the organization, the people change, that is the hard part, and I think that is what is restricting most companies of getting started. That is, how do you start on that change? And that is again, start small. It is a really complicated thing to change the whole culture of software development—we’re not just talk about testing, this is about the whole DevOps process. And the best way to start is, really, to start small, eat the elephant one bite at a time, and you learn on the journey that you are on and you start to understand how to take the next step. It’s really about the incremental implementation.

Rick: I’d like to add to that, Matt and Steen. I totally agree. I think that I had a lot of organizations are still leveraging a waterfall mentality with regard to even implementing new ways of delivering software. So, if we talk about incorporating or building or transitioning or evolving to a more modern software delivery system. I see a lot of organizations trying to architect a complete solution or architecture for the delivery of software before they begin doing anything. And I think that if you’re trying to define the whole, it becomes an almost impossible task. And so, I see organizations that will, I think somewhat through fear, I’ll just say it, that because of the unknowns are afraid to take on those challenges.

Secondly, they feel like they don’t have the time. But I think that the key is to transition incrementally. I think that resolves some of the fear. It certainly resolves the risk, and maybe most importantly, I think it resolves or addresses some of the cost in a transition to a more modern software delivery system. We’ve got to do this incrementally and learn. That way we manage the cost, we manage the risk, and we learn incrementally and we’re able to apply those learnings to subsequent work to eventually lead to the target state that we’re striving for.

So, I think incrementalism, leveraging Agile techniques with regard to the evolution of your software delivery system, is the way to overcome that and a way to make it happen, and to get those adoption numbers.

Rick: And just one comment to that, because you actually point out a thing here. Many of these large organizations, they have departments describing processes, typically the tools and methods department, and they typically want to describe everything up front before they start the implementation. And I think it’s completely wrong.

They should turn it around, get their hands wet, experiment, have that team that first team try it out, learn from them, and then they can build the processes. So, it’s really… They should take the startup approach, don’t think too much, just get started, try it out and then start to put processes around.

Rick: It’s classic waterfall software development, if you think about it. In a waterfall world, you’re trying to get all your requirements nailed down and get everything perfect, get your design, your architecture perfect before you start writing any code. And we all know what problems that leads to from a business standpoint, and from a technical standpoint. I think the same… We’re seeing the same thing because that’s how we build systems. Again, I’m beating a dead horse here, but the software delivery system is just as critical a business application as any other business service that you as a business are supporting and we’ve got to adopt or start to leverage Agile methods as we refine, as we evolve, as we start to deliver a more modern software delivery system. Incrementalism certainly is the key there.

Well, I see we are completely out of time in. Steen, again, I can’t thank you enough. You bring so much to the table for our organization, certainly for our customers. We appreciate the work you’re doing from a testing standpoint. I believe, as we’ve already stated, it’s the most critical component of the software delivery system, and especially for those applications targeted to the mainframe. So, thank you for joining us. Matt, I’ll let you wrap us up and then I’ll close us out.

Matt: Okay, thanks Rick, and thanks again, Steen, for joining us. To everyone listening, be sure to join us this Friday at11 AM Eastern Daylight Time for our live online Q&A session, Office Hour with Rick Slade. Bring any questions you might have about automated testing or any other part of the SDS or anything that we’ve discussed, or about Agile implementation of Agile. You can also submit questions via Twitter, just use the hashtag #goodcoding. To set a calendar reminder for the Q&A or to check out any of our previous episodes, including Steen’s visit in Chapter 10, our discussion of testing there, visit

So, Rick, I’ll send it over to you for some last words.

Rick: Good deal. Thank you, Matt. Again, thank you again, Steen. I appreciate the conversation. Folks, automation is critical in the evolution of your software delivery systems. We’ve got to look at all of the work that’s being executed in that software delivery lifecycle and start to make determinations as to how much of that work can be executed by the machine and then doing the work required to make that happen.

I think there are three critical components to effective software delivery in the modern world, and it begins with automation. The other two are integration and measurement. The ability to integrate with other technologies has to be there. But automation is key, and so I hope that you start to take the time in your evolution or in your transformational journey to understand where you have opportunities in your cycle to leverage automation, and we hope that you’ll contact us if you’ve got questions.

So, as Matt said, if you’ve got questions about the podcast or about the live Q&As, go to Hopefully we’ll have the information you need to help you with your journey. So, with that, we’ll close out today’s podcast. Thank you very much for downloading this podcast. Thank you for the time you’re spending with us. We hope it’s beneficial. We hope it’s useful. We appreciate your time very much.

Take care. Have a good week, a safe week for those listening on the weekend, a great weekend. Stay safe. Everybody and good coding. Take care.