Measure mainframe development quality

The Best Test to Measure Your Mainframe Development Quality

I recently ran across The Joel Test from Joel Spolsky, CEO and co-founder of Stack Overflow. The test contains 12 questions to rate the quality of a software team. Although he posted this on his blog in 2000 (which has rendered many of the outbound links in the blog 502 errors), I find that the test itself is still valid.

Joel did a great job of explaining the questions his test presents, and you should read what he has to say. However, being on the mainframe side of things, I thought it was worth providing the same test with my own comments here to give you a better sense of how to measure your mainframe development quality in an Agile-DevOps world.

  1. Do you use source control?
    Source control is essential. Without a means to control access, to know who is working on the code, to promote the code to the levels you have set on the way to production, you end up with chaos.
  1. Can you make a build in one step?
    This just gets to automation. If it is difficult to do a build or compile, then there is hesitation. You delay making changes and that delay is built into every estimate and task. It makes it difficult to do the next point, daily builds.
  1. Do you make daily builds?
    After all this time, daily builds should be a given, but the reality is they’re not, particularly in the mainframe. Yet they are so important to help you gauge progress and even more so for continuous testing. It also has the subtle effect of making sure that you have broken tasks down small enough.
  1. Do you have a bug database?
    It’s hard to fix unknown problems. It’s hard to measure mainframe development quality and progress without any idea of the problems. Keeping track of issues, and I might add enhancements, is critical.
  1. Do you fix bugs before writing new code?
    Still solid advice. It goes to your foundation. You shouldn’t build on something that isn’t strong. If you can test applications early on to find bugs and fix them, you can make more confident code changes.
  1. Do you have an up-to-date schedule?
    You are writing the software for a purpose—there are people depending on it. Make a schedule and keep to it to stay focused on the important things. I’d also add that you may have a schedule, but how can you accurately track your progress without things like daily builds and continuous testing?
  1. Do you have a spec?
    Today we’d call this a Story, to use Agile terminology. A good Story has a clear description of the user’s view of the functionality, and along with that, clear acceptance criteria. This is in writing, something everyone can see (Atlassian JIRA is a great team collaboration software for this), which describes the desired behavior and acceptance criteria.
  1. Do programmers have quiet working conditions?
    This one just makes sense, but is very easy to overlook. I’d add not only quiet, but you need to avoid other distractions so they can focus.
  1. Do you use the best tools money can buy?
    In mainframe development, it’s more essential than ever to give your team the best tools available as experts leave and college grads take over. As Compuware CEO Chris O’Malley stated in an interview with CIO, the best tools integrate with traditional toolsets that let people work with the mainframe using the tools they’re familiar with, making it so the difference is only in syntax.
  1. Do you have testers?
    To me it’s more about acknowledging that you need to allocate for testing. It’s not the thing you do right before you release if you have time. It needs to be part of every task in the DevOps lifecycle. If it hasn’t been tested, it’s not complete.
  1. Do new candidates write code during their interview?
    You will own the code they write, you will have opinions on it later, why not see some before you hire?
  1. Do you do hallway usability testing?
    Sometimes we see usability testing as having to be so formal, so structured that we just end up bypassing it. “There just isn’t the time,” I hear. The point is that it doesn’t have to be this way and if that stops you, don’t let it. Do usability testing and do it often. Even if it is casual—as with Joel’s recommendation that you grab the first five people you pass in a hallway and force them to use code you just wrote—you will get results you can use.

These 12 questions are things that I’ve always believed but never seen in one place. Now they’re here for you to use, or you can read The Joel Test in its original form here. This is a great reminder for us as to what goes into quality mainframe development teams. They don’t just happen; they are the result of these best practices.

For more help, I recommend reading my colleague Glenn Everritt’s blog series on the two-week COBOL sprint, why you need it and how it’s done, as Agile sprints are yet another vital means to helping you increase mainframe development quality in your shop.