This morning, Shrini Kulkarni sent me an email asking what I thought of requirements based testing approaches. Specifically, his email has this line in it, quoting from a web page:
“XXYYY Inc is the world’s leading expert in Requirements Based Testing (RBT). The RBT process first ensures that the specifications are correct, complete, unambiguous, and logically consistent”
Which made my spidey-sense go off. Shrini asked me to elaborate on why and what that means, and liked my reply enough to encourage me to blog about it.
1) On day 90 of a 150-day project, the business senses an opportunity that will allow them to double revenue. They change what the software must do. This renders the perfect spec imperfect.
2) On day 100, the VP of engineering is fired. The new VP of engineering thinks that the fribble feature is “all wrong.” The spec is now imperfect.
3) On day 110, the CEO promises BIGCO that the software will do X. The spec is now imperfect.
4) On day 120, the VP of sales interprets a line item as doing X+1, and promises it to the customer.
5) On day 125, the new VP of Engineering brings back the famous consultant Rickaro BlenderBurger, to explain how he personally inspected the specifications to make sure they were unambiguous, and the X+1 promise was not right. The customer responds “But he promised us.”
6) On day 126, the CEO creates a time warp field and goes back to day 120, and personally sits in the meeting with an invisibility cloak, to figure out how it is possible for the VP of sales to get the unambiguous document wrong. He brings along a linguistics professor.
The linguistics professor watches the meeting and says “Oh, yeah, that is going to happen. English was pretty much hobbled together by illiterate peasants, and codified by writers like Chaucer who knew just enough Latin to grok the letters. It is a pre-rennesaince, pre-age-of-science language. Anyone who tells you they can write an unambiguous spec in english is selling you something.”
—> In other words, to quote James Bach:
“Here is a simple litmus test for an ambiguous spec: It is written in English”
Seriously. Ask an engineer the difference between and English spec and a CAD drawing.
—-> People who think you can write an unambiguous spec, so you can slide it under the wall, are denying how most human beings actually communicate and collaborate.
It’s not about hand-offs. It’s about working together. Or, in Japanese “Gung Ho.”
Hence, my spidey sense.
UPDATE: I am not saying that we should completely reject the idea of requirements or specs and only go by word of mouth. As with many things, the danger is in the extremes. Instead of just picking an extreme, we could have a more healthy conversation about which way we move the needle – to do a little more conversations or a little more documentations.
UPDATE TWO: Over the great back-channel of email, Jonathan Kohl points out that even though the claims of XXXYY Co. may be unrealistic, the ideas and concepts that XXXYY uses could be valuable and might be worth looking into. There are exercises that can be used to get less ambiguous documentation, and they can be helpful. As the expression goes “Don’t throw the baby out with the bathwater”, and I think he has a point. Thanks, Jon.