The Future of Maintainability is CI

Perhaps the most influential, and most humbling work in my career has been on Clean Code, by Robert C. Martin, where I was “just” a reviewer.

Separating software along clean lines of interface, adding the ability to swap parts out, combined with clean implementation and unit tests allow teams to make truly maintainable software.

Despite that, over and over again, I see legacy software that “can’t” be unit tested. Attempts to start unit testing begin with great fanfare … and later fail. This happens so often with companies I’ve worked with, for so long, that I have started to discover patterns of failure. Put this system of forces in a box, and if I come back in two years, the unit test effort will have produced a net negative impact. The refactoring and clean code efforts will be net negative.

The solution, and the future, is in continuous integration and feedback.

Let’s talk about why that is.

Down The Rabbit Hole

Generally, when I am working with a programmer on a consulting assignment, I ask to see the actual code they are working on. As organizations that bring in consultants are usually large and established enough to afford consultants, and people tend to struggle the most with core systems, I’m often looking at legacy code.

So they show me the 3,000 source line of code class. In some cases, it is a 3,000 or 6,000 source line of code method.

That’s okay. It is what I signed up for.

Usually there is a motivated programmer in the group, someone who of their own volition read the work of Bob Martin or Kent Beck, who took to wrestling the code beast, isolating it, and creating a unit test. They spent hours, perhaps days, just to get this one method under test.

The “unit test” they show me consists of 50-200 lines of setup, a single function call, a couple of assertions, and perhaps 50-200 line so teardown. It tests one path through the code. If we look only at lines of code, there “should” probably be dozens or hundreds of tests to get reasonable coverage. If we look at branches, at covering this if but not that if and that if not not the other and neither and both … we’re looking at thousands of potential tests.

Ugh.

There is no way the team is going to do that. Not sustainably.

So instead we talk about the strangler pattern and the boy scout rule and merciless refactoring. For every little change, extract some code to a method and test that method. It’s solid advice, and the right thing to do. They even try.

They had unit tests. They followed the advice of the gurus. Perhaps the guru three years ago was not me. Perhaps I am instead coming in three years later, after they follow the advice. I find the TDD proponents are frustrated. That legacy method is now four thousand lines of code instead of three.

Here’s what’s happening.

A Simple PICNIC Analysis

It’s a week after the coaches have left, and the people doing the work are sitting back down to do the real work of production. Two new feature requests comes come in, picked up by two different programmers. ProgrammerA is test-infected and highly motivated. ProgrammerB is … not. They finish the work at about the same time. Perhaps ProgrammerB is a tiny bit faster, as pulling code out and creating unit tests takes a little bit of time.

Then the next set of features comes. ProgrammerB goes about their merry way, adding the new feature. ProgrammerA starts by running the unit test runner …

… and there are a bunch of “failures.” I put failures in quotes, because ProgrammerB points out the code is not broken, it is simply that the expectations have changed because of the change he made. ProgrammerA then needs to spend time “greening” the tests. As the unit tests get more and more coverage, they break more an more often due to programmerB and his ilk.

This idea enters the language. People say “the tests are broken”, not the code.

ProgrammerB gets a Positive, Immediate, Certain (“PIC”) incentive for ignoring unit tests. ProgrammerA gets an Negative, Immediate, Certain (“NIC”) for having to make them run green. Meanwhile, the magical idea that this sort of work will save time later is Delayed and Uncertain.

Positive Immediate and Certain versus Delayed and Uncertain is the sort of reason that people stay addicted to cigarettes when they know it may give them cancer.

Eventually the ProgrammerAs of the world give up. Without unit tests, the staff is not comfortable doing refactoring, and the code gets worse.

You know what I noticed in all of the teams with this problem?

None of them had Continuous Integration.

CI As the Future Of Maintainability

If the unit tests are tied into Continuous Integration (CI), then they can run after every code change. If the tests fail, you know who broke them. The tightens the feedback look from every few days to every hour or so – faster when starting out with unit tests.

Making unit tests pass part of the definition of done makes “cheating” and borrowing trouble much harder. Having them tied to a commit in version control means you know who has to fix what.

There is a second test, which is simple enough. Expect code to be lower in complexity over time, and have the CI system measure that complexity.

CI, a unit test runner, a complexity measure. These three things change the game. So why don’t more people do them?

Without an official investment of time by the company, there is no incentive to make CI. The product owner’s incentive is nearly always more features, more features, and any CI project will take time and money away from the feature monster. Again, there is a negative immediate certain incentive to do CI, while the benefit is delayed, positive, and uncertain.

Most teams need a Charles.

Find your Charles

Years ago I worked with a programmer named Charles who stayed late to implement a Continuous Integration System. Specifically, he moved the team to Jenkins. I was contracting away from home and only had an empty hotel to go to, so I stayed late and saw how much he worked. Charles worked several hours a month for no extra compensation. In Nike terms, he “Just Did It.”

No one saw how work Charles put in. Once it was done, he got almost no credit. Most team members assumed the work was free, or at least easy. To some extent, Charles was okay with that, because he found the work rewarding for its own merit.

If you want a solid CI system, your team could use a Charles.

If you don’t have one, that might be okay as well. Increasingly, Excelon has been doing CI as a product and service. We can come in, set up a CI Service, team you to use it, create some reference tests, and maybe train a little. This simple change to having meaningful CI, along with a few policies, can have a positive impact on code quality.

So convince your management. Or recruit a Charles to do it anyway. If you have CI and it isn’t working as you’d hoped, take it off the shelf and re-jigger it. Or call us and let us do it for you.

CI can provide the incentives (and consequences) to have clean code and running tested features.

What is driving your development?