It's Not The Methodology, Stupid

One of the more incredulous spit-takes I’ve done recently was the claim that developers use Agile because they want to avoid doing documentation.

I’m incredulous for a bunch of reasons, but the core one is this: I’ve never, in almost two decades, worked with any decent developer who flat-out wouldn’t do any documentation. Hell, I’ll go further than that and note that most of the good developers I’ve worked with want documentation and are keen on writing it (up to a point; more on that in a moment).

Antipathy to documentation and other goodies normally swept up under the tattered carpet of “non-functional requirements”, one of the greatest misnomers of software development and feployment, comes, in my experience, from other sources, most prominenently project managers looking for ‘fat’ to trim, interested only in what they can chuck into production to call the project a success, and project managers themselves would note this is mostly a function of their role being judged on delivery timelines, money, and not whether the delivered product works well in a year’s time. Oh, sure, in theory theres’s more to it, but if you’ve worked or contracted to large enterprises you’ll know that the PMs who get promotions and contract renewels are the ones who get it over the fence in time for the financial reporting to look right for a sponsor’ss performance appraisal; doing it right is merely incidental to this process.

But back to developers: I have to confess that my argument that decent developers not wanting to avoid documentation will take a bit of a hit when I reveal that I consider documenting your code to be one of the hallmarks of a decent developer in the first place. At this point the sceptic may leap up and wave their Rhetoric 101 textbook around as though my tautology undermines my argument.

It doesn’t, really.

Here’s the thing: anyone who’s spent any reasonable portion of their life supporting their own code, or being within throwing distance of the desks of the people who support their code, quickly becomes a fan of the idea that life is a lot easier when that code is adequately documented, whether that’s with sensible logging, adequate comments in the source, or a standalone document explaining design decisions. These things ultimately lead to less time explaining the same tedious shit again and again, or waking up at 2 in the morning because you’re the only person who can frob the blort.

It’s a fundamental part of the learning experience of coding with more than, well, yourself. And there are, in my experience, fewer geniuses-who-don’t-document than pop culture might lead you to think. So, yes: I’m quite comfortable with the idea being a decent coder includes the ability to document.

And not just because of the positives cases I’ve just described: the flipside is that almost all of the worst code I’ve had to work with was written by people who don’t document, the nadir of which was surely the >3,000 line Java class file with cut and paste sections, chains of if/else if logic, magic numbers, and not a single comment explaining the logic behind any of it.

But that doesn’t mean those decent programmers will want to do all the documentation demanded, of course. Oh, no. But one reason for that is the kind of environments that take on Agile methodologies are typically large. And in large environments are full of people who want acres of documentation; worse yet. They want acres of documentation which is basically useless for its nominal purpose.

Moreover often the person empowered to demand documentatio seems to have comically timewasting ideas about what should be in it: when I document, say, my design decisions around a virtualisation platform, such as our HA strategy, I’m documenting for my peers (other infrastructure designers) and SMEs (developers builing aps on the platform, admins implementing it); it is a wate of time to write that document such that someone with no background in virtualisation could use it as a instruction manual to bootstrap themselves into technical competence; likewise when I document TCL, Java, or Python code I am profoundly uninterested in trying to teach someone how procs, inheritance, or anonymous functions work, either.

And that documentation, worse yet, is actually useless for the supposed target audience. By the time you’ve explained to a gatekeeper how KVM, Linux, and Python work, the person who has been woken up at 2 in the morning to restore the virtualisation manager get two pages in trying to look for the details about where and how you back up the RHEV-M database so they can kick of a restore, finds only acres of worthless bumf they could get from Google, gives up, and rings whoever they think is mostly likely to be able to tell them.

In enterprise-type environments this becomes, alas, a real source of conflict; there are a never-ending stream of people who feel that unless they, personally, feel they understand a system, it isn’t properly documented, and more often than not they seem to be in gatekeeper positions; rather than asking “Do the people who do the work have the information needed for their jobs?” - an entirely proper gatekeeping function - they ask “Do I now feel I understand the ins and outs of this is system to my personal satisfaction, such that I could do this job?” - something which creates a massive time sink for your productive teams. Your architectural overviews, designs, and operations manuals no longer become concise brain dumps between peers, but rather the technical equivalent of “Run, Spot, Run” for an anxious bikeshedding audience. Worse yet, the very process and concept of producing that potentially useful documentation becomes discredited: everyone will agree that an operations manual would be a delightful thing to have, but no-one wants to go through the pain of writing what will become effectively a write-only document of minimal practical value.

The TL;DR version:

  • Your programmers and infrastructure techs don’t avoid writing documentation because of Agile, or XP, or RUP, or whatever process du jour you have. They hate writing documentation because you ask them to write the wrong documentation.

  • They don’t get to do the documentation they like writing, anway, because you won’t fund it.

  • If your relationship with your staff is such that you see them as lazy rebellious good-for-nothings, per the consultants linked at the start of this post, you’re fucked beyond any menthodolgy’s ability to save you.