A Parable of the Polar Vortex

The best time to shovel your driveway is immediately after it snows, but failing that, any time before you have driven on it. Driving on fresh-fallen snow compacts it into something between “ice” and “pure evil”, depending on the temperature, humidity of the air, humidity of the snow, etc.

I just went out and shoveled and chipped our driveway down to bare concrete because we’re about to get another 6 inches of snow, and the very last thing you want is to add fresh snow to existing ice. That’s how you get glaciers. I was thinking about technical debt, because physical labor induces contemplation.

Of course, the best time to clear up technical debt is right after you create it, when everything is crisp and fluffy and even the heaviest snow isn’t stuck to anything.

About a foot of snow covering a porch with an outdoor table and chairs
Unsquished snow

If we could all fix technical debt as soon as we incur it, that would be amazing. But more likely, someone had to go get milk, or go to the airport, or deploy for testing or a major client, and the snow has been driven on and something has been changed in the code.

A snowy driveway with tire tracks and footprints on it.
Driveway in the process of glaciation

Now that compressed snow/ice technical debt is something that’s going to take real effort to deal with, and the longer you leave it, the more likely it is to get driven on again and coded against again, and get even worse. In snow-shovelling, we have a tool for that:

Wooden-handled tool with a hoe-shaped metal cutting edge, scraping compacted snow off a driveway

I call this an ice-chopper. If it has a proper name, I don’t know it. At the end of a long wooden handle is a metal head that would look like a hoe, if a hoe didn’t have any bend in it. The way you use it is you either get under the ice and scrape it off, or, if it’s gotten very bad, you drive the edge down into the ice to break it up, bit by bit. It’s quite a lot of work, and that amount of work relates directly to how often you do it and how often the snow gets away from you, like a codebase.

Some things are beyond our control, like plow berms, which are the compacted ridges at the end of the driveway caused by the street plows pushing the excess snow to the edges of the road, which are, by definition, the end of our driveways. That’s where the worst of the ice is.

Ice chopper lifting up 3 cm slabs of broken ice.

There are more and less favorable conditions for removing technical debt. Like it’s hard and risky to do it right before a big launch, because you don’t know what all the load pressures will be. For shoveling, cold days are actually best, because the ice and snow are so dry that they are more brittle. I try to maximize for that, and for driving, and for one other factor….

I was shoveling this afternoon because it’s going to snow 6-9 inches tonight. It is better for the rest of my winter to clear the driveway down to bare concrete today, because then, when I go to take the next snowfall off, I won’t have any glaciers, or slippery spots, or high areas that can catch my shovel. Shoveling off a fresh fall will be work, but it won’t have hidden, dangerous, and complicated parts, because I just removed them all, while I could still see them.

Much like software projects, winter in Minnesota is long, and we can’t count on favorable conditions. It’s entirely possible we won’t have a day above freezing for the next month. So even though it was 5 degrees Farenheit, addressing my technical debt will pay off for me.

Mostly looks like flat snow.
An almost-pristine driveway
One shoveled path through 6+ inches of snow
Start where you are
A shoveled driveway, by the light of the garage door
Do what you can

Thank you for indulging my extended metaphor, and do think about when you want to clear your technical debt. You can’t help incurring it – it’s the cost of making things, much like student debt is the cost of higher education for most Americans. But you can think about when it’s useful to remove it, and be mindful of your tools and needs.

Speaking of tools, a side note.

Orange/deerskin colored suite mittens with knitted black and white mittens that fit in them
Utility is my favorite beauty

These are choppers that came from the local farm-supply store. They are a very old form-factor, and very efficient. The knitted mittens go next to your skin. The suede-y deerskin mittens go over them. They are both, ideally, a little oversized so there is lots of trapped air insulating your hands. The deerskin (or moosehide or caribou, depending on where you are) protects your hands from the wind and traps the air inside. The woolen knitted mittens trap more air and provide warmth and insulation. When you’ve been working hard, and have broken the Arctic prohibition (DO NOT SWEAT), you can come inside and take the pieces apart and they will dry/air out much better than any combined/technical/nylon wondermitten would.

Why on earth would developers WANT to write documentation?

This week I was at DevOpsDays Minneapolis, which was an amazing, warm, and inclusive event. There were at least three open spaces on documentation, communication, and related problems, and I didn’t propose any of them!

Minneapolis DevOpsDays 2017 logo

The question the devops world keeps asking, over and over again is,

How do I get developers to write anything?

There aren’t a lot of easy answers for this.

The first one is that if you care about developer-produced documentation, you should hire for it. But almost always, the people hiring are not the same as the people feeling the pain of missing documentation, so that may not happen.

The second is that you should hire someone other than the developers to write, if that’s important to you. I’m going to mention that all the times I’ve interviewed at Google, it was to be as an internal systems writer, documenting how things in the company work for other people in the company. Big, smart, companies hire writers for internal documents. But that is a tough budgetary sell.

The third, partial, answer, is that you make it significantly easier for developers to write. You give them forms and templates to fill in instead of expecting them to invent documentation theory. You specifically allocate writing time that can’t be budged for “one more tweak to the feature”.

Here’s the key problem though:

No developer has ever been fired for lack of documentation.

No developer I know of has gotten a bonus for documentation.

There is no stick. There is no carrot. There are a lot of competing demands on a developer’s time. Why on earth would they go do something that is not in their core competency just because you want it?

My instinctive response, whenever my kids want something, is to say, “It’s good to want things.” I think that is the instinctive response of most developers who are asked to do extra work. It’s not malicious, it’s just true. There is no reason they should do it – they won’t lose their jobs, the way they might if they prioritized writing over code. They won’t derive any benefit except the long-term and intangible one of being asked fewer questions. If they like being the source of knowledge, they’d be giving that away. If they like coding, you’re asking them to do something other than what they want. There is no upside.

So knowing that, how can we encourage developers to write documentation that we need?

First, reward it. Give people prizes for writing docs, or for throwing away or revising old docs. This needs to be something outside their usual compensation structure. Gift cards, time off, merit badges, whatever would be meaningful to that person. Ask them. They know what rewards them, because they use it to reward themselves.

A variety of rewards

Second, require it. I don’t mean “You have to write some docs or I will, uh, be disappointed in you.” I mean “The presence of documentation is a quality gate and you cannot release without it.” I mean “QA is using your docs as the test story.”.

Companies are never altruistic, humans are never purely motivated. We need to accept this and stop expecting people to do extra work because it’s the right thing to do, and start making the work not-extra, and the reward tangible.

Documentation Debt

It was a good question, and a lot of people jumped in to answer it. I’ve given a talk on avoiding product design debt — That’s what #7fights is. But I haven’t given one explicitly about avoiding debt in documentation.

To understand the problem of avoiding debt, you have to understand the economy you’re working in. Many of the companies I’ve worked with over the years identify documentation as something they need to do to reduce support costs and manage the problem of training internal people. It’s not exactly glamorous. Some companies do value and invest in docs, but it’s a minority. That means that documentation is constantly underfunded, and depending on how much discipline people have about retiring old software products, it means that writers are supporting an increasingly large corpus of text. I’ve worked on projects that were literally thousands of pages long.

The debt comes down to three parts:

  • Format
  • Legacy
  • Unknown unknowns

Format

Format is whatever your tool stack is, and how the documentation is packaged and presented. If, many years ago, someone decided you were a FrameMaker shop, well, you’re probably still a FrameMaker shop, albeit one that can now output “FrameML” or DITAFrame. But the soul of the documentation, the way it is stored is in a proprietary file format that only the priestly class of technical writers can access with relatively expensive tools.

The only way to move something out of a format like that is to strip it to plain text and then re-code all the meta-text like headings and pictures. We call it a conversion project, but it’s just as much fun as changing any other codebase on the fly while trying to still deliver things. Theoretically, there are conversion tools and companies that make that less painful, but I have never seen one work properly, and I’ve been through at least 7 conversion projects. Wailing, gnashing of teeth, and a permanent repetitive stress injury were the usual style of results.

Some parts of the industry, notably Read the Docs, Corilla, Gnome/Mallard, and other open-source-y people have been trying to treat this problem by using extremely flat files, in Sphinx or Markdown. These are essentially text files with very minimal markup that can then be parsed by a variety of different engines. It’s a good theory, but it moves the locus of control from the software knowing how to wrap text and manage hyperlinks to the writer needing to know how to do so using a constrained palette of commands. Thus, the markdown technical writer needs to know their product, and the basics of technical communication, and ALSO a syntax, instead of an interface. Not really ONE syntax, but several, with nuances and subtleties between them. The writer must be a coder.

I hang out with a lot of writer/coders, but I also keep in touch with the rest of the industry, and not everyone wants to do or be that person. They want to put more of their career development into writing and understanding their product, less into the means of production.

The debt here is the very nature of everything that is around your text. The storage, the format, the numbering, the images (oh, IMAGES, the worst kind of debt), they all contribute to what you will have to pay down.

Legacy

Aside from the problem of legacy formats, you have the problem of legacy information. In 5 years, almost all of what I am writing now will be wrong or outdated. That’s just the nature of things. But if we continue to support the software, then we need to continue to support the documentation that makes the software work. This means that in an older company, you may be maintaining documentation for products that are fifteen years old. And it is maintenance. You have to update them to have the new format, new branding, whatever you change for the new documents has to be projected back to the old documents. If you move from FrameMaker to MadCap Flare, you have a conversion problem again. And it’s not like you can just ignore it and republish old PDFs – there WILL be a reason you have to get into that document to fix something.

I really recommend making your company and product name a variable that you can easily and universally replace. Just trust me on that. It will happen.

So the older your company is, the more documentation debt you have, just like with a codebase that is several layers of abstraction balanced on an OS/390 system kept in the basement.

Unknown Unknowns

The worst kind of debt is the kind that you can’t accurately predict, but when I thought about just leaving my list two items long, I realized that this third type is the kind that has gotten me in the most trouble over my career. I can predict how long it will take to convert x-many pages from one format to another, and what percentage of my time we will have to spend updating dead documents with new logos. What I can’t predict are the documentation debts that come out of left field. The merger that brings you a new set of docs and tooling and writers to integrate. The drop-everything crisis response to a serious security breach. The training that no one realized needed to be written. Those things are almost impossible to reject, and yet they can scramble a tidy documentation roadmap all to hell.

Mitigating Documentation debt

Here are my suggestions for reducing your documentation debt as much as possible. None of them will completely rescue you from the problem, but they are all things I’ve either implemented or tried to talk people into for debt reduction.

Portability

Whatever tool you pick, make sure that it exports your intellectual property in a usable format — you don’t want PDFs, you want text, or XML, or in a pinch, HTML. Have you ever had that frustrating moment when someone sends you a Photoshop file and not a PNG and you can’t open it because you don’t have freakin’ expensive Photoshop?

Yeah, don’t end up feeling like that about your documentation. You must always be able to extract something that could be reduced to text by means of a simple script. Your software can store it in a weird relational database for all you care, as long as there is an exit scenario.

Referencing

Rigidly use universal resources for common items. Your logo? There should only be one file location for that, and the software should retrieve it every time it builds the document. That way you don’t have to change it 4 places per document (my life this week), but rather change the file in one place, keep the name the same, and all your branding is refreshed, poof!

The same goes for templates, fonts, product names, and anything else that is going to be true across multiple documents or a large document.

If you’ve chosen software that is not sophisticated enough to handle universal variables and referenced images, do what you can to imitate them. Code it yourself or get some help, but never have local images of your logo scattered throughout your files. That’s just like burning money.

Lifespans

Retire documents. The same way we say, “Hey, IE 7 still technically runs and we know some of you are locked in by third-party software, but you’re out of support”, we need to give our products and our documents a lifespan. “Hey, this document is 8 years old. We are no longer updating it, but you can download the PDF and good luck to you, traveller from the past!” Probably you shouldn’t be that snarky, though. No one using super old software is happy about it, they just have constraints that we don’t in shiny new-MacbookPro-land.*

Images

Images are the most heinous form of debt. Schematics and conceptual drawings are pretty much fine, but screenshots are a personal nightmare. They’re wrong almost instantly, expensive to translate, prone to rebranding, frequently unreadable, and everyone likes to put them in because they find it reassuring to have pictures.

My suggestion is to use as few screenshots as humanly possible, and to give them names that correspond with the page name, NOT the step they appear in. If you’re lucky you’ll be able to re-use them, but numbering with any kind of sequencing is an act of madness, because there will always be insertions and deletions.

But mostly, don’t use screenshots.

If you have to, I’m excited about this idea for automating capture with Python:

Automation

Automate as much as you can. Automate production, using the same build tools as the coders. Automate screen captures if you can. If there’s anything that can be scripted or regularized, do it. This applies to the writing itself. If you can create a template that helps people give you more usable input, it’s worth a couple days of your effort. For example, you could create templates for bug reports, for user-visible changes for release notes, for change requests, for steps.

Never accept a set of typed commands as instructions if you can distribute a script instead. Fingers are fallible.

If you’re looking for inspiration on how to automate documentation and get to the part where you’re only writing the interesting juicy bits, look to API documentation, especially OpenAPI and Readme.io. They are automating away all the tedious parts of the API reference doc creation and giving us more time to write about WHY someone would want to use this call over that one. THat is the ideal of automation – not doing less work, but doing less useless work.


Very few “professional-grade” writing toolsuites work on Macs. Your writer is not any happier about this than you are, but it’s the way it works. That said, if anyone wants to torture me by getting me a Macbook Air that won’t run my software, feel free.


I hope these tips for reducing your documentation debt help you make some good arguments in your organization. I’ll keep thinking on the topic and maybe someday it will be a talk.

The Seven Righteous Fights: Affordance

This is the fifth fight in my series The Seven Righteous Fights. For an introduction, see The Seven Righteous Fights: Overview.

Affordance is what the interface makes easy and obvious. Affordances tell us all sorts of things about the tiny interactions we have with the world, and with software.

Most people talk about the affordances of hardware, which way your USB plug goes, things like that. But software interfaces also offer us affordances, with a big green yes button and a small grey more information button.

Affordance is making the world encourage the behavior you want

This picture is where I crossed the street to go to work. As you can see, the place that the sidewalk goes does not correspond to either the crosswalk button or the crosswalk itself. This is bad for the grass, bad for the mobility impaired, and a nightmare when the snow gets plowed off the sidewalk and in front of the button. I live in Minneapolis, we measure our snow in feet. In the winter, the crossing button can be literally covered by a plow drift.

This is a bad affordance, but sometimes you are doing this with software design, where you are putting the sidewalk “where it belongs” and not where users need it.

The righteous fight in affordance is to figure out how to make it easier to do the right thing than the wrong thing. Sometimes the wrong thing is depositing your check to the wrong account because the app has tiny buttons on your phone. Sometimes the wrong thing is that it’s easy to harass other users without consequence.  Whatever your intentions for the thing you’re building, users will see it a different way.

Privileges

One of the things that I classify as affordance is the privilege level it takes to do something. To test this, perform one of your user’s daily tasks.

Now step your permissions down to the level of ordinary mortals. Oh, is that less fun? Are there more hoops and confirmations and things you need to do? Can you even do the job without advanced permissions? Setting excessive security or blocking on ordinary tasks is an affordance and security anti-pattern. It means that people demand elevated access, and makes them and your organization more vulnerable to accidents and malice.

Task frequency

This is a homunculus. It is a weirdly shaped human because the size of the body parts represents nerve density and sensation, instead of physical appearance.

As developers, we often look at things in terms of the architecture, or where the data is. But the user has a really different idea of what is important, what gets used most. Our software needs to reflect what they are actually doing, not what is most logical to us.

Summary

Affordance is our opportunity to nudge users into patterns that they will find useful, rewarding, and not hideously irritating. We can only take advantage of that opportunity if we are thinking about their needs as well as our own.


This blog post is part 5 of my Seven Fights series. You can hear me give this talk at SpringOne Platform (August 1-4) or Abstractions (August 18-20).

The Seven Righteous Fights: Overview

This is a series derived from my popular talk “The Seven Righteous Fights”. It’s not an exact transcript, but I think having a written form to refer people to greatly increases accessibility and gives me room to expand my thoughts in a way that is not compatible with speaking.

There are seven fights that I have over and over again, whenever I start at a company. The more software you build, the more it’s obvious that there are seven fights that it will alway pay to have.

My background as a writer means that I am trained to perform a lot of analysis and synthesis. If I don’t understand the product from value proposition through implementation, it means that I am not the writer I want to be. I call it being a full-stack writer. The problem with having all this training in analysis is that you can’t exactly turn it off. I don’t stay in my lane well, I can’t notice problem that only fall into the narrow category of “documentation”. They’re all documentation problems, they’re all software problems, they’re all design and usability problems.

I’ve worked for roughly 15 companies and my conclusion is that when we create something new, we tend to make similar mistakes every time. This is probably because we all tend to settle in a comfortable stage of product development. Some people prefer mature products, some people like the very beginning and hate the mature product maintenance. But that preference means we don’t understand pain points that happen at other stages.

I’m an early-stage person. I love writing the first version of documentation. I love that green field and that new product smell. But I have done enough mid-stage stuff to know the emergent consequences of early-stage decisions. And like everything you’ve ever read about compound interest, early mistakes compound until they are enormous.

I was working with a startup that was pushing super hard to get their product ready to demo at a big industry conference. They were stripping off every feature they didn’t absolutely need in order to make sure they made their deadline, and I was helping. We made the deadline, there was documentation, and they got a POC contract… with a company in Brazil.

In the process of rushing to get ready, we had stripped out all of the plans to make the interface labels link to a file we could localize, and we had hand-coded all the labels. Now instead of handing off a single file to be translated into Portuguese (and having standard labels that could have been done with Google Translate, in a pinch), we either had to branch and hand-code Portuguese labels or spend twice as long digging into the original code and fixing what we had hacked. I’d love to tell you we decided to do it right, but you all know too much about software to believe that happened, don’t you?

The compounded interest from not using a best practice of referred labels when we built the user interface cost us five days of developer time to fix the wrong way, and two weeks to fix and rebuild the right way. That’s a relatively common example of things people skip and pay for later.

Leaving important considerations “until we get a working prototype” means that it’s going to be harder and more expensive to retrofit. Having these fights early prevents you from doing the software equivalent of poking chocolate chips into already-baked cookies.

cookies

Next: The Seven Righteous Fights: Localization

If you’d prefer to watch the talk as I delivered it:
Shortest recorded version: At The Lead Developer 2016

Longest recorded version: At SpringOne Platform 2016