It’s Not Just Usability

For years and years, self-fashioned pundits, like, uh, me, have been nattering endlessly about usability, and how important it is to make software usable. Jakob Nielsen has a mathematical formula he’ll reveal to you in exchange for $122 which you can use to calculate the value of usability. (If the expected value of usability is greater than $122, I guess you make a profit.)

I have a book you can buy for a lot less that tells you some of the principles of designing usable software, but there’s no math involved, and you’ll be out the price of the book.

Napster UIIn that book, on page 31, I showed an example from what was, at the time, the most popular software application on Earth, Napster. The main Napster window used buttons to flip between the five screens. Due to a principle in usability called affordance, instead of buttons, it really should have had tabs, which was the point I was trying to make.

And yet, Napster was the most popular software application on Earth.

In an early version of the manuscript, I actually wrote something like “this just goes to show you that usability ain’t all that important,” which was an odd thing to be writing in a book about usability. I was sort of relieved when the typesetter announced that I had to shorten that paragraph. So I deleted the sentence.

But there’s a scary element of truth to it—scary to UI professionals, at least: an application that does something really great that people really want to do can be pathetically unusable, and it will still be a hit. And an application can be the easiest thing in the world to use, but if it doesn’t do anything anybody wants, it will flop. UI consultants are constantly on the defensive, working up improbable ROI formulas about the return on investment clients will get from their $75,000 usability project, precisely because usability is perceived as “optional,” and the scary thing is, in a lot of cases, it is. In a lot of cases. The CNN website has nothing to be gained from a usability consultant. I’ll go out on a limb and say that there is not a single content-based website online that would gain even one dollar in revenue by improving usability, because content-based websites (by which I mean, websites that are not also applications) are already so damn usable.

Anyway.

My goal today is not to whine about how usability is not important… usability is important at the margins, and there a lots of examples where bad usability kills people in small planes, creates famine and pestilence, etc.

My goal today is to talk about the next level of software design issues, after you’ve got the UI right: designing the social interface.

I need to explain that, I guess.

Software in the 1980s, when usability was “invented,” was all about computer-human interaction. A lot of software still is. But the Internet brings us a new kind of software: software that’s about human-human interaction.

Discussion groups. Social networking. Online classifieds. Oh, and, uh, email. It’s all software that mediates between people, not between the human and the computer.

When you’re writing software that mediates between people, after you get the usability right, you have to get the social interface right. And the social interface is more important. The best UI in the world won’t save software with an awkward social interface.

The best way to illustrate social interfaces is with a few examples of failures and successes.

Some Examples

First, a failing social interface. Every week I get an email from somebody I’ve never heard of asking me to become a part of his or her social network. I usually don’t know the person, so I feel a little bit miffed and delete the message. Someone told me why this happens: one of those social networking software companies has a tool that goes through your email address book and sends email to everyone asking them to join in. Now, combine this with the feature that some email software saves the sender’s address of every incoming message, and the feature that when you go to sign up for the Joel on Software email bulletin you get a confirmation message asking if you really want to join, and voila: all kinds of people who I don’t know are running software that is inadvertently asking me to confirm that I’m their friend. Thank you for subscribing to my newsletter, but no, I’m not going to introduce you to Bill Gates. I currently have a policy of not joining any of these social networks, because they strike me as going strongly against the grain of how human networks really work.

Now, let’s look at a successful social interface. Many humans are less inhibited when they’re typing than when they are speaking face-to-face. Teenagers are less shy. With cellphone text messages, they’re more likely to ask each other out on dates. That genre of software was so successful socially that it’s radically improving millions of people’s love lives (or at least their social calendars). Even though text messaging has a ghastly user interface, it became extremely popular with the kids. The joke of it is that there’s a much better user interface built into every cellphone for human to human communication: this clever thing called “phone calls.” You dial a number after which everything you say can be heard by the other person, and vice versa. It’s that simple. But it’s not as popular in some circles as this awkward system where you break your thumbs typing huge strings of numbers just to say “damn you’re hot,” because that string of numbers gets you a date, and you would never have the guts to say “damn you’re hot” using your larynx.

Another social software success is ebay. When I first heard about ebay, I said, “Nonsense! That will never work. Nobody’s going to send money to some random person they encountered on the Internet in hopes that person will out of the goodness of their hearts actually ship them some merchandise.” A lot of people thought this. We were all wrong. Wrong, wrong, wrong. Ebay made a big bet on the cultural anthropology of human beings and won. The great thing about ebay is that it was a huge success precisely because it seemed like a terrible idea at the time, and so nobody else tried it, until ebay locked in the network effects and first-mover advantage.

In addition to absolute success and failures in social software, there are also social software side effects. The way social software behaves determines a huge amount about the type of community that develops. Usenet clients have this big-R command which is used to reply to a message while quoting the original message with those elegant >’s in the left column. And the early newsreaders were not threaded, so if you wanted to respond to someone’s point coherently, you had to quote them using the big-R feature. This led to a particularly Usenet style of responding to an argument: the line-by-line nitpick. It’s fun for the nitpicker but never worth reading. (By the way, the political bloggers, newcomers to the Internet, have reinvented this technique, thinking they were discovering something fun and new, and called it fisking, for reasons I won’t go into. Don’t worry, it’s not dirty.) Even though human beings had been debating for centuries, a tiny feature of a software product produced a whole new style of debating.

Small changes in software can make big differences in the way that software supports, or fails to support, its social goals. Danah Boyd has a great critique of social software networks, Autistic Social Software, blasting the current generation of this software for forcing people to behave autistically:

Consider, for a moment, the recent surge of interest in articulated social networks such as Friendster, Tribe, LinkedIn, Orkut and the like. These technologies attempt to formalize how people should construct and manage their relationships. They assume that you can rate your friends. In some cases, they procedurally direct how people can engage with new people by giving you an absolute process through which you can contact others.

While this approach certainly has its merits because it is computationally possible, i’m terrified when people think that this models social life. It’s so simplistic that people are forced to engage as though they have autism, as though they must interact procedurally. This approach certainly aids people who need that kind of systematization, but it is not a model that universally makes sense. Furthermore, what are the implications of having technology prescribe mechanistic engagement? Do we really want a social life that encourages autistic interactions?

When software implements social interfaces while disregarding cultural anthropology, it’s creepy and awkward and doesn’t really work.

Designing Social Software

Let me give you an example of social interface design.

Suppose your user does something they shouldn’t have done.

Good usability design says that you should tell them what they did wrong, and tell them how to correct it. Usability consultants are marketing this under the brand name “Defensive Design.”

When you’re working on social software, this is too naive.

Maybe the thing that they did wrong was to post an advertisement for Viagra on a discussion group.

Now you tell them, “Sorry, Viagra is not a suitable topic. Your post has been rejected.”

Guess what they’ll do? They’ll post an advertisement for V1agra. (Either that, or they’ll launch into a long boring rant about censorship and the First Amendment.)

With social interface engineering, you have to look at sociology and anthropology. In societies, there are freeloaders, scammers, and other miscreants. In social software, there will be people who try to abuse the software for their own profit at the expense of the rest of the society. Unchecked, this leads to something economists call the tragedy of the commons.

Whereas the goal of user interface design is to help the user succeed, the goal of social interface design is to help the society succeed, even if it means one user has to fail.

So a good social interface designer might say, let’s not display an error message. Let’s just pretend that the post about Viagra was accepted. Show it to the original poster, so he feels smug and moves on to the next inappropriate discussion group. But don’t show it to anyone else.

Indeed one of the best ways to deflect attacks is to make it look like they’re succeeding. It’s the software equivalent of playing dead.

No, it doesn’t work 100% of the time. It works 95% of the time, and it reduces the problems you’ll have twenty-fold. Like everything else in sociology, it’s a fuzzy heuristic. It kind of works a lot of the time, so it’s worth doing, even if it’s not foolproof. The Russian mafia with their phishing schemes will eventually work around it. The idiot Floridians in trailer parks trying to get rich quick will move on. 90% of the spam I get today is still so hopelessly naive about spam filters that it would even get caught by the pathetic junk filter built into Microsoft Outlook, and you’ve got to have really lame spam to get caught by that scrawny smattering of simplistic searchphrases.

Marketing Social Interfaces 

A few months ago I realized that a common theme in the software we’ve built at Fog Creek is an almost obsessive attention to getting the social interfaces right. For example, FogBugz has lots of features, and even more non-features, designed to make bug tracking actually happen. Time and time again customers tell me that their old bug tracking software was never getting used, because it did not align with the way people wanted to work together, but when they rolled out FogBugz, people actually starting using it, and became addicted to it, and it changed the way they worked together. I know that FogBugz works because we have a very high upgrade rate when there’s a new version, which tells me FogBugz is not just shelfware, and because even customers who buy huge blocks of licenses keep coming back for more user licenses as the product spreads around their organization and really gets used. This is something I’m really proud of. Software used in teams usually fails to take hold, because it requires everyone on the team to change the way they work simultaneously, something which anthropologists will tell you is vanishingly unlikely. For that reason FogBugz has lots of design decisions which make it useful even for a single person on a team, and lots of design features which encourage it to spread to other members of the team gradually until everyone is using it.

The discussion group software used on this site, which will soon be for sale as a feature of FogBugz 4.0, is even more obsessed with getting the social interface aspects exactly right. There are dozens of features and non-features and design decisions which collectively lead to a very high level of interesting conversation with the best signal-to-noise ratio of any discussion group I’ve ever participated in. I wrote a lot about this in my article Building Communities with Software.

Since then, I’ve become even more devoted to the idea of the value of good social interface design: we bring in experts like Clay Shirky (a pioneer in the field), we do bold experiments on the poor citizens of the Joel on Software discussion group (many of which are so subtle as to be virtually unnoticeable, for example, the fact that we don’t show you the post you’re replying to while you type your reply in hopes of cutting down quoting, which makes it easier to read a thread), and we’re investing heavily in advanced algorithms to reduce discussion group spam.

A New Field

Social interface design is still a field in its infancy. I’m not aware of any books on the subject; there are only a few people working in the research side of the field, and there’s no organized science of social interface design. In the early days of usability design, software companies recruited ergonomics experts and human factors experts to help design usable products. Ergonomics experts knew a lot about the right height for a desk, but they didn’t know how to design GUIs for file systems, so a new field arose. Eventually the new discipline of user interface design came into its own, and figured out the concepts like consistency, affordability, feedback, etc., which became the cornerstone of the science of UI design.

Over the next decade, I expect that software companies will hire people trained as anthropologists and ethnographers to work on social interface design. Instead of building usability labs, they’ll go out into the field and write ethnographies. And hopefully, we’ll figure out the new principles of social interface design. It’s going to be fascinating… as fun as user interface design was in the 1980s… so stay tuned.

Discuss this topic on the Joel on Software Social Interface Design Forum.

Building Communities with Software

The social scientist Ray Oldenburg talks about how humans need a third place, besides work and home, to meet with friends, have a beer, discuss the events of the day, and enjoy some human interaction. Coffee shops, bars, hair salons, beer gardens, pool halls, clubs, and other hangouts are as vital as factories, schools and apartments [“The Great Good Place”, 1989]. But capitalist society has been eroding those third places, and society is left impoverished. In “Bowling Alone,” Robert Putnam brings forth, in riveting and well-documented detail, reams of evidence that American society has all but lost its third places. Over the last 25 years, Americans “belong to fewer organizations that meet, know our neighbors less, meet with friends less frequently, and even socialize with our families less often.” [2000] For too many people, life consists of going to work, then going home and watching TV. Work-TV-Sleep-Work-TV-Sleep. It seems to me that the phenomenon is far more acute among software developers, especially in places like Silicon Valley and the suburbs of Seattle. People graduate from college, move across country to a new place where they don’t know anyone, and end up working 12 hour days basically out of loneliness.

So it’s no surprise that so many programmers, desperate for a little human contact, flock to online communities – chat rooms, discussion forums, open source projects, and Ultima Online. In creating community software, we are, to some extent, trying to create a third place. And like any other architecture project, the design decisions we make are crucial. Make a bar too loud, and people won’t be able to have conversations. That makes for a very different kind of place than a coffee shop. Make a coffee shop without very many chairs, as Starbucks does, and people will carry their coffee back to their lonely rooms, instead of staying around and socializing like they do in the fantasy TV coffeehouse of “Friends,” a program we watch because an ersatz third place is less painful than none at all.

In software, as in architecture, design decisions are just as important to the type of community that develops or fails to develop. When you make something easy, people do it more often. When you make something hard, people do it less often. In this way you can gently encourage people to behave in certain ways which determine the character and quality of the community. Will it feel friendly? Is there thick conversation, a European salon full of intellectuals with interesting ideas? Or is the place deserted, with a few dirty advertising leaflets lying around on the floor that nobody has bothered to pick up?

Look at a few online communities and you’ll instantly notice the different social atmosphere. Look more closely, and you’ll see this variation is most often a byproduct of software design decisions.

On Usenet, threads last for months and months and go off onto so many tangents that you never know where they’ve been. Whenever a newbie stumbles by and asks a germane question, the old timers shout him down and tell him to read the FAQ. Quoting, with the “>” symbol, is a disease that makes it impossible to read any single thread without boring yourself to death by re-reading the whole history of a chain of argument which you just read in the original, seconds ago, again and again and again. Shlemiel the Painter reading.

On IRC, you can’t own your nickname and you can’t own a channel — once the last person leaves a room, anyone can take it over. That’s the way the software works. The social result was that it was often impossible to find your friends when you came back the next day, because someone else might have locked you out of your chatroom and your friends might have been forced to choose different nicknames. The only way to prevent gay bashers in Perth, Australia from taking over gay chat channels when the boys went to sleep was to create a software robot to hang around 24 hours a day and guard the channel. Many IRC participants put more effort into complicated bot-wars, attempts to take over channels, and general tomfoolery than actually having a conversation, often ruining things for the rest of us.

On most investment discussion boards, it’s practically impossible to follow a thread from beginning to end, because every post is its own page, which makes for a lot of banner ad inventory, but the latency in reading a conversation will eventually drive you nuts. The huge amount of flashing commercial crap on all four sides of the conversation makes you feel like you were trying to make friends in Times Square, but the neon lights keep demanding all the attention.

On Slashdot, every thread has hundreds of replies, many of which are identical, so the conversation there feels insipid and stupid. In a moment I’ll reveal why Slashdot has so many identical replies and the Joel on Software forum doesn’t.

And on FuckedCompany.com, the discussion board is completely, utterly worthless; the vast majority of posts consist of irrelevant profanity and general abusiveness and it feels like a fraternity rudeness contest, without any fraternity.

So, we have discovered the primary axiom of online communities:

Small software implementation details result in big differences in the way the community develops, behaves, and feels.

IRC users organize themselves around bot warfare because the software doesn’t let you reserve a channel. Usenet threads are massively redundant because the original Usenet reader, “rn,” designed for 300 baud modems, never shows you old posts, only new ones, so if you want to nitpick about something someone said, you had to quote them or your nitpick won’t make sense.

With that in mind, I’d like to answer the most common questions about the Joel on Software forum, why it was designed the way it was designed, how that makes it work, and how it could be improved.

Q. Why is the software so dang simplistic?

A. In the early days of the Joel on Software forum, achieving a critical mass to get the conversation off the ground was important to prevent the empty restaurant phenomenon (nobody goes into an empty restaurant, they’ll always go into the full one next door even if it’s totally rubbish.) Thus a design goal was to eliminate impediments to posting. That’s why there’s no registration and there are literally no features, so there’s nothing to learn.

The business goal of the software that runs the forum was to provide tech support for Fog Creek’s products. That’s what paid for the development. To achieve that goal, nothing was more important than making the software super simple so that anyone could be comfortable using it. Everything about how the forum works is incredibly obvious. I don’t know of anyone who hasn’t been able to figure out how to use it immediately.

Q. Could you make a feature where I check a box that says “email me if somebody replies to my post?”

A. This one feature, so easy to implement and thus so tempting to programmers, is the best way to kill dead any young forum. Implement this feature and you may never get to critical mass. Philip Greenspun’s LUSENET has this feature and you can watch it sapping the life out of young discussion groups.

Why?

What happens is that people go to the group to ask a question. If you offer the “notify me” checkbox, these people will post their question, check the box, and never come back. They’ll just read the replies in their mailbox. The end.

If you eliminate the checkbox, people are left with no choice but to check back every once in a while. And while they’re checking back, they might read another post which looks interesting. And they might have something to contribute to that post. And in the critical early days when you’re trying to get the discussion group to take off, you’ve increased the “stickiness” and you’ve got more people hanging around, which helps achieve critical mass a lot quicker.

Q. OK, but can’t you at least have branching? If someone gets off on a tangent, that should be its own branch which you can follow or go back to the main branch.

A. Branching is very logical to a programmer’s mind but it doesn’t correspond to the way conversations take place in the real world. Branched discussions are disjointed to follow and distracting. You know what I find distracting? When I’m trying to do something on my bank’s web site and the site is so slow I can’t remember what I’m doing from one click to the next. That reminds me of a joke. Three old ladies talking. Lady 1: “I’m so forgetful the other day I was on the steps to my apartment with a bag, and I couldn’t remember if I was taking out the trash or going upstairs with the groceries.” Lady 2: “I’m so forgetful I was in my car in the driveway and I couldn’t remember if I was coming home or going to shul.” Lady 3: “Thank God, I still have my memory, clear as a bell, knock on wood. (knock knock knock). Come in, door’s open!” Branching makes discussions get off track, and reading a thread that is branched is discombobulating and unnatural. Better to force people to start a new topic if they want to get off topic. Which reminds me…

Q. Your list of topics is sorted wrong. It should put the topic with the most recent reply first, rather than listing them based on the time of the original post.

A. It could do that; that’s what many web-based forums do. But when you do that certain topics tend to float near the top forever, because people will be willing to argue about H1B visas, or what’s wrong with Computer Science in college, until the end of the universe. Every day 100 new people arrive in the forum for the first time, and they start at the top of the list, and they dive into that topic with gusto.

The way I do it has two advantages. One, topics rapidly go away, so conversation remains relatively interesting. Eventually people have to just stop arguing about a given point.

Two, the order of topics on the home page is stable, so it’s easier to find a topic again that you were interested in because it stays in the same place relative to its neighbors.

Q. Why don’t you have some kind of system so I can see what posts I’ve already read?

A. We have the best system that can be implemented in a distributed, scalable fashion: we let everyone’s browser keep track of it. Web browsers will change the color of the links you’ve already visited from blue to purple. So all we have to do is subtly change the URL for each topic to include the number of replies available; that way when there are additional replies the post will appear in the “unread” color again.

Anything more elaborate than this would be harder to build and would needlessly complicate the UI.

Q. The damn “reply” link is all the way at the bottom. This is a usability annoyance because you have to scroll all the way to the bottom.

A. This is intentional. I would prefer that you read all the posts before you reply, otherwise you may post something which is repetitive or which sounds disjointed coming after the previous last post. Of course, I can’t physically grab your eyeballs and move them from left to right, forcing you to read the entire thread before letting you post, but if I put a “reply” link anywhere but the bottom of the page that would positively encourage people to spew their little gems before they’ve read what’s already there. This is why Slashdot topics have 500 replies but only 17 interesting replies, and it’s why nobody likes to read Slashdot discussions: they sound like a classroom full of children all shouting out the same answer at the same time. (“ha ha … Bill Gates! That’s an oxymoron!”)

Q. The damn “Start a New Topic” link is all the way at the bottom…

A. Uh huh, same thing.

Q. Why don’t you show people their posts to confirm them before you post them? Then people wouldn’t make mistakes and typos.

A. Empirically, that is not true. Not only is it not true, it’s the opposite of true.

Part one: when you have a confirmation step, most people just click past it. Very few people reread their post carefully. If they wanted to reread their post carefully, they could have done it while they were editing it, but they are bored by their post already, it’s yesterday’s newspaper, they are ready to move on.

Part two: the lack of the confirmation step actually makes people more cautious. It’s like those studies they did that showed that it’s safer, on twisty mountain roads, to remove the crash barrier, because it makes people scared and so they drive more carefully, and any way, that little flimsy aluminum crash barrier ain’t gonna stop a 2 ton SUV moving at 50 mph from flying off the cliff. You’re better off, statistically, just scaring the bejesus out of drivers so they creep along at 2 miles per hour around the hairpins.

Q. Why don’t you show me the post I’m replying to, while I compose my reply?

A. Because that will tempt you to quote a part of it in your own reply. Anything I can do to reduce the amount of quoting will increase the fluidity of the conversation, making topics interesting to read. Whenever someone quotes something from above, the person who reads the topic has to read the same thing twice in a row, which is pointless and automatically guaranteed to be boring.

Sometimes people still try to quote things, usually because they are replying to something from three posts ago, or because they’re mindlessly nitpicking and they need to rebut 12 separate points. These are not bad people, they’re just programmers, and programming requires you to dot every i and cross every t, so you get into a frame of mind where you can’t leave any argument unanswered any more than you would ignore an error from your compiler. But I’ll be damned if I make it EASY on you. I’m almost tempted to try to find a way to show posts as images so you can’t cut and paste them. If you really need to reply to something from three posts ago, kindly take a moment to compose a decent English sentence (“When Fred said blah, he must not have considered…”), don’t litter the place with your <<<>>>s.

Q. Why do posts disappear sometimes?

A. The forum is moderated. That means that a few people have the magick powah to delete a post. If the post they delete is the first one in a thread, the thread itself appears deleted because there’s no way to get to it.

Q. But that’s censorship!

A. No, it’s picking up the garbage in the park. If we didn’t do it, the signal to noise ratio would change dramatically for the worse. People post spam and get rich schemes, people post antisemitic comments about me, people post nonsense that doesn’t make any sense. Some idealistic youngsters may imagine a totally uncensored world as one in which the free exchange of intelligent ideas raises everyone’s IQ, an idealized Oxford Debate Society or Speakers’ Corner. I am pragmatic and understand that a totally uncensored world just looks like your inbox: 80% spam, advertising, and fraud, rapidly driving away the few interesting people.

If you are looking for a place to express yourself in which there will be no moderation, my advice to you would be to (a) create a new forum and (b) make it popular. [Apologies to Larry Wall].

Q. How do you decide what to delete?

A. First of all, I remove radically off-topic posts or posts which, in my opinion, are only of interest to a very small number of people. If something is not about the same general topics as Joel on Software is about, it may be interesting as all heck to certain people but it’s not likely to interest the majority of people who came to my site to hear about software development.

My policy in the past has been that “off topic” includes any discussion of the forum itself, its design or usability. There’s a slightly different reason for this, almost another axiom. Every forum, mailing list, discussion group, and BBS will, all else being equal, lapse into conversations about the forum itself every week or two. Literally once a week somebody strolls in and announces his list of improvements to the forum software which he demands be made right away. And then somebody says, “look buddy you’re not paying for it Joel’s doing us a favor get lost.” And somebody else says “Joel’s not doing this out of the goodness of his heart it’s marketing for Fog Creek.” And it’s just SOOOO BORING because it happens EVERY WEEK. It’s like talking about the weather when you have nothing else to talk about. It may be exciting to the new person who just appeared on the board but it is only barely about software development, so, as Strong Bad says, “DELETED”. Unfortunately what I have learned is that trying to get people to stop talking about the forum is like trying to stop a river. But please, if you’re reading this article and you want to discuss it on the forum, please, please, do me a huge favor, and resist the urge.

We will delete posts which are personal, ad hominem attacks on non public personalities. I better define that. Ad hominem means it is an attack on the individual, rather than on his ideas. If you say “that is a stupid idea because…” it’s OK. If you say “you are stupid” then it’s an ad hominem attack. If it’s vicious or uncivil or libelous, I delete it. There’s one exception: because the Joel on Software forum is the best place to criticize Joel, vicious or uncivil posts about Joel are allowed to stand but only if they contain some tiny sliver of a useful argument or idea.

I automatically delete posts which comment on the spelling or grammar of a previous poster. We’ll be talking about interviews and someone will say, “It’s a wonder you can get a job with spelling like that.” It’s just super boring to talk about other people’s spelling. SUPER, SUPER boring.

Q. Why don’t you just post the rules instead of leaving it as a mystery?

A. The other day I was taking the train from the Newark Airport back to Manhattan. Besides being in general disrepair, the only thing to read was a large sign that explained very sternly and in great detail that if you misbehaved, you would be put off the train at the next stop and the police would be summoned. And I thought, 99.99999% of the people who read that sign ain’t gonna be misbehavin’, and the misbehavors couldn’t care less what the sign says. So the net result of the sign is to make honest citizens feel like they’re being accused of something, and it doesn’t deter the sociopaths at all, and it just reminds the good citizens of New Jersey endlessly that they’re in Newark, Crime Capital, where sociopaths get on the train and do Unpleasant Things and Make a Scene and have to be Put Off and the Police Summoned.

Almost everyone on the Joel on Software forum, somehow, was born with the part of the brain that tells them that it’s not civilized to post vicious personal attacks, or to post questions about learning French on a software forum, or to conduct an argument by criticizing someone’s spelling. And the other .01% don’t care about the rules. So posting rules is just a way to insult the majority of the law-abiding citizens and it doesn’t deter the morons who think their own poo smells delicious and nothing they post could possibly be against the rules.

When you address troublemakers in public, everyone else thinks you’re paranoid or feels angry at being scolded when they did nothing wrong. It’s like being in grade school again, and one idiot-child has broken a window, and now everyone has to sit there listening to the teacher giving the whole class a stern lecture on why you mustn’t break windows. So any public discussion of why a particular post got deleted, for example, is taboo.

Q. Instead of deleting posts, why don’t you have a moderation scheme, where people vote on how much they like a post, and people can choose how high the vote has to be before they read it?

A. This is, of course, how Slashdot works, and I’ll bet you 50% of the people who read Slashdot regularly have never figured it out.

There are three things I don’t like about this. One: it’s more UI complication, a feature that people need to learn how to use. Two: it creates such complicated politics that it make the Byzantine Empire look like 3rd grade school government. And three: when you read Slashdot with the filters turned up high enough that you only see the interesting posts, the narrative is completely lost. You just get a bunch of random disjointed statements with no context.

Q. Why don’t you have a registration scheme to eliminate rude posters?

A. As I explained earlier, the goal of the forum is to make it easy to post. (Remember, the software was written for tech support.) Registration schemes eliminate at least 90% of the people who might have posted, and in a tech support scenario, those 90% are going to call my toll free number.

Besides, I don’t think registration would help. If somebody is being abusive, it doesn’t help to ban them, they can trivially reregister. The idea of improving the community by requiring registration is an old one, and it’s appropriate, I think, for the Echo/Well type of conferences where you’re creating a network of people as much as you’re discussing a topic, and you charge people cash money to belong.

But requiring registration does NOT improve the quality of the conversation or the average quality of the participants. If you look closely at the signal-to-noise ratio on the Joel on Software forum, you might start to notice that the noisiest people (i.e. the people who post the most words while contributing the fewest ideas) are often the long time, hard core members who visit the forum every ten minutes. These are the people who feel the need to chime in with a “I agree with that” and replies to Every Single Topic even when they haven’t got an original thought to contribute. And they would certainly register.

Q. Any plans for the future?

A. Working on the software for the discussion forum is not a priority for me or my company: it’s good enough, it works, it has created an interesting place to talk about hard computer management problems and get ideas from some of the smartest people in the world. And I’ve got too many better things to work on. Somebody else can create the next big leap in usability for discussion forums.

I just created a New York City forum, to see if geographically based forums encourage people to get to know each other in person as well as online. In my experience, regionally- based communities cause the community to take a giant leap from a simple website to a real society, a true third place.

Creating community, in any case, is a noble goal, because it’s sorely missing for so many of us. Let’s keep plugging away at it.

Nothing is as Simple as it Seems

We had a little usability problem in CityDesk.

Here was the problem: you could import files from the web using a menu command (“Import Web Page”). And you could import files from a disk into CityDesk by dragging them with the mouse. But there was no menu command to import files from a disk. So either people didn’t discover that it was possible, or people tried to use the Import Web Page feature to import from disk, which didn’t work right.

I thought that it would be easy to fix with a two page wizard. Roughly speaking, page one of the wizard would ask you “Where do you want to import from?” If you chose “disk,” page two would prompt you for a file. If you chose “web,” page two would prompt you for a URL.

I almost started implementing this, but something stopped me, and instead, I started to write a mini-spec. Here’s the spec, in its entirety:

Page One
Where do you want to import from? (Disk/Web)

Page Two (Disk)
Standard File/Open dialog

Page Two (Web)
URL prompt with mini-web-browser

Suddenly something occurred to me. Can you put the Windows standard file open dialog, which is usually supplied in toto by the OS, into a wizard?

Hmm.

I investigated. Yes, you can, but it’s no fun and takes a few hours of work. How could I make this not be a wizard? I rewrote the spec:

Two Menu Items:
1) Import Web Page From Internet -> Pops Up URL Dialog
2) Import Web Page From Disk -> Pops Up File Open Dialog

Much better. Three minutes of design work saved me hours of coding.

If you’ve spent more than 20 minutes of your life writing code, you’ve probably discovered a good rule of thumb by now: nothing is as simple as it seems.

Something as simple as copying a file is full of perils. What if the first argument is a directory? What if the second argument is a file? What if a file with the same name already exists in the destination? What if you don’t have write permission?

What if the copy fails in the middle? What if the destination is on a remote machine which is available, but which requires authentication to continue? What if the files are large and the link is slow and you need to show a progress indicator? What if the transfer speed slows down to almost zero… when do you give up and return an error message?

A good way to interview candidates for testing jobs is to give them a simple operation and ask them to enumerate all the things that can possibly go wrong. A classic Microsoft test interview question: how do you test the File Open dialog box? A good tester will be able to rattle off several dozen weird things to test (“file is deleted by another user between the time it is listed in the box and the time you select it and click Open“).

OK, so we have one axiom: nothing is as simple as it seems.

There’s another axiom in software engineering, which is, always try to reduce risk. One particularly important piece of risk to avoid is schedule risk, when something takes longer than expected. Schedule risk is bad because your boss yells at you, which makes you unhappy. If that’s not enough motivation for you, the economic reason why schedule risk is bad is because you decided to do a feature based on information that it would take 1 week. Now that you realize that it has taken 20 weeks to accomplish, that decision might well have been wrong. Perhaps if you knew it was going to take 20 weeks, you would have made a different decision. The more wrong decisions you make, the more likely all those tote bags with your company logo will end up in the liquidator’s warehouse while your ex-CEO mopes that “what sucks is, we weren’t even successful enough to get mentioned on fuckedcompany when we shut down!”

The combination of nothing-is-as-simple-as-it-seems and reduce-risk can only lead you to one conclusion:

You have to design things before you implement them.

I’m sorry to disappoint you. Yeah, I know, you read Kent Beck and now you think it’s OK to not design things before you implement them. Sorry, it’s not OK. You can not change things in code “just as easily” as you could change them in the design documents. People say this all the time and it’s wrong. “We use high-level tools these days, like Java and XML. We can change things in minutes in the code. Why not design it in code?” My friend, you can put wheels on your mama but that doesn’t make her a bus, and if you think you can refactor your wrongly-implemented file-copy function to make it preemptive rather than threaded as quickly as I could write that sentence, you’re in deep denial.

Anyway, I don’t think Extreme Programming really advocates zero design. They just say “don’t do any more design than needed,” which is fine. But that’s not what people hear. Most programmers are looking for any excuse they can find not to do basic design before implementing features. So they latch onto the “no-design” idea like flies in a bug zapper. Dzzzzzzt! It’s one of those weird forms of laziness where you end up doing more work than you would have done otherwise. I’m too lazy to design the feature on paper first, so I just write some code, and then it’s not right, so I have to fix it, and I spend more time than I would have otherwise. Or, more commonly, I write some code, and then it’s not right, but it’s too late, and my product is inferior and I spend until the end of time making up excuses for why it “has to be that way.” It’s just sloppy and unprofessional.

When Linus Torvalds bashes design, he’s talking about huge systems, which have to evolve, or they become Multics. He’s not talking about your File Copy code. And when you consider that he had a pretty clear road map of exactly where he was going, it’s no wonder Linus doesn’t see much value in design. Don’t fall for it. Chances are it doesn’t apply to you. And anyway, Linus is much smarter than we are, so things that work for him don’t work for us normal people.

Incremental design and implementation is good. Frequent releases are fine (although for shrink-wrapped or mass market software, it drives customers crazy, never a good idea — instead do frequent internal milestones.) Too much formality in design is a waste of time — I’ve never seen a project benefit from mindless flowcharting or UMLing or CRCing or whatever the flavor-du-jour is. And those huge 10 million lines-of-code behemoth systems Linus is talking about should evolve, because humans don’t really know how to design software on that scale.

But when you sit down to write File Copy, or when you sit down to plan the features of the next release of your software, you gotta design. Don’t let the sirens persuade you otherwise.

Come argue with me about this in person at the Cutter Summit, April 29-May 1st, in Cambridge, MA, where I’ll be on a panel with Tom DeMarco, Kent Beck, and others.

The Iceberg Secret, Revealed

“I don’t know what’s wrong with my development team,” the CEO thinks to himself. “Things were going so well when we started this project. For the first couple of weeks, the team cranked like crazy and got a great prototype working. But since then, things seem to have slowed to a crawl. They’re just not working hard any more.” He chooses a Callaway Titanium Driver and sends the caddy to fetch an ice-cold lemonade. “Maybe if I fire a couple of laggards that’ll light a fire under them!”

Meanwhile, of course, the development team has no idea that anything’s wrong. In fact, nothing is wrong. They’re right on schedule.

Don’t let this happen to you! I’m going to let you in on a little secret about those non-technical management types that will make your life a million times easier. It’s real simple. Once you know my secret, you’ll never have trouble working with non-technical managers again (unless you get into an argument over the coefficient of restitution of their golf clubs).

It’s pretty clear that programmers think in one language, and MBAs think in another. I’ve been thinking about the problem of communication in software management for a while, because it’s pretty clear to me that the power and rewards accrue to those rare individuals who know how to translate between Programmerese and MBAese.

[Image]

Since I started working in the software industry, almost all the software I’ve worked on has been what might be called “speculative” software. That is, the software is not being built for a particular customer — it’s being built in hopes that zillions of people will buy it. But many software developers don’t have that luxury. They may be consultants developing a project for a single client, or they may be in-house programmers working on a complicated corporate whatsit for Accounting (or whatever it is you in-house programmers do; it’s rather mysterious to me).

Have you ever noticed that on these custom projects, the single most common cause of overruns, failures, and general miserableness always boils down to, basically, “the (insert expletive here) customer didn’t know what they wanted?”

Here are three versions of the same pathology:

  1. “The damn customer kept changing his mind. First he wanted Client/Server. Then he read about XML in Delta Airlines Inflight Magazine and decided he had to have XML. Now we’re rewriting the thing to use fleets of small Lego Mindstorms Robots.”
  2. “We built it exactly the way they wanted. The contract specified the whole thing down to the smallest detail. We delivered exactly what the contract said. But when we delivered it, they were crestfallen.”
  3. “Our miserable sales person agreed to a fixed price contract to build what was basically unspecified, and the customer’s lawyers were sharp enough to get a clause in the contract that they don’t have to pay us until ‘acceptance by customer,’ so we had to put a team of nine developers on their project for two years and only got paid $800.”

If there’s one thing every junior consultant needs to have injected into their head with a heavy duty 2500 RPM DeWalt Drill, it’s this: Customers Don’t Know What They Want. Stop Expecting Customers to Know What They Want. It’s just never going to happen. Get over it.

Instead, assume that you’re going to have to build something anyway, and the customer is going to have to like it, but they’re going to be a little bit surprised. YOU have to do the research. YOU have to figure out a design that solves the problem that the customer has in a pleasing way.

Put yourself in their shoes. Imagine that you’ve just made $100,000,000 selling your company to Yahoo!, and you’ve decided that it’s about time to renovate your kitchen. So you hire an expert architect with instructions to make it “as cool as Will and Grace’s Kitchen.” You have no idea how to accomplish this. You don’t know that you want a Viking stove and a Subzero refrigerator — these are not words in your vocabulary. You want the architect to do something good, that’s why you hired him.

The Extreme Programming folks say that the solution to this is to get the customer in the room and involve them in the design process every step of the way, as a member of the development team. This is, I think, a bit too “extreme.” It’s as if my architect made me show up while they were designing the kitchen and asked me to provide input on every little detail. It’s boring for me, if I wanted to be an architect I would have become an architect.

Anyway, you don’t really want a customer on your team, do you? The customer-nominee is just as likely to wind up being some poor dweeb from Accounts Payable who got sent to work with the programmers because he was the slowest worker over there and they would barely notice his absence. And you’re just going to spend all your design time explaining things in words of one syllable.

Assume that your customers don’t know what they want. Design it yourself, based on your understanding of the domain. If you need to spend some time learning about the domain or if you need a domain expert to help you, that’s fine, but the design of the software is your job. If you do your domain homework and create a good UI, the customer will be pleased.

Now, I promised to tell you a secret about translating between the language of the customers (or nontechnical managers) of your software and the language of programmers.

You know how an iceberg is 90% underwater? Well, most software is like that too — there’s a pretty user interface that takes about 10% of the work, and then 90% of the programming work is under the covers. And if you take into account the fact that about half of your time is spent fixing bugs, the UI only takes 5% of the work. And if you limit yourself to the visual part of the UI, the pixels, what you would see in PowerPoint, now we’re talking less than 1%.

That’s not the secret. The secret is that People Who Aren’t Programmers Do Not Understand This.

There are some very, very important corollaries to the Iceberg Secret.

Important Corollary One. If you show a nonprogrammer a screen which has a user interface that is 90% worse, they will think that the program is 90% worse.

I learned this lesson as a consultant, when I did a demo of a major web-based project for a client’s executive team. The project was almost 100% code complete. We were still waiting for the graphic designer to choose fonts and colors and draw the cool 3-D tabs. In the meantime, we just used plain fonts and black and white, there was a bunch of ugly wasted space on the screen, basically it didn’t look very good at all. But 100% of the functionality was there and was doing some pretty amazing stuff.

What happened during the demo? The clients spent the entire meeting griping about the graphical appearance of the screen. They weren’t even talking about the UI. Just the graphical appearance. “It just doesn’t look slick,” complained their project manager. That’s all they could think about. We couldn’t get them to think about the actual functionality. Obviously fixing the graphic design took about one day. It was almost as if they thought they had hired painters.

Important Corollary Two. If you show a nonprogrammer a screen which has a user interface which is 100% beautiful, they will think the program is almost done.

People who aren’t programmers are just looking at the screen and seeing some pixels. And if the pixels look like they make up a program which does something, they think “oh, gosh, how much harder could it be to make it actually work?

The big risk here is that if you mock up the UI first, presumably so you can get some conversations going with the customer, then everybody’s going to think you’re almost done. And then when you spend the next year working “under the covers,” so to speak, nobody will really see what you’re doing and they’ll think it’s nothing.

Important Corollary Three. The dotcom that has the cool, polished looking web site and about four web pages will get a higher valuation than the highly functional dotcom with 3700 years of archives and a default grey background.

Oh, wait, dotcoms aren’t worth anything any more. Never mind.

Important Corollary Four. When politics demands that various nontechnical managers or customers “sign off” on a project, give them several versions of the graphic design to choose from.

Vary the placement of some things, change the look and feel and fonts, move the logo and make it bigger or smaller. Let them feel important by giving them non-crucial lipstick-on-a-chicken stuff to muck around with. They can’t do much damage to your schedule here. A good interior decorator is constantly bringing their client swatches and samples and stuff to choose from. But they would never discuss dishwasher placement with the client. It goes next to the sink, no matter what the client wants. There’s no sense wasting time arguing about where the dishwasher goes, it has to go next to the sink, don’t even bring it up; let the clients get their design kicks doing some harmless thing like changing their mind 200 times about whether to use Italian Granite or Mexican Tiles or Norwegian wood butcher-block for the countertops.

Important Corollary Five. When you’re showing off, the only thing that matters is the screen shot. Make it 100% beautiful.

Don’t, for a minute, think that you can get away with asking anybody to imagine how cool this would be. Don’t think that they’re looking at the functionality. They’re not. They want to see pretty pixels.

Steve Jobs understands this. Oh boy does he understand this. Engineers at Apple have learned to do things that make for great screen shots, like the gorgeous new 1024×1024 icons in the dock, even if they waste valuable real estate. And the Linux desktop crowd goes crazy about semitransparent xterms, which make for good screenshots but are usually annoying to use. Every time Gnome or KDE announces a new release I go straight to the screenshots and say, “oh, they changed the planet from Jupiter to Saturn. Cool.” Never mind what they really did.

Remember the CEO at the beginning of this article? He was unhappy because his team had showed him great PowerPoints at the beginning — mockups, created in Photoshop, not even VB. And now that they’re actually getting stuff done under the covers, it looks like they’re not doing anything.

What can you do about this? Once you understand the Iceberg Secret, it’s easy to work with it. Understand that any demos you do in a darkened room with a projector are going to be all about pixels. If you can, build your UI in such a way that unfinished parts look unfinished. For example, use scrawls for the icons on the toolbar until the functionality is there. As you’re building your web service, you may want to consider actually leaving out features from the home page until those features are built. That way people can watch the home page go from 3 commands to 20 commands as more things get built.

More importantly, make sure you control what people think about the schedule. Provide a detailed schedule in Excel format. Every week, send out self-congratulatory email talking about how you’ve moved from 32% complete to 35% complete and are on track to ship on December 25th. Make sure that the actual facts dominate any thinking about whether the project is moving forward at the right speed. And don’t let your boss use Callaway Titanium Drivers, I don’t care how much you want him to win, the USGA has banned them and it’s just not fair.

Discuss

Humane Programming

Lazy programmers who didn’t read about how users don’t read anything:

Click for Larger View

  1. Most decent web programmers can always figure out how to design an interaction that won’t fail if you hit back or reload. These techniques are even older than cookies.
  2. Having three paragraphs of text telling you not to hit back or reload isn’t going to help, because people won’t read it.
  3. The programmer who wrote this screen admits that 10% of users “forget this warning.” It’s not because they forgot. It’s because humans are fallable and we’re used to hitting refresh when the screen is messed up, and we’re used to hitting back when we realize we went forward too soon. Don’t argue with me, Yahoo, when you’re the ones quoting statistics to prove it!

The main purpose of this screen seems to be so that users blame themselves when they hit reload and find themselves blown back to step one. Oh darn! I’m so stupid! say the users. Yes, that’s what happens. Watch any usability test where the product is failing – the users inevitably blame “their own stupidity.” Better that 100,000 users should feel stupid than one programmer admit he didn’t do a very good job.

Don’t let anyone tell you that as a programmer you don’t have to make moral or ethical decisions. Every time you decide that making users feel stupid is better than fixing your code, you’re making an ethical decision.

Painless Functional Specifications – Part 4: Tips

OK, we’ve talked about  why you need a spec, what a spec has in it, and who should write them. In this fourth and final part of the series I’ll share some of my advice for writing good specs.

The biggest complaint you’ll hear from teams that do write specs is that “nobody reads them.” When nobody reads specs, the people who write them tend to get a little bit cynical. It’s like the old Dilbert cartoon in which engineers use stacks of 4-inch thick specs to build extensions to their cubicles. At your typical big, bureaucratic company, everybody spends months and months writing boring specs. Once the spec is done, it goes up on the shelf, never to be taken down again, and the product is implemented from scratch without any regard to what the spec said, because nobody read the spec, because it was so dang mind-numbing. The very process of writing the spec might have been a good exercise, because it forced everyone, at least, to think over the issues. But the fact that the spec was shelved (unread and unloved) when it was completed makes people feel like it was all a bunch of work for naught.

Also, if your spec never gets read, you get a lot of arguments when the finished product is delivered. Somebody (management, marketing, or a customer) says: “wait a minute! You promised me that there would be a Clam Steamer! Where’s the clam steamer?” And the programmers say, “no, actually, if you look on the spec on chapter 3, subchapter 4, paragraph 2.3.0.1, you’ll see it says quite explicitly ‘no clam steamer.'” But that doesn’t satisfy the customer, who is always right, so the grumpy programmers have to go retrofit a clam steamer into the thing (making them even more cynical about specs). Or a manager says, “hey, all the wording on this dialog is too verbose, and there should be an advertisement at the top of every dialog box.” And the programmers say, in frustration, “but you approved the spec which precisely listed the layout and contents of every dialog box!” But of course, the manager hadn’t actually read the spec, because when he tried, his brain started seeping out through his eye sockets, and anyway, it was interfering with his Tuesday golf game.

So. Specs are good, but not if nobody reads them. As a spec-writer, you have to trick people into reading your stuff, and you should also probably make an effort not to cause any already-too-small brains to leak out through eye-sockets.

Tricking people into reading your stuff is usually just a matter of good writing. But it’s not fair of me to just say “be a good writer” and leave it at that. Here are four easy rules that you absolutely must follow to make specs that get read.

Rule 1: Be Funny

Yep, rule number one in tricking people into reading your spec is to make the experience enjoyable. Don’t tell me you weren’t born funny, I don’t buy it. Everybody has funny ideas all the time, they just self-censor them because they think that it’s “unprofessional.” Feh. Sometimes you have to break the rules.

If you read the volumes of garbage I’ve written on this web site, you’ll notice that there are a few lame attempts at being funny scattered throughout. Just four paragraphs ago I was making a gross body-fluid joke and making fun of managers for playing golf. Even though I’m not really that funny, I still try pretty hard, and even the act of flailing around trying to be funny is in itself amusing, in a sad-clown sort of way. When you’re writing a spec, an easy place to be funny is in the examples. Every time you need to tell a story about how a feature works, instead of saying:

  • The user types Ctrl+N to create a new Employee table and starts entering the names of the employees.

write something like:

  • Miss Piggy, poking at the keyboard with a eyeliner stick because her chubby little fingers are too fat to press individual keys, types Ctrl+N to create a new Boyfriend table and types in the single record “Kermit.”

If you read a lot of Dave Barry, you’ll discover that one of the easiest ways to be funny is to be specific when it’s not called for. “Scrappy pugs” are funnier than “dogs.” “Miss Piggy” is funnier than “the user”. Instead of saying “special interests,” say “left-handed avocado farmers.” Instead of saying “People who refuse to clean up after their dogs should be punished,” say that they should be “sent to prisons so lonely that the inmates have to pay spiders for sex.”

Oh, and, by the way, if you think that it’s unprofessional to be funny, then I’m sorry, but you just don’t have a sense of humor. (Don’t deny it. People without senses of humors always deny it. You can’t fool me.) And if you work in a company where people will respect you less because your specs are breezy, funny, and enjoyable to read, then go find another company to work for, because life is just too damn short to spend your daylight hours in such a stern and miserable place.

Rule 2: Writing a spec is like writing code for a brain to execute

Here’s why I think that programmers have trouble writing good specs.

When you write code, your primary audience is the compiler. Yeah, I know, people have to read code, too, but it’s generally very hard for them. For most programmers it’s hard enough to get the code into a state where the compiler reads it and correctly interprets it; worrying about making human-readable code is a luxury. Whether you write:

void print_count( FILE* a, char  *  b, int c ){
    fprintf(a, “there are %d %s\n”, c, b);}

main(){ int n; n =
10; print_count(stdout, “employees”, n) /* code
deliberately obfuscated */ }

or

printf(“there are 10 employees\n”);

you get the same output. Which is why, if you think about it, you tend to get programmers who write things like:

Assume a function AddressOf(x) which is defined as the mapping from a user x, to the RFC-822 compliant email address of that user, an ANSI string. Let us assume user A and user B, where A wants to send an email to user B. So user A initiates a new message using any (but not all) of the techniques defined elsewhere, and types AddressOf(B) in the To: editbox.

This could also have been speced as:

Miss Piggy wants to go to lunch, so she starts a new email and types Kermit’s address in the “To:” box. 

Technical note: the address must be a standard Internet address (RFC-822 compliant.)

They both “mean” the same thing, theoretically, except that the first example is impossible to understand unless you carefully decode it, and the second example is easy to understand. Programmers often try to write specs which look like dense academic papers. They think that a “correct” spec needs to be “technically” correct and then they are off the hook.

The mistake is that when you write a spec, in addition to being correct, it has to be understandable, which, in programming terms, means that it needs to be written so that the human brain can “compile” it. One of the big differences between computers and human brains is that computers are willing to sit there patiently while you define the terms that you want to use later. But humans won’t understand what you’re talking about unless you motivate it first. Humans don’t want to have to decode something, they just want to read it in order and understand it. For humans, you have to provide the big picture and then fill in the details. With computer programs, you start at the top and work your way to the bottom, with full details throughout. A computer doesn’t care if your variable names are meaningful. A human brain understands things much better if you can paint a vivid picture in their mind by telling a story, even if it’s just a fragment of a story, because our brains have evolved to understand stories.

If you show a chess board, in the middle of a real game of chess, to an experienced chess player for even a second or two, they will instantly be able to memorize the position of every piece. But if you move around a couple of pieces in nonsensical ways that couldn’t happen in normal play (for example, put some pawns on the first row, or put both black bishops on black squares), it becomes much, much harder for them to memorize the board. This is different from the way computers think. A computer program that could memorize a chess board could memorize both possible and impossible layouts with equal ease. The way the human brain works is not random access; pathways tend to be strengthened in our brains and some things are just easier to understand than other things because they are more common.

So, when you’re writing a spec, try to imagine the person you are addressing it to, and try to imagine what you’re asking them to understand at every step. Sentence by sentence, ask yourself if the person reading this sentence will understand it at a deep level, in the context of what you’ve already told them. If some members of your target audience don’t know what RFC-822 is, you either have to define it, or, at the very least, bury the mention of RFC-822 in a technical note, so that the executive-types who read the spec won’t give up and stop reading the first time they see a lot of technical jargon. 

Rule 3: Write as simply as possible

Don’t use stilted, formal language because you think it’s unprofessional to write in simple sentences. Use the simplest language you can. 

People use words like “utilize” because they think that “use” looks unprofessional. (There’s that word “unprofessional” again. Any time somebody tells you that you shouldn’t do something because it’s “unprofessional,” you know that they’ve run out of real arguments.) In fact I think that many people think that clear writing means that something is wrong.

Break things down to short sentences. If you’re having trouble writing a sentence clearly, break it into two or three shorter sentences. 

Avoid walls of text: entire pages with just text. People get scared and don’t read them. When was the last time you noticed a popular magazine or newspaper with entire pages of text? Magazines will go so far as to take a quote from the article and print it, in the middle of the page, in a giant font, just to avoid the appearance of a full page of text. Use numbered or bulleted lists, pictures, charts, tables, and lots of whitespace so that the reading “looks” fluffier.

             

“Magazines will go so far as to take a quote from the article and print it, in the middle of the page, in a giant font, just to avoid the appearance of a full page of text.”


 

Nothing improves a spec more than lots and lots of screenshots. A picture can be worth a thousand words. Anyone who writes specs for Windows software should invest in a copy of Visual Basic, and learn to use it at least well enough to create mockups of the screens. (For the Mac, use REAL Basic; for Web pages, use Front Page or Dreamweaver). Then capture these screenshots (Ctrl+PrtSc) and paste them into your spec.

Rule 4: Review and reread several times

Um, well, I was originally planning to have a lengthy exegesis of this rule here, but this rule is just too simple and obvious. Review and reread your spec several times, OK? When you find a sentence that isn’t super easy to understand, rewrite it.

I’ve saved so much time by not explaining Rule 4 that I’m going to add another rule.

Rule 5: Templates considered harmful

Avoid the temptation to make a standard template for specs. At first you might just think that it’s important that “every spec look the same.” Hint: it’s not. What difference does it make? Does every book on your bookshelf at home look exactly the same? Would you want them to?

Worse, if you have templates, what tends to happen is that you add a bunch of sections to the template for things that you think are important for every feature. Example: Big Bill decrees that from here on forward, every Microsquish product shall have an Internet component. So the spec template now has a section that says “Internet Component.” Whenever somebody writes a spec, no matter how trivial, they have to fill in that section that says “Internet Component”, even if they’re just creating the spec for the Microsquish Keyboard. (And you wondered why those useless Internet shopping buttons started cropping up like mushrooms on keyboards).

As these sections accumulate, the template gets pretty large. (Here is an example of a very, very bad template for specifications. Who needs a bibliography in a spec, for heaven’s sake? Or a glossary?) The trouble with such a large template is that it scares people away from writing specs because it looks like such a daunting task.

A spec is a document that you want people to read. In that way, it is no different than an essay in The New Yorker or a college paper. Have you ever heard of a professor passing out templates for students to write their college papers? Have you ever read two good essays that could be fit into a template? Just drop the idea.

Painless Functional Specifications – Part 3: But… How?

Now that you’ve read all about why you need a spec and what a spec has in it, let’s talk about who should write them.

Who writes specs?

Let me give you a little Microsoft history here. When Microsoft started growing seriously in the 1980s, everybody there had read The Mythical Man-Month, one of the classics of software management. (If you haven’t read it, I highly recommend it.) The main point of that book was that when you add more programmers to a late project, it gets even later. That’s because when you have n programmers on a team, the number of communication paths is n(n-1)/2, which grows at O(n2).

So the programmers at Microsoft were worried about how to write bigger and bigger programs, when the prevailing wisdom of the day was that adding programmers just makes things worse.

Charles Simonyi, Microsoft’s long time “chief architect”, suggested the concept of master programmers. The idea was basically that one master programmer would be responsible for writing all the code, but he or she would rely on a team of junior programmers as “code slaves”. Instead of worrying about debugging every function, the master programmer would basically just prototype each function, creating the bare outline, and then throw it to one of the junior programmers to implement.  (Of course, Simonyi would be the Master Master Programmer.) The term “Master Programmer” was a bit too medieval, so Microsoft went with “Program Manager.”

Theoretically, this was supposed to solve the Mythical Man-Month problem, because nobody has to talk to anyone else — every junior programmer only talks to the one program manager, and so communication grows at O(n) instead of O(n2).

Well, Simonyi may know Hungarian Notation, but he doesn’t know Peopleware. Nobody wants to be a code slave. The system didn’t work at all. Eventually, Microsoft discovered that despite the alleged Mythical Man Month, you can still add smart people to a team and get increased output, although at decreasing marginal values. The Excel team had 50 programmers when I was there, and it was marginally more productive than a team of 25 would have been — but not twice as productive.

The idea of master/slave programming was discredited, but Microsoft still had these people called program managers bouncing around. A smart man named Jabe Blumenthal basically reinvented the position of program manager. Henceforth, the program manager would own the design and the spec for products.

Since then, program managers at Microsoft gather requirements, figure out what the code is supposed to do, and write the specs. There are usually about 5 programmers for every program manager; these programmers are responsible for implementing in code what the program manager has implemented in the form of a spec. A program manager also needs to coordinate marketing, documentation, testing, localization, and all the other annoying details that programmers shouldn’t spend time on. Finally, program managers at Microsoft are supposed to have the “big picture” of the company in mind, while programmers are free to concentrate on getting their bits of code exactly right.

Program managers are invaluable. If you’ve ever complained about how programmers are more concerned with technical elegance than with marketability, you need a program manager. If you’ve ever complained about how people who can write good code never do a good job of writing good English, you need a program manager. If you’ve ever complained about how your product seems to drift without any clear direction, you need a program manager.

How do you hire a program manager?

Most companies don’t even have the concept of program manager. I think that’s too bad. In my time, the groups at Microsoft with strong program managers had very successful products: Excel, Windows 95, and Access come to mind. But other groups (such as MSN 1.0 and Windows NT 1.0) were run by developers who generally ignored the program managers (who weren’t very good anyway, and probably deserved to be ignored), and their products were not as successful.

Here are three things to avoid.

1. Don’t promote a coder to be a program manager.  The skills for being a good program manager (writing clear English, diplomacy, market awareness, user empathy, and good UI design) are very rarely the skills for being a good coder. Sure, some people can do both, but they are rare. Rewarding good coders by promoting them to a different position, one that involves writing English, not C++, is a classic case of the Peter Principle: people tend to be promoted to their level of incompetence.

2. Don’t let the marketing people be program managers. No offense, but I think my readers will agree that good marketing people rarely have a good enough grasp of the technology issues to design products.

Basically, program management is a separate career path. All program managers need to be very technical, but they don’t have to be good coders.  Program managers study UI, meet customers, and write specs. They need to get along with a wide variety of people — from “moron” customers, to irritating hermit programmers who come to work in Star Trek uniforms, to pompous sales guys in $2000 suits. In some ways, program managers are the glue of software teams. Charisma is crucial.

3. Don’t have coders report to the program manager. This is a subtle mistake. As a program manager at Microsoft, I designed the Visual Basic (VBA) strategy for Excel and completely speced out, to the smallest detail, how VBA should be implemented in Excel. My spec ran to about 500 pages. At the height of development for Excel 5.0, I estimated that every morning, 250 people came to work and basically worked off of that huge spec I wrote. I had no idea who all these people were, but there were about a dozen people on the Visual Basic team alone just writing documentation for this thing (not to mention the team writing documentation from the Excel side, or the full time person who was responsible for hyperlinks in the help file.) The weird thing was that I was at the “bottom” of the reporting tree. That’s right. NOBODY reported to me. If I wanted people to do something, I had to convince them that it was the right thing to do. When Ben Waldman, the lead developer, didn’t want to do something I had speced out, he just didn’t do it. When the testers complained that something I had speced was impossible to test completely, I had to simplify it. If any of these people had reported to me, the product wouldn’t have been as good. Some of them would have thought that it’s inappropriate to second-guess a superior. Other times, I would have just put my foot down and ordered them to do it my way, out of conceit or nearsightedness. As it was, I had no choice but to build consensus. This form of decision making was the best way to get the right thing done.

The final article in my series on specs talks about how to write good specs that people want to read.

Painless Functional Specifications – Part 2: What’s a Spec?

(Have you already read part one? If not, that’s here .)

This series of articles is about functional specifications, not technical specifications. People get these mixed up. I don’t know if there’s any standard terminology, but here’s what I mean when I use these terms.

  1. A functional specification describes how a product will work entirely from the user’s perspective. It doesn’t care how the thing is implemented. It talks about features. It specifies screens, menus, dialogs, and so on.
  2. A technical specification describes the internal implementation of the program. It talks about data structures, relational database models, choice of programming languages and tools, algorithms, etc.

When you design a product, inside and out, the most important thing is to nail down the user experience. What are the screens, how do they work, what do they do. Later, you worry about how to get from here to there. There’s no use arguing about what programming language to use before you’ve decided what your product is going to do. In this series, I’m only talking about functional specifications.

I’ve written a short sample spec which should give you an idea for what a good functional specification looks like. Before we go further, please read the sample spec.

Did you read it?

No you didn’t. Go read it now and then come back, so we can talk more about what a good spec should and shouldn’t have in it. I’ll wait here for you. Thanks.

(waiting patiently…)

picture-OldSanJuan:

Ah, good. You’re back.

Here are some of the things I put in every spec.

A disclaimer. Pure self defense. If you put a paragraph saying something like “This spec is not complete”, people won’t come into your office to bite your head off. As time goes on, when the spec starts to be complete, you can change it to say “this spec is complete, to the best of my knowledge, but if I forgot something, please tell me.” Which reminds me, every spec needs:

An author. One author. Some companies think that the spec should be written by a team. If you’ve ever tried group writing, you know that there is no worse torture. Leave the group writing to the management consulting firms with armies of newly minted Harvard-educated graduates who need to do a ton of busywork so that they can justify their huge fees. Your specs should be owned and written by one person. If you have a big product, split it up into areas and give each area to a different person to spec separately. Other companies think that it’s egotistic or not “good teamwork” for a person to “take credit” for a spec by putting their name on it. Nonsense. People should take responsibility and ownership of the things that they specify. If something’s wrong with the spec, there should be a designated spec owner, with their name printed right there on the spec, who is responsible for fixing it. 

Scenarios. When you’re designing a product, you need to have some real live scenarios in mind for how people are going to use it. Otherwise you end up designing a product that doesn’t correspond to any real-world usage (like the Cue?Cat). Pick your product’s audiences and imagine a fictitious, totally imaginary but totally stereotypical user from each audience who uses the product in a totally typical way. Chapter 9 of my UI design book (available online for free) talks about creating fictional users and scenarios. This is where you put them. The more vivid and realistic the scenario, the better a job you will do designing a product for your real or imagined users, which is why I tend to put in lots of made-up details. 

Nongoals. When you’re building a product with a team, everybody tends to have their favorite, real or imagined pet features that they just can’t live without. If you do them all, it will take infinite time and cost too much money. You have to start culling features right away, and the best way to do this is with a “nongoals” section of the spec. Things we are just not going to do. A nongoal might be a feature you won’t have (“no telepathic user interface!”) or it might be something more general (“We don’t care about performance in this release. The product can be slow, as long as it works. If we have time in version 2, we’ll optimize the slow bits.”) These nongoals are likely to cause some debate, but it’s important to get it out in the open as soon as possible. “Not gonna do it!” as George Sr. puts it. 

An Overview. This is like the table of contents for your spec. It might be a simple flowchart, or it might be an extensive architectural discussion. Everybody will read this to get the big picture, then the details will make more sense.

Details, details, details. Finally you go into the details. Most people will skim this until they need to know a particular detail. When you’re designing a web-type service, a good way to do this is to give every possible screen a canonical name, and provide a chapter describing each one in utter and mind-numbing detail.

Details are the most important thing in a functional spec. You’ll notice in the sample spec how I go into outrageous detail talking about all the error cases for the login page. What if the email address isn’t valid? What if the password is wrong? All of these cases correspond to real code that’s going to be written, but, more importantly, these cases correspond to decisions that somebody is going to have to make. Somebody has to decide what the policy is going to be for a forgotten password. If you don’t decide, you can’t write the code. The spec needs to document the decision.

Open Issues. It’s OK for the first version of the spec to leave open issues. When I write a first draft, I always have lots of open issues, but I flag them (using a special style so I can search for them) and, if appropriate, discuss the alternatives. By the time the programmers start work, all of these need to be stomped out. (You might think it’s OK to just let the programmers start on all the easy stuff, and you’ll solve the open issues later. Bad idea. You will have enough problems resolving the new issues that come up when the programmers try to implement the code, without having old open issues around that you knew about in advance and could have solved then. Besides, the way you resolve anything non-trivial may have a major impact on how the code should be written.)

Side notes. While you’re writing a spec, remember your various audiences: programmers, testers, marketing, tech writers, etc. As you write the spec you may think of useful factoids that will be helpful to just one of those groups. For example, I flag messages to the programmer, which usually describe some technical implementation detail, as “Technical Notes”. Marketing people ignore those. Programmers devour them. My specs are often chock full of “Testing Notes,” “Marketing Notes,” and “Documentation Notes.”

Specs Need To Stay Alive. Some programming teams adopt a “waterfall” mentality: we will design the program all at once, write a spec, print it, and throw it over the wall at the programmers and go home. All I have to say is: “Ha ha ha ha ha ha ha ha!”

This approach is why specs have such a bad reputation. A lot of people have said to me, “specs are useless, because nobody follows them, they’re always out of date, and they never reflect the product.”

Excuse me. Maybe your specs are out of date and don’t reflect the product. My specs are updated frequently. The updating continues as the product is developed and new decisions are made. The spec always reflects our best collective understanding of how the product is going to work. The spec is only frozen when the product is code complete (that is, when all functionality is complete, but there’s still testing and debugging work.)

To make people’s life easier, I don’t rerelease the spec daily. I usually keep an up to date version on a server somewhere where the team can use it as a reference. On occasional milestones, I print a copy of the spec with revision marks so that people don’t have to reread the whole thing — they can scan the revision marks to see what changes have been made.

Who should write the specs? Read all about it in Part 3.

Painless Functional Specifications – Part 1: Why Bother?

When The Joel Test first appeared, one of the biggest sore points readers reported had to do with writing specs. It seems that specs are like flossing: everybody knows they should be writing them, but nobody does.

Why won’t people write specs? People claim that it’s because they’re saving time by skipping the spec-writing phase. They act as if spec-writing was a luxury reserved for NASA space shuttle engineers, or people who work for giant, established insurance companies. Balderdash. First of all, failing to write a spec is the single biggest unnecessary risk you take in a software project. It’s as stupid as setting off to cross the Mojave desert with just the clothes on your back, hoping to “wing it.” Programmers and software engineers who dive into code without writing a spec tend to think they’re cool gunslingers, shooting from the hip. They’re not. They are terribly unproductive. They write bad code and produce shoddy software, and they threaten their projects by taking giant risks which are completely uncalled for.

I believe that on any non-trivial project (more than about 1 week of coding or more than 1 programmer), if you don’t have a spec, you will always spend more time and create lower quality code. Here’s why.

The most important function of a spec is to design the program. Even if you are working on code all by yourself, and you write a spec solely for your own benefit, the act of writing the spec — describing how the program works in minute detail — will force you to actually design the program.

Let’s visit two imaginary programmers at two companies. Speedy, at Hasty Bananas Software, never writes specs. “Specs? We don’t need no stinkin’ specs!” At the same time, Mr. Rogers, over at The Well-Tempered Software Company, refuses to write code until the spec is completely nailed down. These are only two of my many imaginary friends.

Speedy and Mr. Rogers have one thing in common: they are both in charge of backwards compatibility for version 2.0 of their respective products.

Speedy decides that the best way to provide backwards compatibility is to write a converter which simply converts 1.0 version files into 2.0 version files. She starts banging that out. Type, type, type. Clickety clickety clack. Hard drives spin. Dust flies. After about 2 weeks, she has a reasonable converter. But Speedy’s customers are unhappy. Speedy’s code will force them to upgrade everyone in the company at once to the new version. Speedy’s biggest customer, Nanner Splits Unlimited, refuses to buy the new software. Nanner Splits needs to know that version 2.0 will still be able to work on version 1.0 files without converting them. Speedy decides to write a backwards converter and then hook it into the “save” function. It’s a bit of a mess, because when you use a version 2.0 feature, it seems to work, until you go to save the file in 1.0 format. Only then are you told that the feature you used half an hour ago doesn’t work in the old file format. So the backwards converter took another two weeks to write, and it don’t work so nice. Elapsed time, 4 weeks.

Now, Mr. Rogers over at Well-Tempered Software Company (colloquially, “WellTemperSoft”) is one of those nerdy organized types who refuses to write code until he’s got a spec. He spends about 20 minutes designing the backwards compatibility feature the same way Speedy did, and comes up with a spec that basically says:

  • When opening a file created with an older version of the product, the file is converted to the new format.

The spec is shown to the customer, who says “wait a minute! We don’t want to switch everyone at once!” So Mr. Rogers thinks some more, and amends the spec to say:

  • When opening a file created with an older version of the product, the file is converted to the new format in memory. When saving this file, the user is given the option to convert it back.

Another 20 minutes have elapsed.

Mr. Rogers’ boss, an object nut, looks at this and thinks something might be amiss. He suggests a different architecture.

  • The code will be factored to use two interfaces: V1 and V2.  V1 contains all the version one features, and V2, which inherits from V1, adds all the new features. Now V1::Save can handle the backwards compatibility while V2::Save can be used to save all the new stuff. If you’ve opened a V1 file and try to use V2 functionality, the program can warn you right away, and you will have to either convert the file or give up the new functionality.

20 more minutes.

Mr. Rogers is grumpy. This refactoring will take 3 weeks, instead of the 2 weeks he originally estimated! But it does solve all the customer problems, in an elegant way, so he goes off and does it.

Total elapsed time for Mr. Rogers: 3 weeks and 1 hour. Elapsed time for Speedy: 4 weeks, but Speedy’s code is not as good.

The moral of the story is that with a contrived example, you can prove anything. Oops. No, that’s not what I meant to say. The moral of the story is that when you design your product in a human language, it only takes a few minutes to try thinking about several possibilities, revising, and improving your design. Nobody feels bad when they delete a paragraph in a word processor. But when you design your product in a programming language, it takes weeks to do iterative designs. What’s worse, a programmer who’s just spend 2 weeks writing some code is going to be quite attached to that code, no matter how wrong it is. Nothing Speedy’s boss or customers could say would convince her to throw away her beautiful converting code, even though that didn’t represent the best architecture. As a result, the final product tends to be a compromise between the initial, wrong design and the ideal design. It was “the best design we could get, given that we’d already written all this code and we just didn’t want to throw it away.” Not quite as good as “the best design we could get, period.”

So that’s giant reason number one to write a spec. Giant reason number two is to save time communicating. When you write a spec, you only have to communicate how the program is supposed to work once. Everybody on the team can just read the spec. The QA people read it so that they know how the program is supposed to work and they know what to test for. The marketing people use it to write their vague vaporware white papers to throw up on the web site about products that haven’t been created yet. The business development people misread it to spin weird fantasies about how the product will cure baldness and warts and stuff, but it gets investors, so that’s OK. The developers read it so that they know what code to write. The customers read it to make sure the developers are building a product that they would want to pay for. The technical writers read it and write a nice manual (that gets lost or thrown away, but that’s a different story). The managers read it so that they can look like they know what’s going on in management meetings. And so on.

When you don’t have a spec, all this communication still happens, because it has to, but it happens ad hoc. The QA people fool around with the program willy-nilly, and when something looks odd, they go and interrupt the programmers yet again to ask them another stupid question about how the thing is supposed to work. Besides the fact that this ruins the programmers’ productivity, the programmers tend to give the answer that corresponds to what they wrote in the code, rather than the “right answer.” So the QA people are really testing the program against the program rather than the program against the design, which would be, um, a little bit more useful.

When you don’t have a spec, what happens with the poor technical writers is the funniest (in a sad kind of way). Tech writers often don’t have the political clout to interrupt programmers. In many companies, if tech writers get in the habit of interrupting programmers to ask how something is supposed to work, the programmers go to their managers and cry about how they can’t get any work done because of these [expletive deleted] writers, and could they please keep them away, and the managers, trying to improve productivity, forbid the tech writers to waste any more of their precious programmers’ time. You can always tell these companies, because the help files and the manuals don’t give you any more information than you can figure out from the screen. When you see a message on a screen which says

  • Would you like to enable LRF-1914 support?

… and you click “Help”, a tragicomic help topic comes up which says something like

  • Allows you to choose between LRF-1914 support (default) or no LRF-1914 support. If you want LRF-1914 support, choose “Yes” or press “Y”. If you don’t want  LRF-1914 support, choose “No” or press “N”.

Um, thanks. It’s pretty obvious here that the technical writer was trying to cover up the fact that they didn’t know what LRF-1914 support is. They couldn’t ask the programmer, because (a) they’re embarrassed, or (b) the programmer is in Hyderabad and they’re in London, or (c) they have been prohibited by management from interrupting the programmer, or any other number of corporate pathologies too numerous to mention, but the fundamental problem is that there wasn’t a spec.

Number three giant important reason to have a spec is that without a detailed spec, it’s impossible to make a schedule. Not having a schedule is OK if it’s your PhD and you plan to spend 14 years on the thing, or if you’re a programmer working on the next Duke Nukem and we’ll ship when we’re good and ready. But for almost any kind of real business, you just have to know how long things are going to take, because developing a product costs money. You wouldn’t buy a pair of jeans without knowing what the price is, so how can a responsible business decide whether to build a product without knowing how long it will take and, therefore, how much it will cost? For more on scheduling, read Painless Software Schedules.

A terribly common error is having a debate over how something should be designed, and then never resolving the debate. Brian Valentine, the lead developer on Windows 2000, was famous for his motto “Decisions in 10 minutes or less, or the next one is free.”

In too many programming organizations, every time there’s a design debate, nobody ever manages to make a decision, usually for political reasons. So the programmers only work on uncontroversial stuff. As time goes on, all the hard decisions are pushed to the end. These are the most likely projects to fail. If you are starting a new company around a new technology and you notice that your company is constitutionally incapable of making decisions, you might as well close down now and return the money to the investors, because you ain’t never gonna ship nothing.

Writing a spec is a great way to nail down all those irritating design decisions, large and small, that get covered up if you don’t have a spec. Even small decisions can get nailed down with a spec. For example, if you’re building a web site with membership, you might all agree that if the user forgets their password, you’ll mail it to them. Great. But that’s not enough to write the code. To write the code, you need to know the actual words in that email. At most companies, programmers aren’t trusted with words that a user might actually see (and for good reason, much of the time). So a marketing person or a PR person or some other English major is likely to be required to come up with the precise wording of the message. “Dear Shlub, Here’s the password you forgot. Try not to be so careless in the future.” When you force yourself to write a good, complete spec (and I’ll talk a lot more about that soon), you notice all these things and you either fix them or at least you mark them with a big red flag.

OK. We’re on the same page now. Specs are motherhood and apple pie. I suspect that most people understand this, and my rants, while amusing, aren’t teaching you anything new. So why don’t people write specs? It’s not to save time, because it doesn’t, and I think most coders recognize this. (In most organizations, the only “specs” that exist are staccato, one page text documents that a programmer banged out in Notepad after writing the code and after explaining that damn feature to the three hundredth person.)

I think it’s because so many people don’t like to write. Staring at a blank screen is horribly frustrating. Personally, I overcame my fear of writing by taking a class in college that required a 3-5 page essay once a week. Writing is a muscle. The more you write, the more you’ll be able to write. If you need to write specs and you can’t, start a journal, create a weblog, take a creative writing class, or just write a nice letter to every relative and college roommate you’ve blown off for the last 4 years. Anything that involves putting words down on paper will improve your spec writing skills. If you’re a software development manager and the people who are supposed to be writing specs aren’t, send them off for one of those two week creative writing classes in the mountains.

If you’ve never worked in a company that does functional specifications, you may never have seen one. In the next part of this series, I’ll show you a short, sample spec for you to check out, and we’ll talk about what a good spec needs to have. Read on!

The Process of Designing a Product

We’ve talked about the principles of good design, but principles only give you a way to evaluate and improve an existing design. But… how do you figure out what the dang design should be in the first place? Many people write big, functional outlines of all the features they thought up. Then they design each one, and hang it off of a menu item (or web page). When they’re done, the program (or web site) has all the functionality they wanted, but it doesn’t flow right. People sit down and they don’t know what it does, and they don’t know how to accomplish what they want.

Microsoft’s solution to this is something called Activity Based Planning. (As far as I can tell, this concept was invented by Mike Conte on the Excel team, who got bored with that and went on to a second career as a race car driver). The key insight is to figure out the activity that the user is doing, and focus on making it easy to accomplish that activity. This is best illustrated with an example.

You’ve decided to make a web site that lets people create greeting cards. Using a somewhat naïve approach, you might come up with a list of features like this:

1. Add text to card
2. Add picture to card
3. Get predesigned card from library
4. Send card:
           a. Using email
           b. By printing it out

For lack of any better way of thinking about the problem, this might lead itself to a typical Macintosh user interface, circa-1984: a program that starts out with a blank card, with menu items for adding text, pictures, loading cards from a library, and sending cards. And then what the user is going to have to do is sit down and browse through the menus, trying to figure out all the commands available, and then do their own synthesis of how to put these atomic commands together to create a card.

Now, activity based planning says that you need to come up with a list of activities that users might do. So, you talk to your potential users, and you come up with this “top three” list:

  1. Birthday Greeting
  2. Party Invitation
  3. Anniversary Greeting

Now, instead of thinking about your program like a programmer (in terms of what features you need to have to make a card), you’re thinking about it like the user, in terms of, what activities is the user doing, specifically:

  1. Sending a birthday card
  2. Planning a party, and inviting people to it
  3. Sending an anniversary card

Suddenly, all kinds of ideas will rush into your head. Instead of starting with a blank card, you might start with a menu like this:

What do you want to do?

  • Send a birthday card
  • Send an anniversary card
  • Send a party invitation
  • Start with a blank card

Suddenly users will find it much easier to get started with your program, without browsing around on the menus, since the program will virtually lead them through the steps to complete the activity. (There is a risk that if you didn’t pick the activities correctly, you will alienate or confuse users who might have been able to use your program, say, to send a Hanukah card, but don’t see that as a choice. So be careful in picking activities that blanket the majority of the market you want to target.)

Just looking at our list of three activities suggests some great features which you might want to add. For example, if you’re sending a birthday or anniversary card, you might want to be reminded next year to send a card to the same person… so you might add a checkbox that says “remind me next year”. And a party invitation needs a way to RSVP, so you might add a feature that lets you collect RSVPs from people electronically. Both of these feature ideas almost fell out of looking at the activity that users were performing instead of the features in the application.

This example is trivial; for any serious application, the rewards of activity based planning are even greater. When you’re designing a program from scratch, you already have a vision of what activities your users are going to be doing. Figuring out this vision is not hard at all, it takes almost no effort at all to do some brainstorming with coworkers, write down a list of potential activities, and then decide which ones you want to focus on. But forcing yourself to list these activities on paper will help your overall design enormously. 

Activity based planning is even more important when you are working on version two of a product that people are already using. Here, it may be a matter of observing a sample of customers to see what they are using your program for. 

In the days of Excel 1.0 through 4.0, most people at Microsoft thought that the most common user activity was doing financial what-if scenarios, where you do things like change the inflation rate and see how this affects your profitability. 

When we were designing Excel 5.0, the first major release to use serious activity-based planning, we only had to watch about five customers using the product before we realized that an enormous number of people just use Excel to keep lists. They are not entering any formulas or doing any calculation at all! We hadn’t even considered this before. Keeping lists turned out to be far more popular than any other activity with Excel. And this led us to invent a whole slew of features that make it easier to keep lists: easier sorting, automatic data entry, the AutoFilter feature which helps you see a slice of your list, and multi-user features which let several people work on the same list at the same time while Excel automatically reconciles everything.

While Excel 5 was being designed, Lotus had shipped a “new paradigm” spreadsheet called Improv. According to the press releases, Improv was a whole new generation of spreadsheet, which was going to blow away everything that existed before it. For various strange reasons, Improv was first available on the NeXT, which certainly didn’t help its sales, but a lot of smart people believed that Improv would be to NeXT as VisiCalc was to the Apple II: it would be the killer app that made people go out and buy all new hardware just to run one program.

Of course, Improv is now a footnote in history. Search for it on the web, and the only links you’ll find are from very over-organized storeroom managers who have, for some reason, made a web site with an inventory of all the stuff they have collecting dust.

Why? Because in Improv, it was almost impossible to just make lists. The Improv designers thought that people were using spreadsheets to create complicated multi-dimensional financial models. Turns out, if they asked people, they would discover that making lists was so much more common than multi-dimensional financial models, and in Improv, making lists was a downright chore, if not impossible.

So activity based planning is helpful in the initial version of your application, where you have to make guesses about what people want to do, but it’s even more helpful when you’re planning the upgrade, because you understand what your customers are doing.

Another example, from the web, is the evolution of deja.com, which started out as an huge, searchable index of Usenet called dejanews. The original interface basically had an edit box and said “search Usenet for blah,” and that was it. In 1999 a bit of activity based planning showed that one common user activity was doing research on a product or service, of the “which car should I buy” nature. Deja was completely reorganized, and today, it is more of a product opinion research service: the Usenet searching ability is almost completely hidden. This annoyed the small number of users who were using the site to search for whether their Matrox video card worked with Redhat Linux 5.1, but it delighted the much larger population of users who just wanted to buy the best digital camera.

The other great thing about activity based planning is that it lets you make a list of what features not to do. When you create any kind of software, the reality is that you will come up with three times as many features as you have time to do. And one of the best ways to decide which features get done, and which features get left out, is to evaluate which features support the most important user activities.

Imaginary Users.

The very best UI designers in the industry all agree on one thing: you have to invent and describe some imaginary users before you can design your UI. You may remember back in the introduction to this book, I introduced an imaginary user Pete:

Pete is an accountant for a technical publisher who has used Windows for six years at the office and a bit at home. He is fairly competent and technical. He installs his own software; he reads PC Magazine, and he has even programmed some simple Word macros to help the secretaries in his office send invoices. He’s getting a cable modem at home. Pete has never used a Macintosh. “They’re too expensive,” he’ll tell you. “You can get a 700 Mhz PC with 128 Meg RAM for the price of…” OK, Pete. We get it.

When you read this, you can almost imagine a user. I could also have invented quite another type of user:

Patricia is an English professor who has written several well-received books of poetry. She has been using computers for word processing since 1980, although the only two programs she ever used are Nota Bene (an ancient academic word processor) and Microsoft Word. She doesn’t want to spend time learning the theory of how the computer works, and she tends to store all her documents in whatever directory they would go in if you didn’t know about directories.

Obviously, designing software for Pete is quite different from designing software for Patricia, who in turn is quite different from Mike, a 16 year old who runs Linux at home, talks on IRC for hours, and uses no “Micro$oft” software.

When you invent these users, thinking about whether your design is appropriate becomes much easier. For example, a lot of programmers tend to overestimate the ability of the typical user to figure things out. Whenever I write something about command line interfaces being hard to use, I get the inevitable email barrage saying that command line interfaces are ultra-powerful because you can do things like ‘gunzip foo.tar.gz | tar xvf -‘. But as soon as you have to think about getting Patricia to type “gunzip…” it becomes obvious that that kind of interface just isn’t going to serve her needs, ever. Thinking about a “real” person gives you the empathy you need to make a feature that serves that person’s need. (Of course, if you’re making Linux backup software for advanced sysadmins, you need to invent a character like “Frank” who refuses to touch Windows, which he only refers to as an “operating system” in quotation marks, uses his own personally modified version of tcsh, and runs X11 with four tiled xterms all day long. And about 11 xperfs.)

To summarize, designing good software takes about six steps:

  1. Invent some users

  2. Figure out the important activities

  3. Figure out the user model — how the user will expect to accomplish those activities

  4. Sketch out the first draft of the design

  5. Iterate over your design again and again, making it easier and easier until it’s well within the capabilities of your imaginary users

  6. Watch real humans trying to use your software. Note the areas where people have trouble, which probably demonstrate areas where the program model isn’t matching the user model.

Good UI sells software, but it also makes people happy, because people are happy when they accomplish the task they wanted to accomplish. Which is why UI design is such a satisfying field to be in. Where else are you going to get a chance to make millions of people just a little bit happier?