Imagine, for a moment, that you came upon a bread factory for the first time. At first it just looks like a jumble of incomprehensible machinery with a few people buzzing around. As your eyes adjust you start to see little piles of things that you do understand. Buckets of sesame seeds. Big vats of dough. Little balls of dough. Baked loaves of bread.

Those things are inventory. Inventory tends to pile up between machines. Next to the machine where sesame seeds are applied to hamburger buns, there’s a big vat of…sesame seeds. At the very end of the assembly line, there are boxes and boxes of bread, waiting for trucks to drive them off to customers.

Keeping inventory costs money. Suppose your bakery has six 50-ton silos to store flour. Whenever they empty out, you fill them up. That means on the average day you have 150 metric tons of wheat flour in stock. At today’s prices, you’ve tied up $73,000. Forever.

Inventory may have other costs too, like spoilage. Flour lasts for months, but the minute bread comes out of the oven it starts dropping in value; after 24 hours it’s nearly worthless.

Why keep inventory at all? Because there are costs associated with running out of things, too. If sesame seeds take two days to order, and you run out of sesame seeds, you are out of the hamburger bun business for two days. Inventory provides a buffer that prevents any part of the process from stalling. There are modern algorithms to optimize how much buffer you need at every point (read up on Toyota’s lean production system and the Theory of Constraints to get started).

Why do I care about any of this? The software production process has several major “inventory” accumulation points, itself. Stuff accumulates at those points and ends up wasting a lot of time and money.

“What? How is software like a factory?” you ask.

Think of product ideas as the raw material. Depending on your process, product ideas may go through several assembly line points before they are delivered as finished features to the customer:

1. A decision-making process (should we implement this feature?)
2. A design process (specs, whiteboards, mockups, etc)
3. An implementation process (writing code)
4. A testing process (finding bugs)
5. A debugging process (fixing bugs)
6. A deployment process (sending code to customers, putting it on web server, etc)

(PS No, this is not “waterfall.” No it isn’t. Is not. Shut up.)

In between each of these stages, inventory can pile up. For example, when a programmer finishes implementing their code (stage 3) they give it to a tester to check (stage 4). At any given time, there is a certain amount of code waiting to be tested. That code is inventory.

The “cost” of code inventory is huge. It might add up to six or twelve months of work that is stuck in the assembly line and not yet in customers’ hands. This could be the difference between having a cutting-edge product (iPhone) or constantly playing catchup (Windows Phone). It’s nearly impossible to get people to buy Windows Phones, even if the iPhone is only six months better. A lot of markets have network effects, and being first has winner-take-all implications. So getting rid of inventory in the development process can make or break a product.

Let’s go over the three places most inventory accumulates.

Feature backlogs. Every product attracts new feature ideas, and you can’t implement ideas as fast as you can think them up, so you write them down, and this list is called the feature backlog. A lot of the ideas on the backlog are bad ideas, and you merely wrote them down to avoid hurting the feelings of the people who thought them up. Backlogs make everyone feel good.

The trouble is that 90% of the things in the feature backlog will never get implemented, ever. So every minute you spent writing down, designing, thinking about, or discussing features that are never going to get implemented is just time wasted. When I hear about product teams that regularly have “backlog grooming” sessions, in which they carefully waste a tiny amount of time and mental energy every day or every week thinking about every single feature which will never be implemented, I want to poke my eyes out.

• Suggestion: Do not allow more than a month or two of work to get into the feature backlog list. Once the backlog is full, do not allow new items to be added unless you remove an item. Do not spend any time speccing, designing, or talking about backlog items: the backlog, in fact, should be seen as a list of things you are not allowed to talk about or work on.

The bug database is obviously a great thing to have. Bug reports should be complete, accurate, and actionable. But I have noticed that in many real-world companies, the desire never to miss any bug report leads to bug bankrupcy, where you wake up one day and discover that there are 3000 open bugs in the database, some of which are so old they may not apply any more, some of which can never be reproduced, and most of which are not even worth fixing because they’re so tiny. When you look closely you realize that months or years of work has gone into preparing those bug reports, and you ask yourself, how could we have 3000 bugs in the database while our product is delightful and customers love it and use it every day? At some point you realize that you’ve put too much work into the bug database and not quite enough work into the product.

• Suggestion: use a triage system to decide if a bug is even worth recording.
• Do not allow more than two weeks (in fix time) of bugs to get into the bug database.
• If you have more than that, stop and fix bugs until you feel like you’re fixing stupid bugs. Then close as “won’t fix” everything left in the bug database. Don’t worry, the severe bugs will come back.

Undeployed features. There are still a lot of teams doing quarterly or annual releases, usually because their deployment process is expensive. Operating systems, or anything where software has to be installed by every user, is usually batched up.

This is one of the most expensive forms of inventory: unshipped feature inventory. It could be earning you money, but it’s sitting on the shipping dock of your factory, while the guy down the street already has a product that does that exact same thing.

Sometimes, perniciously, you don’t even feel the pain, because everyone on your team has been dogfooding the new version for months. I’m sure everyone at Microsoft has been happily using Windows 8 for a year now, so they don’t really feel, on a day to day basis, the pain of OEMs trying to sell Windows 7 in a Mac OS X Lion world.

• Suggestion: Don’t let completed features pile up in ways that don’t make you money. Work on your deployment process so that you can get customers features in months rather than years. If you’re already shipping monthly, figure out how to ship weekly. Keep pushing the bar on more and more frequent deployment of smaller and smaller changes.

So, where am I going with this? We’ve had all three kinds of inventory at Fog Creek: crazy long backlogs, overambitious bug databases, and features which got stuck for a year waiting for the next release to go out. All of these snuck up on us. I realized that we needed a system to constrain inventory so it doesn’t build up. 

My original idea was to make a product called Five Things. It was going to be a project manager where everybody was allowed to have five things assigned to them: two things they were actively doing, one thing that was “up next”, and a couple more that they were planning. That exact design idea didn’t go anywhere (but if you want to build it, go for it), but it did evolve into Trello.

Trello works great for a reasonable amount of inventory, but it intentionally starts to get klunky if you have too many cards in one list. And that’s exactly the point: it makes inventory visible so that you know when it’s starting to pile up. (Click on the image at the right to see the Trello team’s own development board).

Every day you look at your Trello board and see that there are seventeen completed features that are totally ready to ship but which haven’t shipped for some reason, and you go find the bottleneck and eliminate it.

Every time somebody suggests a crazy feature idea, you look at the Feature Backlog and realize it’s just too long, so you don’t waste any time documenting or designing that crazy idea.

And hopefully, you’ll spend less effort working on things that never see the light of day. “Backlog grooming.” Sheeeesh.

Having a good program manager is one of the secret formulas to making really great software. And you probably don’t have one on your team, because most teams don’t.

Charles Simonyi, the brilliant programmer who co-invented WYSIWYG word processing, dated Martha Stewart, made a billion dollars off of Microsoft stock and went into space, first tried to solve the Mythical Man Month problem of organizing really big software teams by creating one super duper überprogrammer writing the top-level functions, while handing off the implementation of the lower-level functions to a team of grunt junior-programmers as needed. They called this position program manager. Simonyi is brilliant, but this idea, not so much. Nobody wanted to be a grunt junior programmer, I guess.

For more on the history, read William Poundstone’s How Would You Move Mount Fuji?

Jabe Blumenthal, a programmer on the Mac Excel team in the late 80s, recycled the title for a different job. He had noticed that software development was getting so complicated that none of the programmers had the time to figure out how to make software that was either usable or useful. The marketing team was ranting and raving about customer needs and nobody had time to talk to them or translate their MBA-speak into actual features. There was a lot of product design stuff that took a lot of work: talking to users, running usability tests, reviewing competitive products, and thinking hard about how to make things easier, and most programmers just didn’t have the time (nor were they particularly good at it). Blumenthal took the title “Program Manager,” but reinvented the job completely.

What does a program manager do?

Henceforth, a program manager would:

1. Design UIs
2. Write functional specs
3. Coordinate teams
4. Serve as the customer advocate, and
5. Wear Banana Republic chinos

On small products, you might just have one program manager, but on larger products, you would probably have more than one. Each can be responsible for some subset of the features. A good rule of thumb is that it takes about one program manager for every four programmers. If you’re having trouble dividing up the work, one approach I learned from Mike Conte is to divide up the product according to user activities. For example, Twitter could be divided into four user activities:

1. Registering and getting started
2. Posting messages and reading replies
3. Configuring your account
4. Searching for news

Tyler Griffin Hicks-WrightMy first program management assignment at Microsoft was on Excel, working on the user activity called “customization,” i.e., scripting and macros. The first thing I had to do was figure out what customers needed, which I did by talking to as many customers as I could until I started to get kind of bored because I kept hearing the same thing. I spent a lot of time talking to the development team to figure out what would be possible and reasonable to implement in a single 18 month release, and I spent a lot of time talking to the Visual Basic team to see if they could supply a compiler, code editor, and dialog box editor that could be used in Excel for our macro language. I also had to talk to Apple, which was developing their own universal macro language called AppleScript, and the other application teams at Microsoft, mainly Word, Access, Project, and Mail, who generally did whatever Excel did. Most of this process consisted of talking. Meetings, email, phone calls. I am scarred for life from this, and now cower in my office in fear that the phone will ring.

The second step was writing a vision statement: sort of a broad document that said, this is how Visual Basic would work in Excel, this is what some sample macros would look like, these are the major pieces we would need to build, and this is how it would solve customers’ problems. When that didn’t generate too many objections, I started working on a much more detailed spec, which explained, down to the smallest detail, how everything looked to the user.

This was a functional spec, not a technical spec, which means, all it talked about was what the user saw, not how it was implemented. (Read all about functional specs here.) A program manager doesn’t care how the development team implements things internally. As I sent chapters of the spec to Ben Waldman, the development lead, he and his team sat down and figured out what they had to do internally to make it work. They came up with a rather brilliant and very compact table that mapped the object-oriented interface I was defining onto internal Excel functions, but that really wasn’t my business. I didn’t know too much about Excel internals and didn’t really know how things should be implemented.

Truth be told, I didn’t know anything about anything. Fresh out of college, I didn’t have enough experience to develop the code, test the code, write the documentation, market the product, or do the usability tests. Luckily, Microsoft had seriously experienced gurus in each of those positions, who taught me everything I know today, and who did the real work of producing an awesome product. For example, I knew that users would want to copy the value of a spreadsheet cell into a variable:

 x = [A1]  

had to work. The trouble was that a cell could hold a number or a string, but Basic was early bound… you had to DIM x as an Integer, Float or String before you could use it.

Basic had to get some kind of dynamic types, but I wasn’t smart enough to figure out how to do that. Didn’t matter. Tom Corbett, a programmer on the Visual Basic team, figured out how. And thus Variants and IDispatch were born, and Basic became a dynamic language with what you kids now call “duck typing”. The point being, my job wasn’t necessarily to solve problems, it was to figure out what customers needed and make sure that programmers figured out how to solve them.

Once the spec was finished and the development team got down to work, I had two responsibilities: resolving any questions that came up about the design, and talking to all the other teams so that the developers didn’t have to. I met with the testers explaining how things were supposed to work and helping them plan how to test everything. I met with the documentation team, making sure they understood how to write a good tutorial and reference for Excel Basic. I met with localization experts to figure out a localization strategy. I sat down with marketing to explain the marketing benefits of VBA. I worked with usability experts to set up usability tests.

A program manager does go to a lot of meetings, but doesn’t produce much other than that written spec, which is why as a twerp fresh out of school I was still able to do the job. You don’t have to be a 14-year veteran programmer to work as a program manager (in fact, with 14 years of programming experience, you might know too much to be a good user advocate.)

Conflict

Tom Chi and Kevin ChengLacking a program manager, your garden-variety super-smart programmer is going to come up with a completely baffling user interface that makes perfect sense IF YOU’RE A VULCAN (cf. git). The best programmers are notoriously brilliant, and have some trouble imagining what it must be like not to be able to memorize 16 one-letter command line arguments. These programmers then have a tendency to get attached to their first ideas, especially when they’ve already written the code.

One of the best things a program manager can add to the software design process is a second opinion as to how things should be designed, hopefully one that is more empathetic to those DOUBLE SUPER UNSMART USERS with their pesky mental feebleness requiring that an application be usable without reading the man page, writing a custom emacs-lisp function, or translating numbers into octal in your head.

A good program manager will come with her own ideas for how the UI should work, which might be better, or worse, than the developer’s idea. And then there’s a long debate. Typically, the program manager wants something simple and easy to understand for the users, featuring a telepathic user interface and a 30″ screen that nonetheless fits in your pocket, while the developer wants something that is trivial to implement in code, with a command-line interface (“what’s so unusable about that?”) and Python bindings.

One of the most monumental debates I remember from the Excel 5 project was between a developer who wanted pivot tables to float on the drawing layer above the spreadsheet, and the program manager, who insisted that pivot tables live right in the cells on the spreadsheet. This debate went on for a really, really long time, and eventually, the program manager prevailed, but the final design came out much much better than any one individual’s design would have been.

To make sure that the debate happens respectfully and on a rational basis of facts, it’s absolutely critical that the program managers and developers be peers. If developers report to the program manager, at some point during the debate the program manager is going to get sick of the whole thing and just say, “OK, enough talking, now we do it my way.” When they’re peers, this can never happen. It’s a little bit like courts of law: we don’t allow a lawyer for one side to be the judge, and we work on the theory that the truth is most likely to be uncovered through a process of debate between equals. The debate can only be a fair one if neither side has an unfair advantage.

This is an important point, so if you were daydreaming about Sally in 11th grade, wondering where she is now, snap out of it. She’s a biotherapist in Scottsdale, and a Republican. Now pay attention. Programmers can’t report to program managers which means, among other things, that the development lead, or the CTO, or the CEO, can’t be the person who writes the specs.

The number one mistake most companies make is having the manager of the programmers writing the specs and designing the product. This is a mistake because the design does not get a fair trial, and is not born out of conflict and debate, so it’s not as good as it could be.

I learned this the hard way. At Fog Creek Software, I did a lot of the program management myself, and it was a constant battle to remind people that they were supposed to argue with me when I said wrong things. We’re not a big company but we are finally big enough to have real program managers now, Dan and Jason, and the programmers love arguing with them.

Of course, when programmers are peers of the program managers, the programmers tend to have the upper hand. Here’s something that has happened several times: a programmer asks me to intervene in some debate he is having with a program manager.

“Who is going to write the code?” I asked.

“I am…”

“OK, who checks things into source control?”

“Me, I guess, …”

“So what’s the problem, exactly?” I asked. “You have absolute control over the state of each and every bit in the final product. What else do you need? A tiara?”

You see, it turns out that this system puts the burden on the program manager to persuade the programmer, because at some point, the program manager runs the risk that the programmer will give up and just do whatever the heck the programmer feels like. Thus, being effective as a program manager means you have to (a) be right, and (b) earn the respect of the programmers so that they concede that you’re right.

How do you earn this respect?

It helps, as a program manager, to be pretty good at coding yourself. This is unfair. Program managers aren’t supposed to write code. But programmers tend to respect programmers a lot more than non-programmers, no matter how smart they are. It is possible to be an effective program manager without being a coder, but the burden of earning the respect of the programming team will be higher.

flip the bozo bit v. Decide that someone is a clown, and stop listening to them.

The other way to earn the programming team’s respect is to demonstrate intelligence, open-mindedness, and fairness in any debates that come up. If a program manager says dumb things, the programmer might flip the bozo bit on them. If a program manager becomes personally or emotionally attached to a certain way of doing things, to the point at which they’re being unreasonable, they’re going to lose a lot of credibility… both sides, but especially the program manager, need to be emotionally detached from the debate and willing to consider new evidence and change their opinions when the facts merit it. Finally, if a program manager is seen as playing politics, having private meetings with the boss or trying to divide-and-conquer to win a debate instead of debating on the merits, they’re going to lose a lot of trust of the programmers.

And when a program manager loses the programming team’s trust, it’s over. They’re not going to be effective. The programmers are going to tune them out and do whatever they want anyway. This leads to worse code and wasted time, since not only are you paying an ineffective program manager a salary, but that ineffective program manager is calling meetings and soaking up everybody else’s time even though they’re not really making the code any better.

Specs? Really? That’s so unagile

There are so many development organizations where specs are a monument to mindless bureaucratic paperwork that entire movements sprung up organized around the idea of not writing specs. These people are misguided. Writing a functional specification is at the very heart of agile development, because it lets you iterate rapidly over many possible designs before you write code. Compared to code, a written spec is trivial to change. The very act of writing a specification forces you to think through the design you thought you had in your head, and helps you see the flaws in it quickly so that you can iterate and try more designs. Teams that use functional specifications have better designed products, because they had the opportunity to explore more possible solutions quickly. They also write code faster, because they have a clearer picture when they start of what’s going to be needed. Functional specifications are so important one of the few hard and fast rules at Fog Creek is “No Code Without Spec.”

The exact form the functional specification takes may vary. All a functional specification has to do is explain how the program will behave. It doesn’t say anything about how the code will work internally. You start at the highest level: a vision statement, no more than one page explaining the gist of the new feature. Once that’s nailed down, you can develop storyboards… mockups of the screens showing the user’s progression through the application, with detailed notes showing how they work. For many types of functionality, especially UI-heavy functionality, once you have these storyboards, you’re done. That’s your spec. Jason Fried, you can go now.

To learn how to write good functional specifications, read my four part series. If you want to see a typical spec I wrote, you can download the full Fog Creek Copilot spec.

For more complex functionality with hidden behavior that’s not expressed in the UI storyboards, you’re going to want more details written down. In any case, the very act of writing down a spec helps you discover problems, conflicts, and design issues long before the first line of code is written, so when you do write the code, you have far fewer unexpected issues popping up which might force a rewrite or, worse, a suboptimal design.

How do you learn to be a Program Manager?

Mostly, becoming a program manager is about learning: learning about technology, learning about people, and learning how to be effective in a political organization. A good program manager combines an engineer’s approach to designing technology with a politician’s ability to build consensus and bring people together. While you’re working on that, though, there are a few books you should read:

As far as I can tell, Scott Berkun’s book Making Things Happen is the only book that’s been written that pretty much covers exactly what a program manager has to do, so start with that. Scott was a program manager on the Internet Explorer team for many years.

Another big part of the program manager’s job is user interface design. Read Steve Krug’s Don’t Make Me Think, then my own book User Interface Design for Programmers.

Finally, and I know it sounds cheesy, but Dale Carnegie’s 1937 book How to Win Friends & Influence People is actually a fantastic introduction to interpersonal skills. It’s the first book I make all the management trainees at Fog Creek read, before anything else, and they always snicker when I tell them to read it, and love it when they’re done.

It was getting time to stop futzing around with FogBugz 4.0 and start working on 5.0. We just shipped a big service pack, fixing a zillion tiny little bugs that nobody would ever come across (and introducing a couple of new tiny little bugs that nobody will ever come across) and it was time to start adding some gen-yoo-ine new features.

By the time we were ready to start development, we had enough ideas for improvement to occupy 1700 programmers for a few decades. Unfortunately, all we have is three programmers, and we wanted to be shipping next fall, so there had to be some prioritization.

Before I tell you how we prioritized our list of features, let me tell you two ways not to do it.

Number one. If you ever find yourself implementing a feature simply because it has been promised to one customer, RED DANGER LIGHTS should be going off in your head. If you’re doing things for one customer, you’ve either got a loose cannon sales person, or you’re slipping dangerously down the slope towards consultingware. And there’s nothing wrong with consultingware; it’s a very comfortable slope to slip down, but it’s just not as profitable as shrinkwrap software.

Shrinkwrap is the take-it-or-leave it model of software development. You develop software, wrap it in plastic, and customers either buy it, or they don’t. They don’t offer to buy it if you implement just one more feature. They don’t call you up and negotiate features. You can’t call up Microsoft and tell them, “Hey, I love that BAHTTEXT function you have in Excel for spelling out numbers in Thai, but I could really use an equivalent function for English. I’ll buy Excel if you implement that function.” Because if you did call up Microsoft here is what they would say to you:

“Thank you for calling Microsoft. If you are calling with a designated 4-digit advertisement code, press 1. For technical support on all Microsoft products, press 2. For Microsoft presales product licensing or program information, press 3. If you know the person at Microsoft you wish to speak to, press 4. To repeat, press Star.”

Notice? None of the choices was, “To negotiate what features need to be added to our products before you’ll buy them, press 5.”

Custom development is that murky world where a customer tells you what to build, and you say, “are you sure?” and they say yes, and you make an absolutely beautiful spec, and say, “is this what you want?” and they say yes, and you make them sign the spec in indelible ink, nay, blood, and they do, and then you build that thing they signed off on, promptly, precisely and exactly, and they see it and they are horrified and shocked, and you spend the rest of the week reading up on whether your E&O insurance is going to cover the legal fees for the lawsuit you’ve gotten yourself into or merely the settlement cost. Or, if you’re really lucky, the customer will smile wanly and put your code in a drawer and never use it again and never call you back.

Somewhere in the middle is consultingware, where you pretend to be doing shrinkwrap while really doing custom development. Here’s how consultingware works:

1. You’re working as a wage-slave writing code for a shoe company, and
2. the company needs shoe-shining software, so
3. you develop shoe-shining software in VB 3.0 using bits and pieces of JavaScript, Franz Lisp, and a FileMaker database running on an old Mac connected over the network using AppleScript, then
4. everyone thinks it’s the cat’s whiskers, so, always having dreamed of starting your own software company and maybe being Bill Gates or perhaps even just Larry Ellison
5. you buy the rights to ShoeShiner 1.0 from your old company and get VC to start your own company, ShoeShiner LLC, marketing shoe-shining software, but
6. none of your beta testers can get it to work because of the obscure dependencies on AppleScript and the hardcoded IP addresses in the source code, so it takes a month to install at each client site, and
7. you have trouble getting clients, because your product is so expensive because of all the installation costs, including the vintage Macintosh IIci running System 7 which they have to buy on ebay from computer museums, so your VCs start to get really nervous,
8. putting pressure on the sales guys,
9. who find out that one of your potential customers doesn’t need a shoe-shining thing but he could really use trousers-pressing software, and
10. the sales guy, being a sales guy, sells him $100K worth of trousers-pressing software,
11. and now you spend 6 months writing a one-off “trousers-pressing module” for this client, which
12. no other client will ever need, thus, effectively,
13. for all intents and purposes you’ve just spent a year raising VC so that you could work as a wage-slave writing code for a trouser company; GOTO 1.

Sparky, I’m gonna have to strongly recommend clinging as strongly as possible to the shrinkwrap side of the equation. That’s because shrinkwrap has no marginal costs for each additional customer, so you can essentially sell the same thing over and over again and make a lot more profit. Not only that, but you can lower the price, because you can spread your development costs out over a lot more customers, and lowering the price gets you more customers because more people will suddenly find your now-cheaper software worthwhile, and life is good and all is sweet.

Thus. If you ever find yourself implementing a feature simply because it has been promised to a customer, you’re drifting over to the land of consultingware and custom development, which is a fine world to operate in if that’s what you like, but it just doesn’t have the profit potential of off-the-shelf commercial software.

Now, I’m not saying you shouldn’t listen to your customers. I for one think that it’s about time Microsoft actually implemented a version of the BAHTTEXT function for those of us who haven’t yet joined the global economy and learned Thai and who still write checks using other currencies. And in fact if you want to tell yourself that the best way to allocate your development resources is effectively to let your biggest customers “bid” on features, well, you can do that too, although you’ll soon find that the kind of features that big, rich customers want are not the same as the kind of features that the mass market wants, and that feature you put in to handle Baht currency is not really helping you sell Excel to health spas in Scottsdale, Arizona, and in fact what you’re really doing is letting your sales force pimp out your developers with the sole goal of maximizing their personal commissions.

The path to being Bill Gates, this is not.

Now, let me tell you the second way not to decide what features to implement. Don’t do things just because they’re inevitable. Inevitability is not a high enough bar. Let me explain.

Some time during the first year of Fog Creek’s operations, I was filing away some papers, and realized that I was all out of blue folders.

Now, I have a system. Blue folders are for clients. Manila folders are for employees. Red folders are receipts. Everything else is yellow. I needed a blue folder and had run out.

So I said to myself, “What the heck, I’m going to need a blue folder eventually anyway, I might as well go to Staples and buy some now.”

Which was, of course, a waste of time.

In fact when I thought about this later, I realized that for a long time, I had been doing dumb shit (that’s a technical term) simply because I figured that eventually it would have to get done, so I might as well do it now.

I used this excuse to weed the garden, patch holes in the walls, sort out MSDN disks (by color, language, and number), etc., etc., when I should have been writing code or selling code, the only two things a startup really needs to do.

In other words, I found myself pretending that all non-optional tasks were equally important, and therefore, since they were inevitable anyway, they could be done in any order! Tada!

But to be honest, I was just procrastinating.

What should I have done? Well, for starters, I could get over my fetish for having file folders all be the right color. It just doesn’t make any difference. You don’t have to color-code your files.

Oh, and those MSDN CD-ROMs? Toss them in a big box. PER-fect.

More importantly, I should have realized that “important” is not a binary thing, it’s an analog thing. There are all kinds of different shades of important, and if you try to do everything, you’ll never get anything done.

So if you want to get things done, you positively have to understand at any given point in time what is the most important thing to get done right now and if you’re not doing it, you’re not making progress at the fastest possible rate.

Slowly, I’m weaning myself off of my tendency to procrastinate. I’m doing this by letting less-important things go undone. There’s some nice lady from the insurance company who has been pestering me for two months to get some data she needs to renew our policy, and I didn’t actually get her the data until she asked about the fiftieth time, along with a stern warning that our insurance is going to expire in three days. And this is a good thing, I think. I’ve grown to think that keeping your desk clean is actually probably a sign that you’re not being effective.

How’s that for a mortifying thought!

So. Don’t do features based on what the sales force has inadvertently promised a single customer, and don’t do unimportant-slash-fun features first because “you’re going to have to do them eventually anyway.”

Anyway, back on the topic of choosing features for FogBugz 5.0. Here’s how we got our initial prioritization.

First, I took a stack of 5×8 cards, and wrote a feature on each one. Then I called the team together. In my experience this works with up to about twenty people, and it’s a good idea to get as many different perspectives in the room: programmers, designers, people who talk to customers, sales, management, documentation writers and testers, even (!) customers.

I asked everyone to bring their own list of feature ideas to the meeting, too. The first part of the meeting was going over each feature very, very quickly and making sure we had a very, very rough common understanding of what the feature was, and that each feature had a card.

At this stage, the idea was not to debate any feature on its merits, or to design the feature, or even to discuss the feature: just to have a vague, rough idea of what it was. Some of the features for FogBugz 5.0 were things like

• Personalizable Home Page
• Painless Software Schedules
• Track Billable Time
• Fork a bug
• (46 others…)

Very vague stuff. Remember we didn’t need to know at this point how each feature would be implemented, or what it involved, because our only goal was getting a rough prioritization that could be used as the basis to start development. This got us a list of about 50 big features.

In part two, we went through all of the features and everybody voted on each feature: just a quick “thumbs up” or “thumbs down.” No discussion, no nothing: just a real quick thumbs up or thumbs down on each feature. This revealed that about 14 of the feature ideas didn’t have much support. I threw out all the features that only got one or two votes, leaving us with 36 potential features.

Next we assigned costs for each of these features, on a scale of 1 to 10, where 1 was a quicky feature and 10 was a big monster feature. Here it’s important to remember that the goal was not to schedule the features: just to separate the tiny features from the medium features from the huge features. I just went through each of the features and asked the developers to call out “small,” “medium,” or “large.” Even without knowing how long a feature is going to take, it’s easy to see that forking a bug is a “small” feature while the big, vague “Personalizable Home Page” feature was large. Based on the consensus estimate of costs and my own judgment, we put down prices on all the features:

Once again, it’s really messy, it’s not exact, and it doesn’t matter. You’re not making a schedule today: you’re just prioritizing. The only thing that you have to get approximately right is the vague idea that you could do two medium features or one large feature or ten small features in about the same amount of time. It doesn’t have to be accurate.

The next step was making a menu of all thirty proposed features and their “costs”. Everybody on the team got a copy of the menu and was given $50 to play with. They could allocate their money any way they wanted, but they only had $50 to spend. They could buy half-features, if they wanted, or buy double features. Someone who really liked that Track Billable Time feature could spend $10 or $15 on it; someone who liked it a little might only spend $1 and hope that enough other people funded it.

Next, we added up how much everyone spent on each feature:

Finally I divided the amount spent by the cost:

And then sorted by this number to find the most popular features:

Ta da! A list of all the features you might want to do, in rough order of everyone’s best idea of which features are the most important.

And now you can start to refine. For example, you can clump together features that naturally belong together, for example, doing software schedules makes billable time easier, so maybe we should either do both or neither. And sometimes looking down the prioritized list it’s just real obvious that something is messed up. So, you change it! Nothing is carved in stone. You can even change the prioritization as you go through development.

But what surprised me the most is that the final list we produced was really a very good prioritization for FogBugz 5.0, and really did reflect our collective consensus about the relative priorities of various features.

Priority list in hand, we set out to more or less work down the list in order until about March, when we plan to stop adding new features and start the integration and testing phase. We’ll write specs for each (nonobvious) feature right before implementing that feature.

(The nattering scorekeepers of the BDUF/Agile beauty contest are now thoroughly confused. “Was that a vote for BDUF? Or Agile? What does he want? Can’t he just take sides for once?!“)

The whole planning process took three hours.

If you’re lucky enough to have the ability to release software more frequently than we do, (see Picking a Ship Date), you still need to work down the list in order, but you can just stop and do releases more often. The good thing about frequent releases is that you can reprioritize the list regularly based on actual customer feedback, but not every product has this luxury.

Mike Conte taught me this system during the planning of Excel 5, where it only took a couple of hours even with a couple of dozen people in a conference room. The cool thing was that the roughly 50% of the features that we didn’t have time to do were really stupid features, and Excel was better because it didn’t have them.

It’s not perfect, but it’s better than going to Staples to get blue folders, I’ll tell ya that.

 

The last time I participated in formal usability testing was at a fancy lab in Colorado, custom built for the purpose at a cost of around $100,000. It was basically a television studio, complete with one-way glass, lots of special video gear, and a giant video console that would have been adequate to broadcast the Superbowl. To do usability testing for Juno, a group of us flew out to Colorado, rented cars, stayed in a hotel, ate at expensive restaurants, and generally consumed massive amounts of money so we could watch people try to sign up for our online service, and, generally, succeed.

At the other extreme, I’ve long been an advocate of hallway usability tests and paper prototypes, which often find some of the biggest usability problems long before they occur for about fifty cents.

Now there’s a middle ground. My friends over at TechSmith in Okemos, Michigan recently released a software product called Morae which lets you use cheap webcams to set up a complete usability lab in your office without fancy equipment or one-way glass. I asked them if they would be willing to usability test their own product by running a usability test in the Fog Creek office for our new remote assistance service, and they graciously agreed.

Here’s how Morae works. You set up your usability testing subject in front of a computer with a webcam and a microphone:

Then any number of people can watch the subject from their own computers:

Here Tyler is watching two screens: one is showing the helper and the other is showing the person being helped. He can see their screens, hear everything they say, and see video of the subject in the corner. We happen to have windows between the offices at Fog Creek so he can actually see the helper through his window. Let’s see if I can zoom in on Tyler’s screens:

In my book on UI design, I wrote about a common problem with usability tests:

In most usability tests, you prepare a list of instructions for the user. For example, if you were usability testing an Internet access provider, you might have an instruction to “sign up for the service.” (I actually did this very usability test, several times, in my career.)

So far so good. The first user comes in, sits down, starts signing up for the service, and gets to the screen asking them how they want to pay. The user looks at me helplessly. “Do I gotta pay for this myself?”

“Oh wait,” you interrupt. “Here, use this fake credit card number.”

The sign up procedure then asks them if they would like to use a regular modem, a cable modem, or a DSL line.

“What do I put here?” asks the user. Possibly because they don’t know the answer, but possibly because they know the answer for their computer, but they’re not using their computer, they’re using your computer, which they’ve never seen before, in a usability lab, where they’ve never been before.

To work around this problem, usability testers have started trying to do field testing. Instead of giving the subject tasks to do in a highly contrived environment, you conspire to watch the subject doing their own work at their own desk while you hide in a nearby shrub and spy on them. Morae, by the way, would be perfect for that. This method is most useful when you already have a version n of your product and you’re trying to figure out how to improve version n+1.

The usability test worked great. Our usability test was a little bit uncommon in that we had two subjects, since Fog Creek Copilot involves a helper and a helpee, and Morae only let us hear one subject at a time from one computer. To work around this we just set up two computers with the Morae Remote Viewer so we could get the sound from both subjects.

So far this morning we’ve run two usability test sessions, with great results: we’ve already realized that 2 out of 2 helpers were confused about how to get reconnected, since the Fog Creek Copilot helper application deletes itself when you’re done with it. This is a classic example of the user model not conforming to the program model … most programs don’t delete themselves! … which is the source of virtually every usability problem. From the very first chapter of UI for Programmers:

The cardinal axiom of all user interface design:

A user interface is well-designed when the program behaves exactly how the user thought it would.

I should have known this. The program design violated a principle I wrote myself in big bold print in my own book: it didn’t do what you expected. The great thing about usability tests is with a day of usability testing and handful of subjects, even if you’re as senile as I am, you can find the biggest areas where you didn’t realize where the program’s behavior diverges from the user’s expected behavior.

Here are a few tips for running a beta test of a software product intended for large audiences — what I call “shrinkwrap“. These apply for commercial or open source projects; I don’t care whether you get paid in cash, eyeballs, or peer recognition, but I’m focused on products for lots of users, not internal IT projects.

1. Open betas don’t work. You either get too many testers (think Netscape) in which case you can’t get good data from the testers, or too few reports from the existing testers.
2. The best way to get a beta tester to send you feedback is to appeal to their psychological need to be consistent. You need to get them to say that they will send you feedback, or, even better, apply to be in the beta testing program. Once they have taken some positive action such as filling out an application and checking the box that says “I agree to send feedback and bug reports promptly,” many more people will do so in order to be consistent.
3. Don’t think you can get through a full beta cycle in less than eight to ten weeks. I’ve tried; lord help me, it just can’t be done.
4. Don’t expect to release new builds to beta testers more than once every two weeks. I’ve tried; lord help me, it just can’t be done.
5. Don’t plan a beta with fewer than four releases. I haven’t tried that because it was so obviously not going to work!
6. If you add a feature, even a small one, during the beta process, the clock goes back to the beginning of the eight weeks and you need another 3-4 releases. One of the biggest mistakes I ever made was adding some whitespace-preserving code to CityDesk 2.0 towards the end of the beta cycle which had some, shall we say, unexpected side effects that a longer beta would have fleshed out.
7. Even if you have an application process, only about one in five people will send you feedback anyway.
8. We have a policy of giving a free copy of the software to anyone who sends any feedback, positive, negative, whatever. But people who don’t send us anything don’t get a free copy at the end of the beta.
9. The minimum number of serious testers you need (i.e., people who send you three page summaries of their experience) is probably about 100. If you’re a one-person shop, that’s all the feedback you can handle. If you have a team of testers or beta managers, try to get 100 serious testers for every employee that is available to handle feedback.
10. Even if you have an application process, only one out of five testers is really going to try the product and send you feedback. So, for example, if you have a QA department with 3 testers, you should approve 1500 beta applications to get 300 serious testers. Fewer than this and you won’t hear everything. More than this and you’ll be deluged with repeated feedback.
11. Most beta testers will try out the program when they first get it, and then lose interest. They are not going to be interested in retesting it every time you drop them another build unless they really start using the program every day, which is unlikely for most people. Therefore, stagger the releases. Split your beta population into four groups and each new release, add another group that gets the software, so there are new beta testers for each milestone.
12. Don’t confuse a technical beta with a marketing beta. I’ve been talking about technical betas, here, in which the goal is to find bugs and get last-minute feedback. Marketing betas are prerelease versions of the software given to the press, to big customers, and to the guy who is going to write the Dummies book that has to appear on the same day as the product. With marketing betas you don’t expect to get feedback (although the people who write the books are likely to give you copious feedback no matter what you do, and if you ignore it, it will be cut and pasted into their book).

One of the best reasons to make a detailed schedule is because it gives you an excuse to cut features. If there’s no way to make the ship date and implement Bob’s Singalong MP3 Chat feature, it’s easy to cut that feature without making Bob feel bad.

So my basic rule for software release cycles is:

1. Set a ship date, which might as well be arbitrary
2. Make a list of features and sort them out by priority
3. Cut low-priority features every time you slip so as to make the date.

If you do this well, you’ll soon discover that you don’t regret the features that you cut. They’re usually kind of dumb. If they were so important, you can do them next time. It’s like editing. If you ever want to write a brilliant 750 word essay, start by writing a 1500 word essay and then edit.

One way to screw this up, by the way, is forgetting to do features in order of priority. If you aren’t careful, your programmers are going to do features in order of fun, and you won’t be able to ship or cut features because all the programmers were busy writing Karaoke Recalc before they even got the menus to work, and now here it is, six months after the day you wanted to ship, and you have one hell of an easter egg but no functionality.

So the obvious question is: how do you pick a ship date?

It’s conceivable that you have outside constraints. The stock market is switching from fractions to decimals on such-and-such a date, and if you haven’t got the new software ready, your firm will be forced out of business and you personally will be taken out behind the loading docks and shot in the head. Or maybe a new version of the Linux kernel is coming out soon with yet another all-new system to implement packet filtering; all your customers are getting it; and your existing application won’t run on it. OK, for those people, your ship date is easy to figure out. You can stop reading this article now. Go cook a nice dinner for your loved ones.

Bye, now!

But how should the rest of us pick a ship date?

There are three approaches you could take.

1. Frequent Small Releases. This is the Extreme Programming approach, and is most appropriate for small-team projects with a small number of customers, such as in-house IT development.
2. Every 12 to 18 months. This is typical of shrinkwrapped software products, desktop applications, etc., where you have larger teams and thousands or millions of customers.
3. Every 3-5 years. This is typical of huge software systems and platforms that are worlds unto themselves. Operating systems, .Net, Oracle, and for some reason Mozilla fall into this category. They often have thousands of developers (VS.Net had 50 people on the installer team) and enormously complex interactions to other shipping software which can’t be allowed to break.

Here are some of the things you need to think about when deciding how often to release software.

Short releases get customer feedback fast. Sometimes the best way to work with a customer is to show them code, let them try it out, and immediately incorporate their feedback into a build which you give them the very next day. You won’t waste a year developing a complicated system with lots of features that nobody uses, because you’ll be so busy doing things that customers are requesting right now. If you have a small number of customers, prefer frequent small releases. The size of each release should be the minimum chunk that does something useful.

Several years ago I was assigned to develop a web content management system for MTV. The requirements called for a database-backed system with templates, and a complete workflow system that allowed unpaid MTV stringers at colleges across the country to input information about clubs, record stores, radio stations, and concerts. “How are you building the site now?” I asked.

“Oh, we just do it all manually with BBEdit,” they told me. “Sure, there are thousands of pages, but BBEdit has a really good global-find-and-replace function…”

I figured the whole system would take six months to deliver. “But let me suggest something else. Let’s get the templating stuff working first. I can get you that in 3 months and it will save tons of manual work right away. Once that’s working we’ll start in on the workflow component; in the meantime you can continue to do workflow with email.”

They agreed. It sounded like a great idea. Guess what? Once I delivered the templating feature, they realized that they didn’t really need workflow that much. And the templating turned out to be useful for lots of other web sites which didn’t need workflow, either. So we never built workflow, saving three months which I used to enhance the templating feature which turned out to be more useful.

Some types of customers don’t appreciate being “guinea pigs” in this fashion. Generally, people who buy off-the-shelf software don’t want to be part of a Grand Development Experiment; they want something that anticipates their needs. As a customer, the only thing better than getting feature requests done quickly is getting them instantaneously because they’re already in the product, because it was designed thoughtfully and extensively usability- and beta-tested before being inflicted on the world. If you have (or want) a large number of paying customers, prefer less frequent releases.

If you ship an anemic commercial program just to get something out the door so you can “start listening to customers,” what you’ll hear those customers saying is “it doesn’t do very much,” which you might think is OK. Hey, it’s 1.0. But then if you release 2.0 four months later, everybody’s going to think, “That feeble program? What am I, supposed to keep evaluating it every four months just to see if it’s gotten better yet?!” And in fact five years down the line, people will still remember their first impression of 1.0, and it will be almost impossible to get them to reevaluate. Think about what happened to poor Marimba. They launched their company with infinite VC in the days of hyper-Java-hype, having lured the key developers from the Java team at Sun. They had a CEO, Kim Polese, who was brilliant at public relations; when she was marketing Java she had Danny Hillis making speeches about how Java was the next step in human evolution; George Gilder wrote these breathless articles about how Java was going to completely upturn the very nature of human civilization. Compared to Java, we were to believe, monotheism, for example, was just a wee blip. Polese is that good. So when Marimba Castanet launched it probably had more unearned hype than any product in history, but the developers had only been working on it for a total of … four months. We all downloaded it and discovered that — tada! — it was a list box that downloaded software. (What do you expect from four months of development?) Big whoop. The disappointment was so thick you could cut it with butter. And here it is, six years later, ask anybody what Castanet is and they’ll tell you it’s a list box that downloads software. Hardly anyone bothered to reevaluate it, and Marimba has had six years to write code; I’m sure it’s just the coolest thing now but, honestly, who has time to find out? Let me tell you a little secret: our strategy for CityDesk is to avoid massive PR until 2.0 is out. That’s the version that we want everybody on earth to get their first impressions from. In the meantime we’ll do quiet guerilla marketing, and anybody who finds it will discover that it’s a completely spiffy program that solves a lot of problems, but Arnold Toynbee won’t have to rewrite anything.

With most commercial software, you’ll discover that the process of designing, prototyping, integrating, fixing bugs, running a full alpha and beta cycles, creating documentation, and so forth takes 6-9 months. In fact if you try to do a full release every year, you only have time for about 3 months worth of new code development. Software that is upgraded annually usually doesn’t feel like it has enough new features to justify the upgrade. (Corel PhotoPaint and Intuit Quickbooks are particularly egregious examples of this; they have a new “major” version every year which is rarely worth buying). As a result many people have learned by now to skip every other release. You don’t want your customers getting in that habit. If you stretch out your schedule to 15 or 18 months between releases, you get 6 months of new features instead of 3 months worth, which makes your upgrade a lot more compelling.

OK, if 15 months is so good, wouldn’t 24 months be better? Maybe. Some companies can get away with it, if they are major leaders in their category. PhotoShop seems to get away with it. But as soon as an application starts to feel old, people stop buying it because they are expecting that new version Any Day Now. This can cause serious cash flow problems for a software business. And of course you may have competitors nipping at your heels.

For large platform software — operating systems, compilers, web browsers, DBMSs — the hardest part of the development process is maintaining compatibility with thousands or millions of existing applications or hardware. When a new version of Windows comes out, you very rarely hear about backwards compatibility problems. The only way they can achieve this is with insane amounts of testing that make the construction of the Panama Canal seem like a weekend do-it-yourself project. Given the typical 3 year cycle between major Windows releases, almost all of that time is spent in the boring integration and testing phase, not writing new features. Releasing new versions any more often than that is just not realistic. And it would drive people crazy. Third party software and hardware developers would simply revolt if they had to test against lots of little incremental releases of an operating system. For Systems With Millions of Customers and Millions of Integration Points, Prefer Rare Releases. You can do it like Apache: one release at the beginning of the Internet Bubble, and one release at the end. Perfect.

If you have a lot of validation and unit tests, and if you write your software carefully, you may get to the point where any daily build is almost high enough quality to ship. This is certainly something to strive for. Even if you’re planning the next release for three years from now, the competitive landscape may suddenly change and there may be a good reason to do a quick interim release to react to a competitor. When your wife is about to give birth, it’s not really a good idea to take apart your car’s engine. Instead build a new one on the side and don’t hook it up until it’s perfect.

But don’t overestimate what you can accomplish by keeping high quality daily builds. Even if you are permanently at zero bugs, if your software has to run In The Wild, you’re never going to find all the compatibility bugs and Windows 95 bugs and It-Doesn’t-Work-With-Large-Fonts-Turned-On bugs until you do a few betas, and there is no realistic way to do a beta cycle in less than 8 weeks.

One final thought. If your software is delivered as a service over the web, like ebay or PayPal, theoretically there’s nothing to stop you from frequent small releases, but this might not be the best thing to do. Remember the cardinal rule of usability: an application is usable if it behaves the way that the user expected it to behave. If you keep changing things around every week, it won’t be so predictable, so it won’t be so usable. (And don’t think you can get around this by having one of those annoying screens with a paragraph of text saying Warning! The UI has changed! Nobody reads those.) From a usability perspective a better approach would probably be less frequent releases that include a bunch of changes all at once, where you make an effort to change the visuals of the whole site so that it all looks weird and users intuit that things have changed a lot and they have to be careful.