Doug Kaye interviewed me for IT Conversations; you can listen to the audio interview. Dave Walker also interviewed me for the Sydney Morning Herald and Melbourne’s The Age; an extended verison of that article is on his site Shorewalker.com. Finally, Mary Jo Foley came by the Fog Creek office to interview me for Microsoft Watch, which is a subscription-only newsletter.
Month / September 2004
Incrementalists versus Completionists
Rands: “Completionists are dreamers. They have a very good idea of how to solve a given problem and that answer is SOLVE IT RIGHT. Their mantra is, ‘If you’re going to spend the time to solve a problem, solve it in a manner that you aren’t going to be solving it AGAIN in three months.’”
We Won’t Be Fooled Again
Dan Appleman: “Sure, if you’re writing software with a lifespan of a few years, Windows Forms is a great way to go. But we all know that software, enterprise software especially, lives a long time. Can Microsoft categorically promise to maintain a full commitment to development, maintenance and support of Windows Forms for the next 15 years?”
Discussion Group Software
We’ve been quietly making some improvements to the beta discussion group software.
Today we rolled out Brett’s new full-text search feature. It relies on the database engine to provide full-text search, and we’re running Microsoft SQL Server, which has rather poor full-text search capabilities: for example, it requires a manual process to rebuild the index, which we schedule for every 15 minutes, so it won’t find anything posted in the last few minutes.
I also added an RSS feed. Originally I wanted to provide full text of all topics and replies in the last three days so that you could use an RSS reader to read the discussion group. Unfortunately that would have resulted in a huge download, and since RSS readers bang on the site every hour or three, our bandwidth usage would have been absurd. So I had to settle for full text of the original topic but not of replies.
And finally we got Summer Intern Ben’s excellent Bayesian filtering code working… due to a couple of configuration problems it wasn’t running right. The idea is to delete comment spam before anyone sees it. It’s hard to tell if the filter works yet because it needs more training, but so far it’s doing pretty well. If you think comment spam is not a big problem, you haven’t moderated a discussion group lately… this is the number one priority for spammers these days, since email filters are starting to work pretty well and spamming a lot of discussion groups is perceived as a good way to trick Google into giving a site prominent placement.
We admit to three strategies to prevent comment spam:
- Bayesian filtering which can be trained to remove comment spam instantly
- Not allowing new comments on old posts, so that comment spam can’t be hidden in posts which nobody but Google visits any more
- Using a META tag to ask search engines not to follow URLs from discussion topics. Although this technique prevents comment spam from working it doesn’t prevent it from happening because spammers don’t seem to particularly care if a given spam works or not.
By “we admit to” I imply that there are other things we do which we don’t talk about too much because revealing them would make it that much easier for spammers to work around them, thus reducing the cost of spamming, thus making it more economically feasible.
One side affect of the Bayesian filter is that if it finds a suspicious topic, rather than letting it through, it will flag it for a human moderator. The moderator can then allow it to be posted (which trains the filter) or leave it unshown. The effect of this is that rarely, new posts won’t appear until a human approves them. This should happen less and less as the filter learns more.
It’s Not Just Usability
For years and years, self-fashioned pundits, like, uh, me, have been nattering endlessly about usability, and how important it is to make software usable. Jakob Nielsen has a mathematical formula he’ll reveal to you in exchange for $122 which you can use to calculate the value of usability. (If the expected value of usability is greater than $122, I guess you make a profit.)
I have a book you can buy for a lot less that tells you some of the principles of designing usable software, but there’s no math involved, and you’ll be out the price of the book.
In that book, on page 31, I showed an example from what was, at the time, the most popular software application on Earth, Napster. The main Napster window used buttons to flip between the five screens. Due to a principle in usability called affordance, instead of buttons, it really should have had tabs, which was the point I was trying to make.
And yet, Napster was the most popular software application on Earth.
In an early version of the manuscript, I actually wrote something like “this just goes to show you that usability ain’t all that important,” which was an odd thing to be writing in a book about usability. I was sort of relieved when the typesetter announced that I had to shorten that paragraph. So I deleted the sentence.
But there’s a scary element of truth to it—scary to UI professionals, at least: an application that does something really great that people really want to do can be pathetically unusable, and it will still be a hit. And an application can be the easiest thing in the world to use, but if it doesn’t do anything anybody wants, it will flop. UI consultants are constantly on the defensive, working up improbable ROI formulas about the return on investment clients will get from their $75,000 usability project, precisely because usability is perceived as “optional,” and the scary thing is, in a lot of cases, it is. In a lot of cases. The CNN website has nothing to be gained from a usability consultant. I’ll go out on a limb and say that there is not a single content-based website online that would gain even one dollar in revenue by improving usability, because content-based websites (by which I mean, websites that are not also applications) are already so damn usable.
My goal today is not to whine about how usability is not important… usability is important at the margins, and there a lots of examples where bad usability kills people in small planes, creates famine and pestilence, etc.
My goal today is to talk about the next level of software design issues, after you’ve got the UI right: designing the social interface.
I need to explain that, I guess.
Software in the 1980s, when usability was “invented,” was all about computer-human interaction. A lot of software still is. But the Internet brings us a new kind of software: software that’s about human-human interaction.
Discussion groups. Social networking. Online classifieds. Oh, and, uh, email. It’s all software that mediates between people, not between the human and the computer.
When you’re writing software that mediates between people, after you get the usability right, you have to get the social interface right. And the social interface is more important. The best UI in the world won’t save software with an awkward social interface.
The best way to illustrate social interfaces is with a few examples of failures and successes.
First, a failing social interface. Every week I get an email from somebody I’ve never heard of asking me to become a part of his or her social network. I usually don’t know the person, so I feel a little bit miffed and delete the message. Someone told me why this happens: one of those social networking software companies has a tool that goes through your email address book and sends email to everyone asking them to join in. Now, combine this with the feature that some email software saves the sender’s address of every incoming message, and the feature that when you go to sign up for the Joel on Software email bulletin you get a confirmation message asking if you really want to join, and voila: all kinds of people who I don’t know are running software that is inadvertently asking me to confirm that I’m their friend. Thank you for subscribing to my newsletter, but no, I’m not going to introduce you to Bill Gates. I currently have a policy of not joining any of these social networks, because they strike me as going strongly against the grain of how human networks really work.
Now, let’s look at a successful social interface. Many humans are less inhibited when they’re typing than when they are speaking face-to-face. Teenagers are less shy. With cellphone text messages, they’re more likely to ask each other out on dates. That genre of software was so successful socially that it’s radically improving millions of people’s love lives (or at least their social calendars). Even though text messaging has a ghastly user interface, it became extremely popular with the kids. The joke of it is that there’s a much better user interface built into every cellphone for human to human communication: this clever thing called “phone calls.” You dial a number after which everything you say can be heard by the other person, and vice versa. It’s that simple. But it’s not as popular in some circles as this awkward system where you break your thumbs typing huge strings of numbers just to say “damn you’re hot,” because that string of numbers gets you a date, and you would never have the guts to say “damn you’re hot” using your larynx.
Another social software success is ebay. When I first heard about ebay, I said, “Nonsense! That will never work. Nobody’s going to send money to some random person they encountered on the Internet in hopes that person will out of the goodness of their hearts actually ship them some merchandise.” A lot of people thought this. We were all wrong. Wrong, wrong, wrong. Ebay made a big bet on the cultural anthropology of human beings and won. The great thing about ebay is that it was a huge success precisely because it seemed like a terrible idea at the time, and so nobody else tried it, until ebay locked in the network effects and first-mover advantage.
In addition to absolute success and failures in social software, there are also social software side effects. The way social software behaves determines a huge amount about the type of community that develops. Usenet clients have this big-R command which is used to reply to a message while quoting the original message with those elegant >’s in the left column. And the early newsreaders were not threaded, so if you wanted to respond to someone’s point coherently, you had to quote them using the big-R feature. This led to a particularly Usenet style of responding to an argument: the line-by-line nitpick. It’s fun for the nitpicker but never worth reading. (By the way, the political bloggers, newcomers to the Internet, have reinvented this technique, thinking they were discovering something fun and new, and called it fisking, for reasons I won’t go into. Don’t worry, it’s not dirty.) Even though human beings had been debating for centuries, a tiny feature of a software product produced a whole new style of debating.
Small changes in software can make big differences in the way that software supports, or fails to support, its social goals. Danah Boyd has a great critique of social software networks, Autistic Social Software, blasting the current generation of this software for forcing people to behave autistically:
Consider, for a moment, the recent surge of interest in articulated social networks such as Friendster, Tribe, LinkedIn, Orkut and the like. These technologies attempt to formalize how people should construct and manage their relationships. They assume that you can rate your friends. In some cases, they procedurally direct how people can engage with new people by giving you an absolute process through which you can contact others.
While this approach certainly has its merits because it is computationally possible, i’m terrified when people think that this models social life. It’s so simplistic that people are forced to engage as though they have autism, as though they must interact procedurally. This approach certainly aids people who need that kind of systematization, but it is not a model that universally makes sense. Furthermore, what are the implications of having technology prescribe mechanistic engagement? Do we really want a social life that encourages autistic interactions?
When software implements social interfaces while disregarding cultural anthropology, it’s creepy and awkward and doesn’t really work.
Designing Social Software
Let me give you an example of social interface design.
Suppose your user does something they shouldn’t have done.
Good usability design says that you should tell them what they did wrong, and tell them how to correct it. Usability consultants are marketing this under the brand name “Defensive Design.”
When you’re working on social software, this is too naive.
Maybe the thing that they did wrong was to post an advertisement for Viagra on a discussion group.
Now you tell them, “Sorry, Viagra is not a suitable topic. Your post has been rejected.”
Guess what they’ll do? They’ll post an advertisement for V1agra. (Either that, or they’ll launch into a long boring rant about censorship and the First Amendment.)
With social interface engineering, you have to look at sociology and anthropology. In societies, there are freeloaders, scammers, and other miscreants. In social software, there will be people who try to abuse the software for their own profit at the expense of the rest of the society. Unchecked, this leads to something economists call the tragedy of the commons.
Whereas the goal of user interface design is to help the user succeed, the goal of social interface design is to help the society succeed, even if it means one user has to fail.
So a good social interface designer might say, let’s not display an error message. Let’s just pretend that the post about Viagra was accepted. Show it to the original poster, so he feels smug and moves on to the next inappropriate discussion group. But don’t show it to anyone else.
Indeed one of the best ways to deflect attacks is to make it look like they’re succeeding. It’s the software equivalent of playing dead.
No, it doesn’t work 100% of the time. It works 95% of the time, and it reduces the problems you’ll have twenty-fold. Like everything else in sociology, it’s a fuzzy heuristic. It kind of works a lot of the time, so it’s worth doing, even if it’s not foolproof. The Russian mafia with their phishing schemes will eventually work around it. The idiot Floridians in trailer parks trying to get rich quick will move on. 90% of the spam I get today is still so hopelessly naive about spam filters that it would even get caught by the pathetic junk filter built into Microsoft Outlook, and you’ve got to have really lame spam to get caught by that scrawny smattering of simplistic searchphrases.
Marketing Social Interfaces
A few months ago I realized that a common theme in the software we’ve built at Fog Creek is an almost obsessive attention to getting the social interfaces right. For example, FogBugz has lots of features, and even more non-features, designed to make bug tracking actually happen. Time and time again customers tell me that their old bug tracking software was never getting used, because it did not align with the way people wanted to work together, but when they rolled out FogBugz, people actually starting using it, and became addicted to it, and it changed the way they worked together. I know that FogBugz works because we have a very high upgrade rate when there’s a new version, which tells me FogBugz is not just shelfware, and because even customers who buy huge blocks of licenses keep coming back for more user licenses as the product spreads around their organization and really gets used. This is something I’m really proud of. Software used in teams usually fails to take hold, because it requires everyone on the team to change the way they work simultaneously, something which anthropologists will tell you is vanishingly unlikely. For that reason FogBugz has lots of design decisions which make it useful even for a single person on a team, and lots of design features which encourage it to spread to other members of the team gradually until everyone is using it.
The discussion group software used on this site, which will soon be for sale as a feature of FogBugz 4.0, is even more obsessed with getting the social interface aspects exactly right. There are dozens of features and non-features and design decisions which collectively lead to a very high level of interesting conversation with the best signal-to-noise ratio of any discussion group I’ve ever participated in. I wrote a lot about this in my article Building Communities with Software.
Since then, I’ve become even more devoted to the idea of the value of good social interface design: we bring in experts like Clay Shirky (a pioneer in the field), we do bold experiments on the poor citizens of the Joel on Software discussion group (many of which are so subtle as to be virtually unnoticeable, for example, the fact that we don’t show you the post you’re replying to while you type your reply in hopes of cutting down quoting, which makes it easier to read a thread), and we’re investing heavily in advanced algorithms to reduce discussion group spam.
A New Field
Social interface design is still a field in its infancy. I’m not aware of any books on the subject; there are only a few people working in the research side of the field, and there’s no organized science of social interface design. In the early days of usability design, software companies recruited ergonomics experts and human factors experts to help design usable products. Ergonomics experts knew a lot about the right height for a desk, but they didn’t know how to design GUIs for file systems, so a new field arose. Eventually the new discipline of user interface design came into its own, and figured out the concepts like consistency, affordability, feedback, etc., which became the cornerstone of the science of UI design.
Over the next decade, I expect that software companies will hire people trained as anthropologists and ethnographers to work on social interface design. Instead of building usability labs, they’ll go out into the field and write ethnographies. And hopefully, we’ll figure out the new principles of social interface design. It’s going to be fascinating… as fun as user interface design was in the 1980s… so stay tuned.
Discuss this topic on the Joel on Software Social Interface Design Forum.
My Latest Article
Software in the 1980s, when usability was “invented,” was all about computer-human interaction. A lot of software still is. But the Internet brings us a new kind of software: software that’s about human-human interaction. When you’re writing software that mediates between people, after you get the usability right, you have to get the social interface right. And the social interface is more important. The best UI in the world won’t save software with an awkward social interface. It’s Not Just Usability
Someone posted a nice summary of links to information about the quality of workspace provided to software developers. “Perhaps it can be useful to other software developers in a position to influence managers or may someday be in a position to make decisions about workspace design themselves.”
I’m happy to announce that my friend, teacher, and competitor Eric Sink has agreed to host the new Business of Software forum.
Eric has been in the software industry from time immemorial. He built one of the first web browsers, Spyglass and founded the AbiWord project to make an open source word processor. His latest company is SourceGear, a highly successful Inc 500 software company in Champaign, Illinois which makes great source control management software. He writes a column on the business of software for MSDN and has his own weblog where he has been extremely generous with his knowledge of software business. I couldn’t have a better host.
In Usenet, whenever a single newsgroup got too large, it tended to fork. So from comp we got comp.sys.ibm.pc which split into smaller and smaller groups like the unloved comp.sys.ibm.pc.hardware.video, created because people were sick of talking about video drivers on the main group.
I didn’t like forks, because they make discussions less interesting. I mean, it’s bad enough there’s a comp.software.windows.nt.40.microsoft.notepad, does there have to be a comp.software.windows.nt.40.microsoft.notepad.helpfile.index? Seriously now.
But my aversion to forking notwithstanding, there’s just too much traffic in the Joel on Software discussion group. Things wind up scrolling off right away and discussion has to be fast and furious or you lose it. So as an experiment in forking, I’ve set up a few new groups:
- The Business of Software
- The Design of Software
- The Human Side of Software
- Whether Or Not It Is A Good Thing That Rory Slept With Dean
Just kidding about #4 there. The Gilmore Girls is holy. Can’t mess with that.
If any of these groups doesn’t have enough daily traffic, it’ll go the way of the old New Yorkers group and the Delphi group, bless them, in the Joel on Software graveyard.
Joel on Software, The Book
My book is about to go into a second printing. Yay! Hurry up and buy one now or you may have to wait while Apress scurries around making more copies.
Try out the new discussion group, now in beta!
A few brief comments:
- Yes, there are bugs! Many bugs! Please report them using the link provided.
- It looks kind of plain right now, I know. And I haven’t implemented a search feature yet, which is planned.
- It works a lot like the old discussion group.
- There’s a new archive section (“Older Topics”).
- You can provide a URL with your name when you post. I’m hoping that people who have their own web sites will do so, and that this will make posters a little less anonymous. (Note that these URLs will not be followed by search engines like Google, so they cannot be abused to “steal” PageRank.)
Most of the changes are really on the back end, in the adminitrator’s interface and in the abuse- and spam-prevention features. I’m hoping that if all works well, you’ll see a lot less spam. And it’s now much easier for me to make a new discussion group, something which used to take an hour of work, so I may set up several new groups in the future (if you have a good idea for a topic please post it.)