Posts by Joel Spolsky

In 2000 I co-founded Fog Creek Software, where we created lots of cool things like the FogBugz bug tracker, Trello, and Glitch. I also worked with Jeff Atwood to create Stack Overflow and served as CEO of Stack Overflow from 2010-2019. Today I serve as the chairman of the board for Stack Overflow, Glitch, and HASH.

2001/10/16

Testing

Does anybody have experience with GUI automated testing tools? I’m going to take advantage of the CityDesk beta to develop an automated test suite (so we don’t keep breaking 2 things for every 1 thing we fix!)

If you have any recommendations, please post them on the discussion group. It needs to support Windows apps and be the sort of thing that doesn’t involve a salesperson visiting your office with a $6000 projector for the demo (in other words, not too expensive.)

Snow

Zillions of emails in my inbox. Big stack of bills to pay. Now I’m snowed under with all the stuff I was deferring until the beta shipped 🙂

2001/10/15

Server Go Boom and Spray Email

Apparently every time our server went boom this morning (4 or 5 times) the stupid mailing list manager software decided it would be a nice idea to resend my notification email. If you got several copies of today’s Joel on Software bulletin, I sincerely apologize and ask your forgiveness.

Discussion Forums

The CityDesk and Joel on Software forums are quite lively, but we’ve also got a relatively quiet FogBUGZ forum which is feeling lonely 🙂

Fog Creek President Michael Pryor figured out a brilliant trick which makes it so that you see new topics, and topics that have followups you haven’t read, in blue. If you’ve read the entire topic, it will be purple. And it’s all done without keeping any state on the server.

Server Go Boom

Dell 2500SCThe home-built server we’ve been using for fogcreek.com keeps crashing whenever the temperature goes above about 80 degrees in the server closet. Grump. I guess it’s time to replace it with a real industrial strength server… I’ve ordered a couple of these.

CityDesk Beta

The CityDesk Beta has started trickling out, and within minutes we’re getting bug reports from the field. Great! That’s why we’re doing a beta. We’ve been banging on this dang thing for months. We fixed 329 bugs. Then Brad W. downloads it, plays for ten seconds, and finds another. And we don’t run so good on NT 4.0. (98, Me, 2000 and XP are all fine).

In Defense of Not-Invented-Here Syndrome

Time for a pop quiz.

Copley Square 1. Code Reuse is:

a) Good
b) Bad

2. Reinventing the Wheel is:

a) Good
b) Bad

3. The Not-Invented-Here Syndrome is:

a) Good
b) Bad

Of course, everybody knows that you should always leverage other people’s work. The correct answers are, of course, 1(a) 2(b) 3(b).

Right?

Not so fast, there!

The Not-Invented-Here Syndrome is considered a classic management pathology, in which a team refuses to use a technology that they didn’t create themselves. People with NIH syndrome are obviously just being petty, refusing to do what’s in the best interest of the overall organization because they can’t find a way to take credit. (Right?) The Boring Business History Section at your local megabookstore is rife with stories about stupid teams that spend millions of dollars and twelve years building something they could have bought at Egghead for $9.99. And everybody who has paid any attention whatsoever to three decades of progress in computer programming knows that Reuse is the Holy Grail of all modern programming systems.

Right. Well, that’s what I thought, too. So when I was the program manager in charge of the first implementation of Visual Basic for Applications, I put together a careful coalition of four, count them, four different teams at Microsoft to get custom dialog boxes in Excel VBA. The idea was complicated and fraught with interdependencies. There was a team called AFX that was working on some kind of dialog editor. Then we would use this brand new code from the OLE group which let you embed one app inside another. And the Visual Basic team would provide the programming language behind it. After a week of negotiation I got the AFX, OLE, and VB teams to agree to this in principle.

I stopped by Andrew Kwatinetz’s office. He was my manager at the time and taught me everything I know. “The Excel development team will never accept it,” he said. “You know their motto? ‘Find the dependencies — and eliminate them.’ They’ll never go for something with so many dependencies.”

In-ter-est-ing. I hadn’t known that. I guess that explained why Excel had its own C compiler.

By now I’m sure many of my readers are rolling on the floor laughing. “Isn’t Microsoft stupid,” you’re thinking, “they refused to use other people’s code and they even had their own compiler just for one product.”

Not so fast, big boy! The Excel team’s ruggedly independent mentality also meant that they always shipped on time, their code was of uniformly high quality, and they had a compiler which, back in the 1980s, generated pcode and could therefore run unmodified on Macintosh’s 68000 chip as well as Intel PCs. The pcode also made the executable file about half the size that Intel binaries would have been, which loaded faster from floppy disks and required less RAM.

“Find the dependencies — and eliminate them.” When you’re working on a really, really good team with great programmers, everybody else’s code, frankly, is bug-infested garbage, and nobody else knows how to ship on time. When you’re a cordon bleu chef and you need fresh lavender, you grow it yourself instead of buying it in the farmers’ market, because sometimes they don’t have fresh lavender or they have old lavender which they pass off as fresh.

Indeed during the recent dotcom mania a bunch of quack business writers suggested that the company of the future would be totally virtual — just a trendy couple sipping Chardonnay in their living room outsourcing everything. What these hyperventilating “visionaries” overlooked is that the market pays for value added. Two yuppies in a living room buying an e-commerce engine from company A and selling merchandise made by company B and warehoused and shipped by company C, with customer service from company D, isn’t honestly adding much value. In fact, if you’ve ever had to outsource a critical business function, you realize that outsourcing is hell. Without direct control over customer service, you’re going to get nightmarishly bad customer service — the kind people write about in their weblogs when they tried to get someone, anyone, from some phone company to do even the most basic thing. If you outsource fulfillment, and your fulfillment partner has a different idea about what constitutes prompt delivery, your customers are not going to be happy, and there’s nothing you can do about it, because it took 3 months to find a fulfillment partner in the first place, and in fact, you won’t even know that your customers are unhappy, because they can’t talk to you, because you’ve set up an outsourced customer service center with the explicit aim of not listening to your own customers. That e-commerce engine you bought? There’s no way it’s going to be as flexible as what Amazon does with obidos, which they wrote themselves. (And if it is, then Amazon has no advantage over their competitors who bought the same thing). And no off-the-shelf web server is going to be as blazingly fast as what Google does with their hand-coded, hand-optimized server.

This principle, unfortunately, seems to be directly in conflict with the ideal of “code reuse good — reinventing wheel bad.”

The best advice I can offer:

If it’s a core business function — do it yourself, no matter what.

Pick your core business competencies and goals, and do those in house. If you’re a software company, writing excellent code is how you’re going to succeed. Go ahead and outsource the company cafeteria and the CD-ROM duplication. If you’re a pharmaceutical company, write software for drug research, but don’t write your own accounting package. If you’re a web accounting service, write your own accounting package, but don’t try to create your own magazine ads. If you have customers, never outsource customer service.

If you’re developing a computer game where the plot is your competitive advantage, it’s OK to use a third party 3D library. But if cool 3D effects are going to be your distinguishing feature, you had better roll your own.

The only exception to this rule, I suspect, is if your own people are more incompetent than everyone else, so whenever you try to do anything in house, it’s botched up. Yes, there are plenty of places like this. If you’re in one of them, I can’t help you.

Discuss

2001/10/14

Not-Invented-Here

“When you’re working on a really, really good team with great programmers, everybody else’s code, frankly, is bug-infested garbage, and nobody else knows how to ship on time.”

In Defense of Not-Invented-Here Syndrome

Discussion Board

The cool new discussion board is done!

Discuss

The URL for the new discussion server is http://discuss.fogcreek.com/joelonsoftware .

Underline Edit Boxes

I like to make fun of designers who reinvent common widgets simply because they can. The big advantage of using a standard HTML edit box is consistency — more people will recognize it and know how to use it.

But when you have a long form with many fields, all those lines start to look ugly, so I’ve gotten into the bad habit of using style sheets so that my edit boxes are just underlines, not boxes. This is “consistent” more with paper forms, and someone even told me he thought he was going to have to print out the page when he saw the beta application form using underlines. 

It probably does reduce learnability, marginally, to use underlines for edit boxes instead of boxes. But then again, they’re really cool looking 🙂 and I can’t bear to switch them back to boxes. What’s your opinion?

Working on CityDesk, Part Two

I’ve been trying to get a discussion board running for three days now, without much success. Installing server software is just much harder than installing Windows software. There are always multiple complicated steps that involve permissions, accounts, database servers, dependencies (do you have the absolute latest version of perl?), superuser, and web servers. If you’re using free software, nobody wants to volunteer to make a decent setup program (or even documentation, half the time), so you’re generally on your own there. And when you’re using commercial software, the vendor would usually like to sell you a three-week integration consulting project so they can make another $50,000 copying some files and editing some configuration files. (We have a double-click SETUP program for FogBUGZ that works pretty well, but there are still people with funny configurations where it doesn’t set all the permissions correctly.)

Untitled, Steven Harvey, Oil on Canvas

I’m also a bit particular about what discussion software we use. The purpose of the board will be a place for CityDesk beta testers (and later, users) to ask questions, provide feedback, and share ideas. In designing a UI for anything, the very first question you always want to ask is: who is the user? Specifically: are most users casual, occasional users, or are these users who will be spending all of their time using your program? For casual users, learnability and simplicity are more important than usability and power. In that sentence, by “learnability,” I mean, the ability for novices to figure out how to get tasks done rapidly. By “usability,” I mean only the ability to do tasks in a convenient and ergonomic way without making mistakes and without needing to do repetitive tasks. A data entry system that minimizes keystrokes by prefilling things and automatically jumping from field to field is more usable for experienced users, but it’s harder to learn because it behaves unexpectedly to a novice.

Most people using the Fog Creek bulletin board will be going there to ask a question or raise an issue. Certainly in the early days, learnability and simplicity are our priority. When I looked a bunch of discussion boards, I found that most of them have their heritage in the BBS world, where the same people log on to chat for 4 hours every night. Those people love features like a geegaw that lets you put a graphical smiley in your posting, and the ability to upload snapshots of their ugly mugs to appear next to their postings, and the ability to click a button and never see postings by the blithering idiot who wrote this one again. All of these features are neat for power users but they just clutter up the interface for novices.

Juno's Read and Write tabs(Longtime Juno email users may have noticed that as time went on, the big Read and Write tabs got smaller and smaller. In the early days, where almost every Juno user was a new Juno user, it was nice to have big giant buttons for reading and writing email, the most common task. As time went on, a larger proportion of our users were experienced users, who know how to “read” and “write” and would rather have more screen real estate available for other features. It’s not uncommon for a program to start out simple and evolve to be more complicated, and you can do this without hurting “average” usability because your users are getting more experienced on average.)

I spent Wednesday and Friday playing around, installing various buggy BBS systems, some of which required a Linux server, others Windows 2000, playing with DNS to move discuss.fogcreek.com hither and yon, figuring out why DNS caches weren’t flushing, installing and reinstalling database servers, and generally getting frustrated. One of the most popular packages, Discus, actually hardcoded its own URL in so many places that it needed to be reinstalled from scratch just to change the URL. (In fact, it wasn’t enough to reinstall it… you had to redownload it. The download package already had your personal URL hardcoded throughout.) And it had a perfectly terrible UI in which there was a little treeview showing folders (so far so good)… but each folder was actually a command, not a container. A broken metaphor, worse than no metaphor at all. That had to go. Then I tried IdealBB, a decidely beta package. I wouldn’t ordinarily run software that is advertised as beta, but this thing seemed to have been through many releases and it was alleged to be very close to shipping. Michael and I actually had to roll up our sleeves and debug the thing ourselves to get it sort of running, but then there were too many ASP errors and it had a tendency to crash the server. (Too bad, because it is one of the finest looking discussion boards, visually). Finally I flirted with Manila, since we have it running anyway for our weblogs, and we’ve already written a little daemon which watches Manila and restarts it when it crashes (about once every two days). But (as far as we could tell) Manila requires membership to post a message, and in my experience that is enough to turn away 90% of the casual visitors who might otherwise use the discussion board. It would be great for small elite communities of people who all post all the time, but I don’t want anything to get in the way of a beta tester casually reporting a bug.

The system I like best, believe it or not, is Lusenet, by Philip Greenspun, because it’s just super simple. That’s what I’ve been using for the Joel on Software discussion group. There are a couple of reasons we couldn’t use that. One, it is really not ready for prime time. There are actually things you can do in Lusenet that still show you the Oracle statement they just executed, as if Philip left in some debug printfs. Second, it’s not hosted on our own servers and we run the risk of it going away, taking our valuable data with it. Third, to host it on our own servers requires AOLServer and Oracle, and we don’t have the former, and can’t afford the latter.

When I got home, grumpy from all the time I had wasted, I realized that the software I want consists of exactly one table, and I could write the thing myself in less time than in took me to install some of these packages. Which I did. Two hours of work (ASP, Microsoft Access, and VBScript) and I had banged out a system that did pretty much everything I wanted (which is not much!) Check it out at http://discuss.fogcreek.com/joelonsoftware. (If you have trouble reaching it that’s because of all my DNS messing around. You’ll have to wait a couple of days for caches to flush.)

There’s a lesson in here somewhere, but I’m already well past 1000 words. In the past I wouldn’t have cared about word counts because I didn’t know what they were, but now I’m using CityDesk which keeps a running word count in the corner of the screen. So we’ll talk about the lesson tomorrow!

Working on CityDesk Part: 1 2 3 4 5

What Does CityDesk Do?

CityDesk is a Windows program that makes it easy to manage a web site which changes often.

With CityDesk, people who don’t know anything about HTML or web servers can easily add, edit, and remove articles from a web site, using a program that is as easy as a word processor. CityDesk will automatically apply standard formatting, according to templates you provide, to those articles. Then it copies them to your web server automatically. Each article can include pictures, sounds, and other media elements.

Here are just a few examples of the kinds of things you can build and maintain with CityDesk:

  • A daily newspaper, a monthly newsletter, or a web-based magazine
  • A personal journal or weblog
  • A company’s list of job openings
  • A real-estate agent’s list of currently available properties
  • A company worldwide knowledge base

The real power in CityDesk comes from the fact that you only design the formatting of your site once. After that, it’s easy to create new articles which use the same design. Because CityDesk keeps the text of the articles separate from their design, you can change the design in one place, and every article on your site changes accordingly.

Once you set up your site, updating it is just as easy as using a simple Windows-based word processor. The built-in word processor is WYSIWYG (“What You See Is What You Get”) and includes a spell checker, word counter, find and replace, and formatting commands.

CityDesk is built around a powerful, robust database engine. This means that different people can update the site simultaneously without any risk of conflict or corruption. You can even have a whole newsroom banging away at the same time; even a virtual newsroom, with contributors all over the Internet!

When you need to manage a site with several different versions, CityDesk is invaluable. Suppose you want to publish a newsletter every week in two languages. CityDesk provides a nice interface for translators where they can see a list of untranslated articles and translate them on the spot in a split-screen environment. If you try to publish the site before everything has been translated, you’ll see a warning.

Or suppose you produce a newsletter that has different regional editions. Most of the articles are the same, but you don’t want to run your poem “Every Civilized Person Loves the New York Yankees” in the Boston edition. CityDesk keeps track of that and publishes both editions completely automatically.

CityDesk can also keep track of articles that need to be held until a certain date. Just write the article and set the date range, and CityDesk will never publish it before or after those dates. This is a good way to keep fresh content on your web site when you’re on vacation.

Many web sites appear in multiple formats. For example, you might have:

  • the normal web version
  • a “printer-friendly” web version without ads or navigation elements
  • a version for Palm Pilots without pictures
  • a large-font version for the web site for people who like larger fonts
  • a printed version that you hand out on street corners to passers-by (who use it to line bird-cages, those scurvy knaves)
  • and so on and so forth.

Setting up template families for each of these versions is a one-time operation. After that, you only have to type in the article once, and it will automatically be published in each and every format without any additional work. Ta da!

We think you’ll find that CityDesk is an extremely powerful tool, but one that’s quite easy to understand. There’s a tutorial which will take you on a whirlwind tour in about ten minutes. After that, you’ll only need to consult the documentation occasionally.

CityDesk will be in beta from Oct 15 – Nov 30, 2001, after which it will be shipping. Click here if you would like to apply to be a beta tester.

Working on CityDesk, Part One

I’ve been rather quiet lately on this weblog — mainly because we’ve been working so hard at Fog Creek getting ready for the beta of CityDesk, our flagship product. But I’d like to spend some time now talking about the design and development of CityDesk, because it’s a great case study for the kind of software development practices that I’ve been advocating here for more than a year. Over the next few columns, I’ll focus on “The Story of CityDesk,” with a look at some of the behind-the-scenes, day-to-day stories of a real software development project.

We’re launching the beta on October 15th — just three days away! This is officially On Time. We made a schedule for shipping CityDesk way back in June. We figured out estimates for all the remaining tasks and bug fixes, added them up, and got October 15th.

Every couple of weeks, we checked through our task list, revised estimates, and so forth. We’ve found a lot of bugs over that time and added a lot of small features, but we’ve also killed a lot of features that we just won’t have time for. And lo and behold, now we are almost completely done. Michael's cool crash handler actually enters diagnostics right into FogBUGZ. Most of what we’re doing these days is getting set up to run the beta. Michael added a dialog box to CityDesk for sending feedback. He also wrote some very spiffy code that catches any unhandled exception on any copy of CityDesk running anywhere in the world, prevents the app from crashing, and pushes the exception info up to our FogBUGZ bug tracking database here in New York City. I’ve been fixing bugs, writing the help file, and writing some pages for our corporate web site that will explain what CityDesk is and why you would buy it.

A common misconception, I assume popularized by Hollywood, is that as you get closer to shipping software, activity becomes frenetic as everybody scrambles to finish all the things that need to be done in time for the deadline. In the typical crappy movie, there’s a mad rush of typing in a room full of cool alterna-dressed programmers with found-object earrings and jeans jackets. Somebody stands up and shouts to the room in general “I need the Jiff subroutine! Somebody give me the Jiff subroutine!” A good looking young woman in Vivienne Tam urbanwear throws a floppy disk at him. “Thanks!”

Yeah, most programmers are as cute as Ryan Phillipe. That's why I'm in this field.As the second hand swoops towards the :00, the whole team waits breathlessly around Ryan Phillipe’s computer and watches the “copy” progress indicator as the final bits are put onto a floppy disk with less than a second to spare before the VC cuts off funding.

I suppose some software shops have last-minute coding frenzies like this. If so, their software is probably marked by incredibly poor quality. No code works the way you expected it to work the first time, and if you’re writing code up until the last minute, it’s going to suck. The first Netscape open source release, documented in the excellent movie Code Rush , demonstrates this.

On good teams, the days before shipping just get quieter and quieter as programmers literally run out of things to do one at a time. (Yesterday I took the day off to explore New York City with my wee niece and nephews.)

Times Square

The number of new bugs being found has decreased to a point at which we feel confident about releasing the beta. It is crucial to get to zero known bugs (what Netscape famously called “Zarro Boogs”) before releasing a beta. If you don’t, you’ll waste a lot of time during the beta reading 200 emails about a bug that you already knew about. And you’ve just used up time and goodwill of those 200 beta testers, so they may not bother telling you about the next bug they find, something you didn’t know about. Or the bug may stop them from trying other parts of the program that needs some pounding. This seems self-evident, but almost every time I’ve been on a real product, everybody starts to think that releasing the beta on time is more important than releasing a Zero Known Bugs beta. (After all, it’s ok to have bugs in the beta, they say. And I agree: it is ok to have bugs in the beta, just not known bugs.)

I’ll keep posting The Story of CityDesk over the next few days, keep an eye out for frequent updates.

Working on CityDesk Part: 1 2 3 4 5

2001/09/24

News.com reports:

“Gartner remains concerned that viruses and worms will continue to attack IIS until Microsoft has released a completely rewritten release of [IIS] that is thoroughly and publicly tested….Gartner believes that this rewriting will probably not occur before the end of 2002.”

Gartner seems to suffer the common but moronic fallacy that new or “completely rewritten” code is somehow less buggy than old code. IIS has been publically tested, for about six years now, on millions of web servers and with thousands of hackers trying to find bugs. Completely rewriting it would just introduce another set of bugs that would take another few years to find. Chances are that nobody on the current IIS team even remembers the bugs they fixed five years ago, even if they were on the team that long ago (unlikely), like the $DATA$ one and adding an extra period to the end of an ASP URL.

Completely rewriting code is a big-time mistake common of immature developers with no real software experience. I would say that “Gartner should know better” but I don’t have very high expectations of them.