2001/10/23

I’ve been meaning to write more about the creation of CityDesk, but actually our beta is going extremely well, meaning, we’re finding lots of bugs, so I’ve been incredibly busy.

To start the beta off, we released CityDesk to about 100 testers. Most of the bug reports arrived within the first 24 hours. We gathered bugs and feedback from several sources: the CityDesk discussion group, email, a menu item we have called “Send Beta Feedback,” and Michael’s excellent bug-catching feature (if CityDesk crashes, it offers to send a bug report to us, which goes right into FogBUGZ.)

Indeed many of the bug reports we’re getting now are just what we expected: configuration issues. People have funny versions of Internet Explorer. People have their computers set to display everything in large fonts. And something we were doing wrong happened to work OK in Windows 2000, but in NT 4.0, it failed. It would be almost impossible for a small company to find these kinds of bugs without beta testers. And it would have been impossible for us to handle the feedback from more than about 100 people!

Next Monday, I hope, we’ll release another version with almost everything fixed, this time to a larger group — about 200 more people (and of course the first 100 are welcome to download the update). The plan is that we’ll have one last beta two weeks after that, putting us on schedule for releasing on or around December 1st.

Working on CityDesk, Part Three

UNM Humanities Building, where my dad had his officeLike all good hackers, I have been programming since junior high, using my dad’s account on the University of New Mexico IBM 360 mainframe. But the first real computer science course I took was during the first year of college. The course covered C and assembler. It had hundreds of students, most of whom, like me, had been fooling around in Pascal or Basic since they were toddlers.

The course proceeded merrily until one day, the professor introduced pointers.

A rock in Bandelier Park, New Mexico

Dread.

Suddenly, the majority of the students in the class were in deep trouble. For some reason, some people just do not seem to be able to write code with pointers in it. They were born without the part of the brain that does indirection, I guess.

Since then, I’ve mentally divided the world into three groups. The largest group of people can’t program at all. There’s another, smaller group of people who can program, but not with pointers. And there’s a tiny group of people who can program, even with pointers. Those elite few can even understand what it means to write CString*& in C++.

My first job at Microsoft was putting a decent scripting language into Excel. Although I was given free rein to implement whatever language I saw fit, we went with Basic for several reasons. (1) We had a Basic compiler team in house. (2) You didn’t need pointers, which meant that (3) more people are comfortable using Basic than any other language. Indeed Visual Basic is the best-selling language product of all time.

Visual Basic is an extremely productive way to write code, especially GUI code. Want bold text on a dialog box? It’s one click in VB. Now try doing it in MFC. You have to create a subclassed control, it’s a big mess, you have to know all about LOGFONTS and Windows window subclassing and a bunch of other things and you need about three lines of code once you have the magic class.

But many VB programs are spaghetti, either because they’re done as quick and dirty one-offs, or because they’re written by hack programmers without training in object oriented programming, or even structured programming.

What I wondered was, what happens if you take top-notch C++ programmers who dream in pointers, and let them code in VB. What I discovered at Fog Creek was that they become super-efficient coding machines. The code looks pretty good, it’s object-oriented and robust, but you don’t waste time using tools that are at a level lower than you need. I’ve spent years writing code for C++/MFC and years writing code in Visual Basic, and let me tell you, VB is just much, much more productive. Michael and I had a good laugh today when we discovered somebody selling a beta crash-reporting product at $5000 for three months that Michael implemented in CityDesk in two days. (And we actually implemented a good part of ours in C++/ATL). And I also guarantee you that our Visual Basic code in CityDesk looks a lot better than most of the code you find written in macho languages like C++, because we’re good programmers, and we write comments, and our variable names are well-chosen, and we do things the simple way, not the clever way, and so forth.

I’ll go out on a limb here. In my years of experience, I have seen many language and programming fads come and go. But there’s only ONE, that’s right, ONE language feature I’ve ever seen that actually improves your productivity significantly. No, it’s not object oriented programming; no, it’s not intentional programming or assertions or programming by example or CASE or UML or XML or Java. The only thing that improves your programming productivity is using managed code – that is, using a language in which memory management is automatic. Java and .NET languages do this with garbage collection; VB does this with reference counting; I don’t care how you do it, just let me concatenate strings without thinking about where the new bigger string will go and I’ll be happy.

One of the things about Visual Basic is that it doesn’t always give you access to the full repertoire of Windows goodies that you need to make a polished application. But what it does do, better than almost any other programming environment, is let you drop into C++ code (or call C APIs) when you’re desperate or when you need that extra speed. For example, you can always get the HWND of a control and do native stuff to it, which is not the case in Java. As another example, a lot of the non-GUI, time-sensitive inner loops, like the word counter, in CityDesk are actually implemented in C or C++ for speed. This ability gave us the confidence to use Visual Basic even though it can’t do everything and it tends to do string processing slowly. But since we’re all C++ programmers, we have no fear of creating a DLL or OCX to count words, or parse script, or call a Windows API. So about 5% of CityDesk is actually in C or C++, and we’ll probably move a little bit more of the code to C++ to speed up a few more inner loops.

Now, Visual Basic is not the perfect programming language. It’s fairly object oriented but there are little things that you can’t do with it, like have base classes with implementation reuse. It’s limited to Windows. And the worst part about coding in VB is that people think you’re not cool because your code doesn’t have {‘s and }’s. I can live with the shame if it means I’m more productive.

Philosophically, I think that C# has a bright future in the Windows GUI programming world. It’s not as embarrassing as VB, and it uses the type of syntax which C/C++/Java programmers have come to love. VB programmers looking to upgrade can’t upgrade painlessly to VB.NET, because there are so many major differences in the programming environment. Even Microsoft admits that you can’t port from VB to VB.NET, you have to rewrite. And that’s enough of a pain that many VB programmers will use this opportunity to look around at what else is out there. I think many will choose C#, because it’s virtually the same language as VB.NET with slightly different syntax and vastly less stigma attached to it.

What about Java? Yes, I’ve used Java extensively, but unfortunately the language, the code libraries, and especially the GUI libraries are just too primitive for a commercial desktop application. I like the language and I appreciate the benefit of write-once-run-anywhere, but frankly not a lot of desktop software is sold for Sun Solaris and I think that WORA benefits Sun more than it benefits software developers, and I’m not willing to write an app that behaves in an inferior way on 95% of my customer’s computers to benefit the 5% with alternate platforms. Every Java app LOOKS like a Java app, takes forever to launch, and just doesn’t feel completely native. Since CityDesk’s competitive advantage comes from having an excellent GUI, that’s one area where I refuse to skimp.

I am virtually certain that I will now receive a million emails from lovers of tcl/tk, or Delphi, or C++ Builder, or NextStep, or Cocoa, or perl, or Python, or RealBASIC, or some other programming environment which may or may not be suitable for creating a professional Windows GUI. That’s nice. I don’t really want to get into a debate about language features or programming environment features — at some point, you have to stop debating and write code! I’m just trying to explain why we chose to use VB for the GUI part and C++ for time sensitive bits of code. If you do want to have a fun religious war over programming languages, please do so on the discussion group!

Working on CityDesk Part: 1 2 3 4 5

2001/10/17

The worst part about coding in Visual Basic is that people think you’re not cool because your code doesn’t have {‘s and }’s. I can live with the shame if it means I’m more productive.

Working on CityDesk, Part 3

PS If you’ve applied for the CityDesk beta, and haven’t heard back, that’s because we’re waiting to fix a bunch of known bugs before we release to the next group of beta testers. It will make your life much easier and we won’t get the same feedback twice.

Discussion Board

I’ve implemented a few more small, invisible-seeming refinements. The big news is that if you include your email address in a posting, it won’t be easily harvestable by spammers.

2001/10/16

Testing

Does anybody have experience with GUI automated testing tools? I’m going to take advantage of the CityDesk beta to develop an automated test suite (so we don’t keep breaking 2 things for every 1 thing we fix!)

If you have any recommendations, please post them on the discussion group. It needs to support Windows apps and be the sort of thing that doesn’t involve a salesperson visiting your office with a $6000 projector for the demo (in other words, not too expensive.)

Snow

Zillions of emails in my inbox. Big stack of bills to pay. Now I’m snowed under with all the stuff I was deferring until the beta shipped 🙂

2001/10/15

Server Go Boom and Spray Email

Apparently every time our server went boom this morning (4 or 5 times) the stupid mailing list manager software decided it would be a nice idea to resend my notification email. If you got several copies of today’s Joel on Software bulletin, I sincerely apologize and ask your forgiveness.

Discussion Forums

The CityDesk and Joel on Software forums are quite lively, but we’ve also got a relatively quiet FogBUGZ forum which is feeling lonely 🙂

Fog Creek President Michael Pryor figured out a brilliant trick which makes it so that you see new topics, and topics that have followups you haven’t read, in blue. If you’ve read the entire topic, it will be purple. And it’s all done without keeping any state on the server.

Server Go Boom

Dell 2500SCThe home-built server we’ve been using for fogcreek.com keeps crashing whenever the temperature goes above about 80 degrees in the server closet. Grump. I guess it’s time to replace it with a real industrial strength server… I’ve ordered a couple of these.

CityDesk Beta

The CityDesk Beta has started trickling out, and within minutes we’re getting bug reports from the field. Great! That’s why we’re doing a beta. We’ve been banging on this dang thing for months. We fixed 329 bugs. Then Brad W. downloads it, plays for ten seconds, and finds another. And we don’t run so good on NT 4.0. (98, Me, 2000 and XP are all fine).

In Defense of Not-Invented-Here Syndrome

Time for a pop quiz.

Copley Square 1. Code Reuse is:

a) Good
b) Bad

2. Reinventing the Wheel is:

a) Good
b) Bad

3. The Not-Invented-Here Syndrome is:

a) Good
b) Bad

Of course, everybody knows that you should always leverage other people’s work. The correct answers are, of course, 1(a) 2(b) 3(b).

Right?

Not so fast, there!

The Not-Invented-Here Syndrome is considered a classic management pathology, in which a team refuses to use a technology that they didn’t create themselves. People with NIH syndrome are obviously just being petty, refusing to do what’s in the best interest of the overall organization because they can’t find a way to take credit. (Right?) The Boring Business History Section at your local megabookstore is rife with stories about stupid teams that spend millions of dollars and twelve years building something they could have bought at Egghead for $9.99. And everybody who has paid any attention whatsoever to three decades of progress in computer programming knows that Reuse is the Holy Grail of all modern programming systems.

Right. Well, that’s what I thought, too. So when I was the program manager in charge of the first implementation of Visual Basic for Applications, I put together a careful coalition of four, count them, four different teams at Microsoft to get custom dialog boxes in Excel VBA. The idea was complicated and fraught with interdependencies. There was a team called AFX that was working on some kind of dialog editor. Then we would use this brand new code from the OLE group which let you embed one app inside another. And the Visual Basic team would provide the programming language behind it. After a week of negotiation I got the AFX, OLE, and VB teams to agree to this in principle.

I stopped by Andrew Kwatinetz’s office. He was my manager at the time and taught me everything I know. “The Excel development team will never accept it,” he said. “You know their motto? ‘Find the dependencies — and eliminate them.’ They’ll never go for something with so many dependencies.”

In-ter-est-ing. I hadn’t known that. I guess that explained why Excel had its own C compiler.

By now I’m sure many of my readers are rolling on the floor laughing. “Isn’t Microsoft stupid,” you’re thinking, “they refused to use other people’s code and they even had their own compiler just for one product.”

Not so fast, big boy! The Excel team’s ruggedly independent mentality also meant that they always shipped on time, their code was of uniformly high quality, and they had a compiler which, back in the 1980s, generated pcode and could therefore run unmodified on Macintosh’s 68000 chip as well as Intel PCs. The pcode also made the executable file about half the size that Intel binaries would have been, which loaded faster from floppy disks and required less RAM.

“Find the dependencies — and eliminate them.” When you’re working on a really, really good team with great programmers, everybody else’s code, frankly, is bug-infested garbage, and nobody else knows how to ship on time. When you’re a cordon bleu chef and you need fresh lavender, you grow it yourself instead of buying it in the farmers’ market, because sometimes they don’t have fresh lavender or they have old lavender which they pass off as fresh.

Indeed during the recent dotcom mania a bunch of quack business writers suggested that the company of the future would be totally virtual — just a trendy couple sipping Chardonnay in their living room outsourcing everything. What these hyperventilating “visionaries” overlooked is that the market pays for value added. Two yuppies in a living room buying an e-commerce engine from company A and selling merchandise made by company B and warehoused and shipped by company C, with customer service from company D, isn’t honestly adding much value. In fact, if you’ve ever had to outsource a critical business function, you realize that outsourcing is hell. Without direct control over customer service, you’re going to get nightmarishly bad customer service — the kind people write about in their weblogs when they tried to get someone, anyone, from some phone company to do even the most basic thing. If you outsource fulfillment, and your fulfillment partner has a different idea about what constitutes prompt delivery, your customers are not going to be happy, and there’s nothing you can do about it, because it took 3 months to find a fulfillment partner in the first place, and in fact, you won’t even know that your customers are unhappy, because they can’t talk to you, because you’ve set up an outsourced customer service center with the explicit aim of not listening to your own customers. That e-commerce engine you bought? There’s no way it’s going to be as flexible as what Amazon does with obidos, which they wrote themselves. (And if it is, then Amazon has no advantage over their competitors who bought the same thing). And no off-the-shelf web server is going to be as blazingly fast as what Google does with their hand-coded, hand-optimized server.

This principle, unfortunately, seems to be directly in conflict with the ideal of “code reuse good — reinventing wheel bad.”

The best advice I can offer:

If it’s a core business function — do it yourself, no matter what.

Pick your core business competencies and goals, and do those in house. If you’re a software company, writing excellent code is how you’re going to succeed. Go ahead and outsource the company cafeteria and the CD-ROM duplication. If you’re a pharmaceutical company, write software for drug research, but don’t write your own accounting package. If you’re a web accounting service, write your own accounting package, but don’t try to create your own magazine ads. If you have customers, never outsource customer service.

If you’re developing a computer game where the plot is your competitive advantage, it’s OK to use a third party 3D library. But if cool 3D effects are going to be your distinguishing feature, you had better roll your own.

The only exception to this rule, I suspect, is if your own people are more incompetent than everyone else, so whenever you try to do anything in house, it’s botched up. Yes, there are plenty of places like this. If you’re in one of them, I can’t help you.

Discuss

2001/10/14

Not-Invented-Here

“When you’re working on a really, really good team with great programmers, everybody else’s code, frankly, is bug-infested garbage, and nobody else knows how to ship on time.”

In Defense of Not-Invented-Here Syndrome

Discussion Board

The cool new discussion board is done!

Discuss

The URL for the new discussion server is http://discuss.fogcreek.com/joelonsoftware .

Underline Edit Boxes

I like to make fun of designers who reinvent common widgets simply because they can. The big advantage of using a standard HTML edit box is consistency — more people will recognize it and know how to use it.

But when you have a long form with many fields, all those lines start to look ugly, so I’ve gotten into the bad habit of using style sheets so that my edit boxes are just underlines, not boxes. This is “consistent” more with paper forms, and someone even told me he thought he was going to have to print out the page when he saw the beta application form using underlines. 

It probably does reduce learnability, marginally, to use underlines for edit boxes instead of boxes. But then again, they’re really cool looking 🙂 and I can’t bear to switch them back to boxes. What’s your opinion?

Working on CityDesk, Part Two

I’ve been trying to get a discussion board running for three days now, without much success. Installing server software is just much harder than installing Windows software. There are always multiple complicated steps that involve permissions, accounts, database servers, dependencies (do you have the absolute latest version of perl?), superuser, and web servers. If you’re using free software, nobody wants to volunteer to make a decent setup program (or even documentation, half the time), so you’re generally on your own there. And when you’re using commercial software, the vendor would usually like to sell you a three-week integration consulting project so they can make another $50,000 copying some files and editing some configuration files. (We have a double-click SETUP program for FogBUGZ that works pretty well, but there are still people with funny configurations where it doesn’t set all the permissions correctly.)

Untitled, Steven Harvey, Oil on Canvas

I’m also a bit particular about what discussion software we use. The purpose of the board will be a place for CityDesk beta testers (and later, users) to ask questions, provide feedback, and share ideas. In designing a UI for anything, the very first question you always want to ask is: who is the user? Specifically: are most users casual, occasional users, or are these users who will be spending all of their time using your program? For casual users, learnability and simplicity are more important than usability and power. In that sentence, by “learnability,” I mean, the ability for novices to figure out how to get tasks done rapidly. By “usability,” I mean only the ability to do tasks in a convenient and ergonomic way without making mistakes and without needing to do repetitive tasks. A data entry system that minimizes keystrokes by prefilling things and automatically jumping from field to field is more usable for experienced users, but it’s harder to learn because it behaves unexpectedly to a novice.

Most people using the Fog Creek bulletin board will be going there to ask a question or raise an issue. Certainly in the early days, learnability and simplicity are our priority. When I looked a bunch of discussion boards, I found that most of them have their heritage in the BBS world, where the same people log on to chat for 4 hours every night. Those people love features like a geegaw that lets you put a graphical smiley in your posting, and the ability to upload snapshots of their ugly mugs to appear next to their postings, and the ability to click a button and never see postings by the blithering idiot who wrote this one again. All of these features are neat for power users but they just clutter up the interface for novices.

Juno's Read and Write tabs(Longtime Juno email users may have noticed that as time went on, the big Read and Write tabs got smaller and smaller. In the early days, where almost every Juno user was a new Juno user, it was nice to have big giant buttons for reading and writing email, the most common task. As time went on, a larger proportion of our users were experienced users, who know how to “read” and “write” and would rather have more screen real estate available for other features. It’s not uncommon for a program to start out simple and evolve to be more complicated, and you can do this without hurting “average” usability because your users are getting more experienced on average.)

I spent Wednesday and Friday playing around, installing various buggy BBS systems, some of which required a Linux server, others Windows 2000, playing with DNS to move discuss.fogcreek.com hither and yon, figuring out why DNS caches weren’t flushing, installing and reinstalling database servers, and generally getting frustrated. One of the most popular packages, Discus, actually hardcoded its own URL in so many places that it needed to be reinstalled from scratch just to change the URL. (In fact, it wasn’t enough to reinstall it… you had to redownload it. The download package already had your personal URL hardcoded throughout.) And it had a perfectly terrible UI in which there was a little treeview showing folders (so far so good)… but each folder was actually a command, not a container. A broken metaphor, worse than no metaphor at all. That had to go. Then I tried IdealBB, a decidely beta package. I wouldn’t ordinarily run software that is advertised as beta, but this thing seemed to have been through many releases and it was alleged to be very close to shipping. Michael and I actually had to roll up our sleeves and debug the thing ourselves to get it sort of running, but then there were too many ASP errors and it had a tendency to crash the server. (Too bad, because it is one of the finest looking discussion boards, visually). Finally I flirted with Manila, since we have it running anyway for our weblogs, and we’ve already written a little daemon which watches Manila and restarts it when it crashes (about once every two days). But (as far as we could tell) Manila requires membership to post a message, and in my experience that is enough to turn away 90% of the casual visitors who might otherwise use the discussion board. It would be great for small elite communities of people who all post all the time, but I don’t want anything to get in the way of a beta tester casually reporting a bug.

The system I like best, believe it or not, is Lusenet, by Philip Greenspun, because it’s just super simple. That’s what I’ve been using for the Joel on Software discussion group. There are a couple of reasons we couldn’t use that. One, it is really not ready for prime time. There are actually things you can do in Lusenet that still show you the Oracle statement they just executed, as if Philip left in some debug printfs. Second, it’s not hosted on our own servers and we run the risk of it going away, taking our valuable data with it. Third, to host it on our own servers requires AOLServer and Oracle, and we don’t have the former, and can’t afford the latter.

When I got home, grumpy from all the time I had wasted, I realized that the software I want consists of exactly one table, and I could write the thing myself in less time than in took me to install some of these packages. Which I did. Two hours of work (ASP, Microsoft Access, and VBScript) and I had banged out a system that did pretty much everything I wanted (which is not much!) Check it out at http://discuss.fogcreek.com/joelonsoftware. (If you have trouble reaching it that’s because of all my DNS messing around. You’ll have to wait a couple of days for caches to flush.)

There’s a lesson in here somewhere, but I’m already well past 1000 words. In the past I wouldn’t have cared about word counts because I didn’t know what they were, but now I’m using CityDesk which keeps a running word count in the corner of the screen. So we’ll talk about the lesson tomorrow!

Working on CityDesk Part: 1 2 3 4 5