Working on CityDesk, Part Four

Boy, what a terrible weekend. I’ve come down with a cold. Fever. Runny nose. General malaise. And WININET.DLL.

WININET.DLL, you see, is a software file provided by Microsoft. It comes with Internet Explorer and with all versions of Windows since about 1996, and it was the bane of my weekend. And it illustrates a fascinating fact about how much software developers’ day-to-day lives have changed in the last decade.

Here’s what happened. Late on Friday afternoon I was setting up some new web servers designed to handle the increased load we’re expecting when we ship CityDesk. One of the first things I did was try to use CityDesk to copy files to the new servers using the FTP protocol. We had a firewall set up that prevented FTP from going through. CityDesk froze up.

“No problem,” I thought, “I’ll use passive-mode FTP, which can get through that firewall.”

That’s when I noticed CityDesk doesn’t support passive-mode FTP.

“OK, how hard can that be to implement? It’s probably available as an option on the file transfer library we’re using.” One checkbox and I’m done.

But … where’s the checkbox?

No mention of it in the documentation.

Searching the object file itself didn’t turn up anything likely.

I checked Google Groups, formerly known as DejaNews. It’s a complete archive of every UseNet discussion. This is where programmers ask each other questions about the most arcane topics. As I told Babak once: it’s a big world out there. You’re never the first person to have this problem.

After about five minutes searching, the conclusion was inescapable. Our file transfer library, which Microsoft gives away for free with the Visual Basic compiler, can’t do passive-mode FTP.

OK, back to the drawing board. What are my choices? I wrote up a list:

  1. Do without Passive FTP. This would make CityDesk useless to a large number of people, something I wasn’t willing to do.
  2. Purchase a commercial FTP library. Honestly, I’ve always had bad luck with commercial libraries, having discovered one too many times that their code quality is rarely up to the meticulous standards we set for Fog Creek. When I looked around the discussion groups where developers were discussing other FTP libraries, they always seemed to have scary bugs that I couldn’t live with.
  3. Use Microsoft’s other file transfer library, the infamous WININET.DLL. This is actually what Microsoft Internet Explorer uses to transfer files, which, despite its dismal reputation, is used so widely that I thought it had to be reasonably bug free (by now, at least). Anyway, lots of programmers use WININET.DLL and if I run into trouble I’m sure to find ample discussion of the problems on DejaNews, er, Google Groups.

I thought that #3 seemed painless enough. In fact I was already using WININET.DLL somewhere else in our code to import web pages, using the HTTP protocol.

A search of Microsoft’s online knowledge base revealed that you can’t really do FTP with WININET.DLL from Visual Basic code; it does some complicated stuff with threads that mean that you have to call it from C or C++. I thought the easiest thing to do would be to create my own custom FTP control written in C++, which talked to WININET. I chose to use Microsoft’s ATL library to create the custom control, because it makes the smallest files. ATL is the most complicated programming environment in the world, requiring a brain the size of Colorado and 10 years of solid experience to understand what’s going on. I have studied ATL in depth three, maybe four times in my career and I can never remember all the bizarre template crap that’s going on in there. Nobody can.

Yes, Virginia, it is possible to create a software development environment which is so difficult to use that no human being can do it. ATL and COM+ are my two favorite examples (the latter is so complicated that only one man on Earth, Don Box, actually understands everything that’s going on). C++ itself comes pretty darn close. But most programmers are too macho to admit this.

Luckily, Microsoft provides some wizards with ATL which write all the hard code for you. If you want to do something unusual, you’re on your own, so my motto was Nothing Unusual. No sudden moves. Just add some simple methods and events and get the hell outta there, hopefully with my brains non-exploded.

At one point, after I had written almost half the code, I discovered that one of the checkboxes that I had checked on the wizard which wrote the inscrutable ATL code for me was wrong. But once the code is written, it’s written. For the life of me, I couldn’t figure out what the checkbox had done and where to change it in the code. I searched MSDN (a gigabyte of documentation on programming Windows which I keep on my hard drive), the online knowledge base, and finally the entire Internet using Google, and didn’t find an easy way to change it. So I created two entirely new projects using the wizard, checking the box in one case and not checking it in the other, and then ran WinDiff, which compares two entire directories listing all the differences, to find what the checkbox really changed so I could change it in my code. (Somewhere, I had to change a hardcoded number to 131473, because I wanted my controls to be visible at runtime. A classic example of why COM programming is not for humans.)

The Microsoft documentation for WinInet was pretty decent, as these things go, but not decent enough. In the page documenting FtpOpenFile, you find both of these mutually contradictory quotes:

  1. “No file handle is returned”
  2. “Return value: Returns a handle if successful”

Well, which is it? Empirically, it wasn’t returning a file handle.

The next thing I discovered is that if there is a packet filter (a simple form of firewall) somewhere between you and the server, the code hangs trying to copy files. That’s normal; it’s the way of the Internet; there’s nothing you can do about it. After a minute, WinInet will realize that packets are not getting through and will time out. But the user is likely to get impatient long before a minute is up and hit the Cancel button.

When you hit the Cancel button, my code tells WinInet to give up and close down the connection. But as I discovered, if you do this in one of these packet-filter situations, WinInet will simply crash, bringing your program down with it. It’s clearly a bug in Microsoft’s code. An exhaustive search of all my Internet sources found a couple of people reporting the same crashing behavior, but nobody had a workaround.

How can that be? I thought. Since Internet Explorer uses the exact same code, wouldn’t Internet Explorer crash in the same situation?

I tried it. What I discovered is that Internet Explorer doesn’t crash in this situation — it shows an hourglass and freezes for a couple of minutes, waiting for the time-out it knows it will get. This proves that Microsoft’s programmers knew about this bug in WinInet and worked around it, instead of just fixing the code in the first place. Stupid stupid stupid. For the umpteenth time, I found myself dependent on a code library which had a crashing bug that was unacceptable in code I shipped. What are you supposed to do if you’re the chef at Les Halles and your fishmonger is giving you smelly fish?

Another two hours of investigation and experimentation. Finally I decided that in this case, when the user hits the Cancel button, instead of freezing like Internet Explorer, I will simply hide the file transfer so it looks like the operation has been cancelled. In the background, invisible to the user, I’ll wait around for the time-out to happen.

So, as I said, developers’ lives have changed. All weekend I couldn’t sleep. Tossing and turning in sweat-drenched sheets, I had feverish ATL nightmares. Sunday morning I got up at 3 am and coded for 4 hours just to avoid the bad dreams about Structured Exception Handling.

Ten years ago, to write code, you needed to know a programming language, and you needed to know a library of maybe 50 functions that you used regularly. And those functions worked, every time, although some of them (gets) could not be used without creating security bugs.

Today, you need to know how to work with libraries of thousands of functions, representing buggy code written by other people. You can’t possibly learn them all, and the documentation is never good enough to write solid code, so you learn to use online resources like Google, DejaNews, MSDN. (I became much more productive after a coworker at Google showed me that you’re better off using Google to search Microsoft’s knowledge base rather than the pathetic search engine Microsoft supplies). In this new world, you’re better off using common languages like Visual Basic and common libraries like WinInet, because so many other people are using them it’s easier to find bug fixes and sample code on the Web. Last week, Michael added a feature to CityDesk to check the installed version of Internet Explorer. It’s not hard code to write, but why bother? It only took him a few seconds to find the code, in VB, on the Web and cut and paste it.

We used to write algorithms. Now we call APIs.

Nowadays a good programmer spends a lot of time doing defensive coding, working around other people’s bugs. It’s not uncommon to set up an exception handler to prevent your code from crashing when your crap library crashes.

Times have changed. Welcome to a world where the programmer who knows how to tap into other people’s brains and experience using the Internet has a decisive advantage.

Working on CityDesk Part: 1 2 3 4 5


For the umpteenth time, I found myself dependent on a code library which had a crashing bug that was unacceptable in code I shipped. What are you supposed to do if you’re the chef at Les Halles and your fishmonger is giving you smelly fish?

Working on CityDesk, Part Four

(Tip: to search the Microsoft knowledge base using Google, add to your query.)


James Pryor is a popular young Harvard philosophy professor who shows The Matrix in class to illustrate points about Epistemology. His article Guidelines on Writing a Philosophy Paper is one of the best tutorials on any kind of writing:

Pretend that your reader is lazy, stupid, and mean. He’s lazy in that he doesn’t want to figure out what your convoluted sentences are supposed to mean, and he doesn’t want to figure out what your argument is, if it’s not already obvious. He’s stupid, so you have to explain everything you say to him in simple, bite-sized pieces. And he’s mean, so he’s not going to read your paper charitably. (For example, if something you say admits of more than one interpretation, he’s going to assume you meant the less plausible thing.)

I can think of no better advice for the kind of writing that programmers should be doing in designing and documenting their code.


I’ve been meaning to write more about the creation of CityDesk, but actually our beta is going extremely well, meaning, we’re finding lots of bugs, so I’ve been incredibly busy.

To start the beta off, we released CityDesk to about 100 testers. Most of the bug reports arrived within the first 24 hours. We gathered bugs and feedback from several sources: the CityDesk discussion group, email, a menu item we have called “Send Beta Feedback,” and Michael’s excellent bug-catching feature (if CityDesk crashes, it offers to send a bug report to us, which goes right into FogBUGZ.)

Indeed many of the bug reports we’re getting now are just what we expected: configuration issues. People have funny versions of Internet Explorer. People have their computers set to display everything in large fonts. And something we were doing wrong happened to work OK in Windows 2000, but in NT 4.0, it failed. It would be almost impossible for a small company to find these kinds of bugs without beta testers. And it would have been impossible for us to handle the feedback from more than about 100 people!

Next Monday, I hope, we’ll release another version with almost everything fixed, this time to a larger group — about 200 more people (and of course the first 100 are welcome to download the update). The plan is that we’ll have one last beta two weeks after that, putting us on schedule for releasing on or around December 1st.

Working on CityDesk, Part Three

UNM Humanities Building, where my dad had his officeLike all good hackers, I have been programming since junior high, using my dad’s account on the University of New Mexico IBM 360 mainframe. But the first real computer science course I took was during the first year of college. The course covered C and assembler. It had hundreds of students, most of whom, like me, had been fooling around in Pascal or Basic since they were toddlers.

The course proceeded merrily until one day, the professor introduced pointers.

A rock in Bandelier Park, New Mexico


Suddenly, the majority of the students in the class were in deep trouble. For some reason, some people just do not seem to be able to write code with pointers in it. They were born without the part of the brain that does indirection, I guess.

Since then, I’ve mentally divided the world into three groups. The largest group of people can’t program at all. There’s another, smaller group of people who can program, but not with pointers. And there’s a tiny group of people who can program, even with pointers. Those elite few can even understand what it means to write CString*& in C++.

My first job at Microsoft was putting a decent scripting language into Excel. Although I was given free rein to implement whatever language I saw fit, we went with Basic for several reasons. (1) We had a Basic compiler team in house. (2) You didn’t need pointers, which meant that (3) more people are comfortable using Basic than any other language. Indeed Visual Basic is the best-selling language product of all time.

Visual Basic is an extremely productive way to write code, especially GUI code. Want bold text on a dialog box? It’s one click in VB. Now try doing it in MFC. You have to create a subclassed control, it’s a big mess, you have to know all about LOGFONTS and Windows window subclassing and a bunch of other things and you need about three lines of code once you have the magic class.

But many VB programs are spaghetti, either because they’re done as quick and dirty one-offs, or because they’re written by hack programmers without training in object oriented programming, or even structured programming.

What I wondered was, what happens if you take top-notch C++ programmers who dream in pointers, and let them code in VB. What I discovered at Fog Creek was that they become super-efficient coding machines. The code looks pretty good, it’s object-oriented and robust, but you don’t waste time using tools that are at a level lower than you need. I’ve spent years writing code for C++/MFC and years writing code in Visual Basic, and let me tell you, VB is just much, much more productive. Michael and I had a good laugh today when we discovered somebody selling a beta crash-reporting product at $5000 for three months that Michael implemented in CityDesk in two days. (And we actually implemented a good part of ours in C++/ATL). And I also guarantee you that our Visual Basic code in CityDesk looks a lot better than most of the code you find written in macho languages like C++, because we’re good programmers, and we write comments, and our variable names are well-chosen, and we do things the simple way, not the clever way, and so forth.

I’ll go out on a limb here. In my years of experience, I have seen many language and programming fads come and go. But there’s only ONE, that’s right, ONE language feature I’ve ever seen that actually improves your productivity significantly. No, it’s not object oriented programming; no, it’s not intentional programming or assertions or programming by example or CASE or UML or XML or Java. The only thing that improves your programming productivity is using managed code – that is, using a language in which memory management is automatic. Java and .NET languages do this with garbage collection; VB does this with reference counting; I don’t care how you do it, just let me concatenate strings without thinking about where the new bigger string will go and I’ll be happy.

One of the things about Visual Basic is that it doesn’t always give you access to the full repertoire of Windows goodies that you need to make a polished application. But what it does do, better than almost any other programming environment, is let you drop into C++ code (or call C APIs) when you’re desperate or when you need that extra speed. For example, you can always get the HWND of a control and do native stuff to it, which is not the case in Java. As another example, a lot of the non-GUI, time-sensitive inner loops, like the word counter, in CityDesk are actually implemented in C or C++ for speed. This ability gave us the confidence to use Visual Basic even though it can’t do everything and it tends to do string processing slowly. But since we’re all C++ programmers, we have no fear of creating a DLL or OCX to count words, or parse script, or call a Windows API. So about 5% of CityDesk is actually in C or C++, and we’ll probably move a little bit more of the code to C++ to speed up a few more inner loops.

Now, Visual Basic is not the perfect programming language. It’s fairly object oriented but there are little things that you can’t do with it, like have base classes with implementation reuse. It’s limited to Windows. And the worst part about coding in VB is that people think you’re not cool because your code doesn’t have {‘s and }’s. I can live with the shame if it means I’m more productive.

Philosophically, I think that C# has a bright future in the Windows GUI programming world. It’s not as embarrassing as VB, and it uses the type of syntax which C/C++/Java programmers have come to love. VB programmers looking to upgrade can’t upgrade painlessly to VB.NET, because there are so many major differences in the programming environment. Even Microsoft admits that you can’t port from VB to VB.NET, you have to rewrite. And that’s enough of a pain that many VB programmers will use this opportunity to look around at what else is out there. I think many will choose C#, because it’s virtually the same language as VB.NET with slightly different syntax and vastly less stigma attached to it.

What about Java? Yes, I’ve used Java extensively, but unfortunately the language, the code libraries, and especially the GUI libraries are just too primitive for a commercial desktop application. I like the language and I appreciate the benefit of write-once-run-anywhere, but frankly not a lot of desktop software is sold for Sun Solaris and I think that WORA benefits Sun more than it benefits software developers, and I’m not willing to write an app that behaves in an inferior way on 95% of my customer’s computers to benefit the 5% with alternate platforms. Every Java app LOOKS like a Java app, takes forever to launch, and just doesn’t feel completely native. Since CityDesk’s competitive advantage comes from having an excellent GUI, that’s one area where I refuse to skimp.

I am virtually certain that I will now receive a million emails from lovers of tcl/tk, or Delphi, or C++ Builder, or NextStep, or Cocoa, or perl, or Python, or RealBASIC, or some other programming environment which may or may not be suitable for creating a professional Windows GUI. That’s nice. I don’t really want to get into a debate about language features or programming environment features — at some point, you have to stop debating and write code! I’m just trying to explain why we chose to use VB for the GUI part and C++ for time sensitive bits of code. If you do want to have a fun religious war over programming languages, please do so on the discussion group!

Working on CityDesk Part: 1 2 3 4 5


The worst part about coding in Visual Basic is that people think you’re not cool because your code doesn’t have {‘s and }’s. I can live with the shame if it means I’m more productive.

Working on CityDesk, Part 3

PS If you’ve applied for the CityDesk beta, and haven’t heard back, that’s because we’re waiting to fix a bunch of known bugs before we release to the next group of beta testers. It will make your life much easier and we won’t get the same feedback twice.

Discussion Board

I’ve implemented a few more small, invisible-seeming refinements. The big news is that if you include your email address in a posting, it won’t be easily harvestable by spammers.



Does anybody have experience with GUI automated testing tools? I’m going to take advantage of the CityDesk beta to develop an automated test suite (so we don’t keep breaking 2 things for every 1 thing we fix!)

If you have any recommendations, please post them on the discussion group. It needs to support Windows apps and be the sort of thing that doesn’t involve a salesperson visiting your office with a $6000 projector for the demo (in other words, not too expensive.)


Zillions of emails in my inbox. Big stack of bills to pay. Now I’m snowed under with all the stuff I was deferring until the beta shipped 🙂


Server Go Boom and Spray Email

Apparently every time our server went boom this morning (4 or 5 times) the stupid mailing list manager software decided it would be a nice idea to resend my notification email. If you got several copies of today’s Joel on Software bulletin, I sincerely apologize and ask your forgiveness.

Discussion Forums

The CityDesk and Joel on Software forums are quite lively, but we’ve also got a relatively quiet FogBUGZ forum which is feeling lonely 🙂

Fog Creek President Michael Pryor figured out a brilliant trick which makes it so that you see new topics, and topics that have followups you haven’t read, in blue. If you’ve read the entire topic, it will be purple. And it’s all done without keeping any state on the server.

Server Go Boom

Dell 2500SCThe home-built server we’ve been using for keeps crashing whenever the temperature goes above about 80 degrees in the server closet. Grump. I guess it’s time to replace it with a real industrial strength server… I’ve ordered a couple of these.

CityDesk Beta

The CityDesk Beta has started trickling out, and within minutes we’re getting bug reports from the field. Great! That’s why we’re doing a beta. We’ve been banging on this dang thing for months. We fixed 329 bugs. Then Brad W. downloads it, plays for ten seconds, and finds another. And we don’t run so good on NT 4.0. (98, Me, 2000 and XP are all fine).

In Defense of Not-Invented-Here Syndrome

Time for a pop quiz.

Copley Square 1. Code Reuse is:

a) Good
b) Bad

2. Reinventing the Wheel is:

a) Good
b) Bad

3. The Not-Invented-Here Syndrome is:

a) Good
b) Bad

Of course, everybody knows that you should always leverage other people’s work. The correct answers are, of course, 1(a) 2(b) 3(b).


Not so fast, there!

The Not-Invented-Here Syndrome is considered a classic management pathology, in which a team refuses to use a technology that they didn’t create themselves. People with NIH syndrome are obviously just being petty, refusing to do what’s in the best interest of the overall organization because they can’t find a way to take credit. (Right?) The Boring Business History Section at your local megabookstore is rife with stories about stupid teams that spend millions of dollars and twelve years building something they could have bought at Egghead for $9.99. And everybody who has paid any attention whatsoever to three decades of progress in computer programming knows that Reuse is the Holy Grail of all modern programming systems.

Right. Well, that’s what I thought, too. So when I was the program manager in charge of the first implementation of Visual Basic for Applications, I put together a careful coalition of four, count them, four different teams at Microsoft to get custom dialog boxes in Excel VBA. The idea was complicated and fraught with interdependencies. There was a team called AFX that was working on some kind of dialog editor. Then we would use this brand new code from the OLE group which let you embed one app inside another. And the Visual Basic team would provide the programming language behind it. After a week of negotiation I got the AFX, OLE, and VB teams to agree to this in principle.

I stopped by Andrew Kwatinetz’s office. He was my manager at the time and taught me everything I know. “The Excel development team will never accept it,” he said. “You know their motto? ‘Find the dependencies — and eliminate them.’ They’ll never go for something with so many dependencies.”

In-ter-est-ing. I hadn’t known that. I guess that explained why Excel had its own C compiler.

By now I’m sure many of my readers are rolling on the floor laughing. “Isn’t Microsoft stupid,” you’re thinking, “they refused to use other people’s code and they even had their own compiler just for one product.”

Not so fast, big boy! The Excel team’s ruggedly independent mentality also meant that they always shipped on time, their code was of uniformly high quality, and they had a compiler which, back in the 1980s, generated pcode and could therefore run unmodified on Macintosh’s 68000 chip as well as Intel PCs. The pcode also made the executable file about half the size that Intel binaries would have been, which loaded faster from floppy disks and required less RAM.

“Find the dependencies — and eliminate them.” When you’re working on a really, really good team with great programmers, everybody else’s code, frankly, is bug-infested garbage, and nobody else knows how to ship on time. When you’re a cordon bleu chef and you need fresh lavender, you grow it yourself instead of buying it in the farmers’ market, because sometimes they don’t have fresh lavender or they have old lavender which they pass off as fresh.

Indeed during the recent dotcom mania a bunch of quack business writers suggested that the company of the future would be totally virtual — just a trendy couple sipping Chardonnay in their living room outsourcing everything. What these hyperventilating “visionaries” overlooked is that the market pays for value added. Two yuppies in a living room buying an e-commerce engine from company A and selling merchandise made by company B and warehoused and shipped by company C, with customer service from company D, isn’t honestly adding much value. In fact, if you’ve ever had to outsource a critical business function, you realize that outsourcing is hell. Without direct control over customer service, you’re going to get nightmarishly bad customer service — the kind people write about in their weblogs when they tried to get someone, anyone, from some phone company to do even the most basic thing. If you outsource fulfillment, and your fulfillment partner has a different idea about what constitutes prompt delivery, your customers are not going to be happy, and there’s nothing you can do about it, because it took 3 months to find a fulfillment partner in the first place, and in fact, you won’t even know that your customers are unhappy, because they can’t talk to you, because you’ve set up an outsourced customer service center with the explicit aim of not listening to your own customers. That e-commerce engine you bought? There’s no way it’s going to be as flexible as what Amazon does with obidos, which they wrote themselves. (And if it is, then Amazon has no advantage over their competitors who bought the same thing). And no off-the-shelf web server is going to be as blazingly fast as what Google does with their hand-coded, hand-optimized server.

This principle, unfortunately, seems to be directly in conflict with the ideal of “code reuse good — reinventing wheel bad.”

The best advice I can offer:

If it’s a core business function — do it yourself, no matter what.

Pick your core business competencies and goals, and do those in house. If you’re a software company, writing excellent code is how you’re going to succeed. Go ahead and outsource the company cafeteria and the CD-ROM duplication. If you’re a pharmaceutical company, write software for drug research, but don’t write your own accounting package. If you’re a web accounting service, write your own accounting package, but don’t try to create your own magazine ads. If you have customers, never outsource customer service.

If you’re developing a computer game where the plot is your competitive advantage, it’s OK to use a third party 3D library. But if cool 3D effects are going to be your distinguishing feature, you had better roll your own.

The only exception to this rule, I suspect, is if your own people are more incompetent than everyone else, so whenever you try to do anything in house, it’s botched up. Yes, there are plenty of places like this. If you’re in one of them, I can’t help you.




“When you’re working on a really, really good team with great programmers, everybody else’s code, frankly, is bug-infested garbage, and nobody else knows how to ship on time.”

In Defense of Not-Invented-Here Syndrome

Discussion Board

The cool new discussion board is done!


The URL for the new discussion server is .

Underline Edit Boxes

I like to make fun of designers who reinvent common widgets simply because they can. The big advantage of using a standard HTML edit box is consistency — more people will recognize it and know how to use it.

But when you have a long form with many fields, all those lines start to look ugly, so I’ve gotten into the bad habit of using style sheets so that my edit boxes are just underlines, not boxes. This is “consistent” more with paper forms, and someone even told me he thought he was going to have to print out the page when he saw the beta application form using underlines. 

It probably does reduce learnability, marginally, to use underlines for edit boxes instead of boxes. But then again, they’re really cool looking 🙂 and I can’t bear to switch them back to boxes. What’s your opinion?