Fog Creek Software Management Training Program

Here’s how I learned to run the night shift at the Oranim bakery when I was a teenager.

First, you do an awful lot of cleaning. A few months of cleaning dough off of machines gives you a really good understanding of what the machines are and how they work and what parts matter.

Then you run the dough mixers for a few months until you get really good at that. Then you learn to make rolls. Then they let you run the computers that control the nightly bread production. If you’re lucky, the computer will break down once or twice, and you can learn how to do it manually. That’s really cool.

Finally, when you’re really really good, they let you hang around with Yussef on the ovens. Yussef was about 100 years old and so good at running the ovens it was scary. When Gabbi tried to show me how to solve the problem of bread sticking to the conveyer belts on the way out of the oven, he ran back and forth like a lunatic for ten minutes, turning knobs, pulling levers, redirecting heat, and burning a few hundred loaves while he struggled to get things under control. But Yussef, facing the same problem, turned one tiny knob on a seemingly-unrelated chimney about one degree to the right. It made no sense, he couldn’t explain why it worked, but it did: it solved the problem instantly and suddenly perfect loaves started popping out. It took me another couple of years to really understand the complex relationships between heat and humidity inside an 80 foot tunnel oven, but it would have taken ten more years before I could solve problems as well as Yussef did.

After a year or so doing every job in the bakery, you have credibility with the crew you’re leading because they all know you could do their job. You have a complete, holistic understanding of the bakery in a way that no individual does. You’ve had a chance to watch good managers and bad, you’ve experienced a lot of the things that can go wrong and seen how to fix them (and how not to fix them), and you can be trusted to drive a fifty million dollar factory that churns out hundreds of thousands of loaves every night.

So they put you in charge of the night shift.

Although Yussef will probably pretty much ignore you no matter how much you think you know.

I told you that so I can tell you this:

We’re planning to hire a new kind of employee at Fog Creek.

Until now, most of our hires have been programmers. That’s a start, but we need to start hiring the next generation of management too.

Now, there’s nothing wrong with promoting a programmer to management, but management is a different job and requires different skills from programming. Many people who are excellent developers are lousy managers, and promoting someone out of a job that they love doing and are good at doing into a job that they hate and are not good at doesn’t make sense. We plan to make sure that programmers have explicit career paths that do not require them to shift into management just to get the next raise in salary level or benefits.

Thus: how do we develop the next generation of managers? We don’t really want to hire MBAs, because there’s too much evidence that MBAs substitute book-learnin’ for common sense or experience. We’d much rather hire someone who created and ran a profitable lemonade stand than someone who has taken two years of finance courses at Harvard, especially since the Harvard MBA is going to think he knows a lot more than he really does.

Our latest thinking is just to train a new generation of leaders from the ground up.

To that end, today we’re launching an experimental new program, the Fog Creek Software Management Training Program. That’s a terrible name, but bear with me!

It’s an entry level program, meaning, significant work experience is not required. A college degree is a big plus. The program will last about three years. It will provide experience working in all aspects of software development except for the actual coding itself, along with some formal training.

It’s a job. It includes a great starting salary, restricted stock in Fog Creek Software, and the full raft of benefits, from Aeron chairs to free lunch. You don’t have to be a programmer or a CS major, although you do have to be ridiculously smart and you have to be the kind of person who gets things done.

The key component of this program is rotating through just about every job at Fog Creek Software. We’ll rotate trainees through about ten different jobs over the course of three years:

  • Project Management
  • Tech Support
  • Inside Sales
  • Software Testing
  • Usability Testing
  • Software Design
  • Program Management
  • Beta Management
  • Marketing
  • Build Management

…sometimes all on the same day! The theory is that nothing can better prepare you for leading a high tech company than gaining significant, substantial experience with everything a high tech company does, under the mentorship of experienced veterans.

To supplement that, we’ll add a component of formal training. There will be some coursework at nearby colleges, long lists of reading material, intensive offsite training programs, and we’ll send you to industry conferences that we think are particularly valuable.

Unlike full-time MBA programs, you won’t have to pay $120,000 in tuition and $200,000 in foregone salary. You’ll be earning money and if Fog Creek does well your stock may be worth something. Unlike management consulting, you won’t have to work 18 hour days and fly to some small town in the midwest every Monday morning and live out of a suitcase in a hotel.

The idea of this program is to develop a new generation of leaders for Fog Creek, but we think that it will be great preparation for a career leading, running, or starting any kind of high-tech company or team. If the program is successful we expect, in the long run, to churn out about twice as many graduates as we need for our own purposes, so many will tend to head off to start their own companies, take a high-level position elsewhere in the industry, or go back to graduate school. Either way we think it’s a fantastic opportunity for ambitious, smart geeks who don’t see themselves as programmers.

We’re taking applications now for the first round of Fog Creek Management Trainees, most of whom will start work in summer 2006. To apply, send a letter explaining why you’re a good fit, a resume, and any standardized test scores to jobs@fogcreek.com.

Small print: We can only accept applicants who have the permanent legal right to work in the United States and are available for an interview in New York. Fog Creek Software, Inc. does not discriminate in employment matters on the basis of race, color, religion, gender, national origin, age, military service eligibility, veteran status, sexual orientation, marital status, disability, or any other protected class. We support workplace diversity.

Introduction to Best Software Writing I

Book Cover IllustrationThis is the introduction to The Best Software Writing I, Selected and Introduced by Joel Spolsky, now in bookstores.

New York City is a blast.

Just the other day, as I was walking the four blocks from my office to the subway entrance, interesting things kept happening.

Not really interesting things, just modestly interesting things.

So, for example, some guy was running down the sidewalk frantically, looking very much like a character in an R. Crumb comic, flapping his arms broadly and making chicken sounds. Running isn’t the right word. He was kind of pratfalling repeatedly and then catching himself right before he hit the ground.

Then a taxi turning the corner nearly knocked over an old man who was crossing the street a little bit too slowly for the taxi driver’s taste.

A couple of chubby, red-faced out of towners asked me if there was a bar anywhere nearby. (There was. We were in front of it.)

Someone was handing out little advertising cards at the entrance to the subway. Of course, the inside of the subway station was completely littered with the cards because everybody who took one immediately hurled it on the ground as violently as you can hurl a four by six postcard. I almost slipped on one on the steps down.

Modestly interesting stuff, but quite forgettable in New York.

The next day I was talking to one of the summer interns we just hired. For some reason, this year’s summer intern class consists of 75% people who are either from Indiana or who went to school in Indiana. Indiana, for those of you not familiar with our American landscape, is somewhere in the middle, a state of farms, wholesome colleges with corn-fed basketball-playing kids, Norman Rockwell towns, and the occasional rust-belt hellmouth industrial city gasping its last breath. (As I write these words I brace for the slew of angry letters from the Indiana Department of Tourism and Infrastructure promoting the exciting cultural scene, the many picturesque lakes, the world-class telephone system, and the variety of ethnic restaurants. You might find a Mexican restaurant and an Italian restaurant on the same block!)

Anyway, the intern said he had never lived in New York City, and asked me what it was like. I didn’t really have a good answer, but I said, “New York is the kind of place where ten things happen to you every day on the way to the subway that would have qualified as interesting dinner conversation in Bloomington, Indiana, and you don’t pay them any notice.”

Feeling smug with myself, I pulled down an atlas from the bookshelf to find another state to insult.

Anyhow, I can’t remember why I told you that story.

Oh, wait, yes I can, but first I have to tell you another story.

A few months ago, I got a review copy of a book from another publisher, other than the publisher of this book, who will remain anonymous, and the book will remain anonymous, and the author will remain anonymous, because I’m afraid I just have nothing good to say about said book.

The publisher wanted to get a quote from me to put on the back cover talking about how wonderful his book was. Normally I’d be happy to do that; I’m a complete publicity slut and will do just about anything to get my name in front of the reading public. My hope is that if I do this enough, telemarketers who call me at home will be able to pronounce my name.

The book started out looking promising. It filled a real need. I remember several times standing in bookstores desperately trying to find a book on the very topic, but there was nothing to be found. So I started reading the manuscript full of high hopes.

Bleah.

I could hardly bear to keep reading.

The author kept saying smart and interesting things.

He even wrote clearly.

But the book was thoroughly, completely, boring. And worse, it was completely unconvincing.

The author had violated the number one rule of good writing, the “Show, don’t tell” rule. There was not a single story in the book. It was chock full of sentences like “A good team leader provides inspiration by setting a positive example.” What the eff?

Pay attention. Here’s the way to say “a good team leader provides inspiration by setting a positive example” without putting your audience to sleep:

For a few months in the army I worked in the mess hall, clearing tables and washing dishes nonstop for 16 hours a day, with only a half hour break in the afternoon, if you washed the dishes really fast. My hands were permanently red, the front of my blouse was permanently wet and smelly, and I couldn’t take it any more.

Somehow, I managed to get out of the mess hall into a job working for a high-ranking Sergeant Major. This guy had years of experience. He was probably twenty years older than the kids in the unit. Even in the field, he was always immaculate, wearing a spotless, starched, pressed full dress uniform with impeccably polished shoes no matter how dusty and muddy the rest of the world was around him. You got the feeling that he slept in 300 threadcount Egyptian cotton sheets while we slept in dusty sleeping bags on the ground.

His job consisted of two things: discipline and the physical infrastructure of the base. He was a bit of a terror to everyone in the battalion due to his role as the chief disciplinary officer. Most people only knew him from strutting around the base conducting inspections, screaming at the top of his lungs and demanding impossibly high standards of order and cleanliness in what was essentially a bunch of tents in the middle of the desert, alternately dust-choked or mud-choked, depending on the rain situation.

Anyway, on the first day working for the Sergeant Major, I didn’t know what to expect. I was sure it was going to be terrifying, but it had to be better than washing dishes and clearing tables all day long (and it’s not like the guy in charge of the mess hall was such a sweetheart, either!)

On the first day he took me to the officer’s bathroom and told me I would be responsible for keeping it clean. “Here’s how you clean a toilet,” he said.

And he got down on his knees in front of the porcelain bowl, in his pressed starched spotless dress uniform, and scrubbed the toilet with his bare hands.

To a 19 year old who has to clean toilets, something which is almost by definition the worst possible job in the world, the sight of this high ranking, 38 year old, immaculate, manicured, pampered discipline officer cleaning a toilet completely reset my attitude. If he can clean a toilet, I can clean a toilet. There’s nothing wrong with cleaning toilets. My loyalty and inspiration from that moment on were unflagging. That’s leadership.

See what I did here? I told a story. I’ll bet you’d rather sit through ten of those 400 word stories than have to listen to someone drone on about how “a good team leader provides inspiration by setting a positive example.”

Anyway, I called up the editor of the book that they wanted me to praise, and said I couldn’t, in good faith, recommend a boring book without any stories in it, even if it was 100% correct and otherwise well-written. I think they hate me now.

 

So be it.

 

The software development world desperately needs better writing. If I have to read another 2000 page book about some class library written by 16 separate people in broken ESL, I’m going to flip out. If I see another hardback book about object oriented models written with dense faux-academic pretentiousness, I’m not going to shelve it any more in the Fog Creek library: it’s going right in the recycle bin. If I have to read another spirited attack on Microsoft’s buggy code by an enthusiastic nine year old Trekkie on Slashdot, I might just poke my eyes out with a sharpened pencil. Stop it, stop it, stop it!

And that’s why when Gary Cornell suggested this book, I leapt at the idea. It would be a chance to showcase some of the best writing about software from the past year “or so.” The original idea was to make it an annual, so the volume you’re holding would be “The Best Software Writing of 2004,” but there were a bunch of great articles from 2003 that we wanted to include, and we were afraid bookstores would return it at the end of the year if there was a date in the title. I solicited nominations from the faithful readers of my website, Joel on Software, and selected the final stories myself, so the blame for what’s included and what isn’t included is entirely my own, but full credit for really incredible writing in a field that doesn’t normally get any goes to the contributors.

 

The Best Software Writing I is now available. It includes:

Ken Arnold – Style Is Substance 
Leon Bambrick – Award for the Silliest User Interface: Windows Search 
Michael Bean – The Pitfalls of Outsourcing Programmers
Rory Blyth – Excel as a Database
Adam Bosworth – ICSOC04 Talk
danah boyd – Autistic Social Software
Raymond Chen – Why Not Just Block the Apps That Rely on Undocumented Behavior?
Kevin Cheng and Tom Chi – Kicking the Llama
Cory Doctorow – Save Canada’s Internet from WIPO
ea_spouse – EA: The Human Story
Bruce Eckel – Strong Typing vs. Strong Testing
Paul Ford – Processing Processing
Paul Graham – Great Hackers
John Gruber – The Location Field is the New Command Line
Gregor Hohpe – Starbucks Does Not Use Two-Phase Commit
Ron Jeffries – Passion
Eric Johnson – C++ — The Forgotten Trojan Horse
Eric Lippert – How Many Microsoft Employees Does it Take to Change a Lightbulb?
Michael “Rands” Lopp – What to do when you’re screwed
Larry Osterman – Larry’s Rules of Software Engineering #2: Measuring Testers by Test Metrics Doesn’t
Mary Poppendieck – Team Compensation
Rick Schaut – Mac Word 6.0
Clay Shirky – A Group is its Own Worst Enemy
Clay Shirky – Group as User: Flaming and the Design of Social Software
Eric Sink – Closing the Gap
Eric Sink – Hazards of Hiring
Aaron Swartz – PowerPoint Remix
why the lucky stiff – A Quick (and Hopefully Painless) Ride Through Ruby (with Cartoon Foxes)

Wall Street Survival 101

Yesterday, Dan Bricklin was interviewing me for his new podcast, and after a lot of questions about open source, which I’m afraid I answered in a way which will likely cause great consternation to people who think open source benefits programmers economically, but anyway, after all the open source questions, he asked me if I had anything to add.

“Yes!” I said. “Never, ever, ever buy bonds at retail from a full-service broker. Especially municipal bonds.”

It wasn’t really related to open source but it’s pretty important that if you have a retail brokerage account and your broker tries to sell you bonds, you don’t buy them. The typical commissions they take will tend to lop off a year of interest, so the day you buy the bond, you’ve lost money. But there’s something even scarier with the retail bond market — basically, a scam — that Wall Street uses to make certain bonds look like really good investments to you, the average investor, when they’re really bad investments.

Here’s the puzzle I published on my website yesterday:

You have $100,000 to invest. Which government bond should you invest in?

Bond A, B or C?

Bond A pays 4.15%. If you buy this bond, you’ll get a check for $4150 in interest at the end of every year for 10 years. With your last check, you’ll get your $100,000 back.

Bond B pays 4.5%. You’ll get a check for $4500 at the end of every year for 30 years, then with your last check, you’ll get your $100,000 back.

Bond C pays even more! It’s 4.75%, w00t. You will get $4750 at the end of every year for 10 years, then get your original investment back, unless the government decides they want to keep your money for a little longer, in which case you’ll get another 20 years of $4750 before you get your money back.

Bond C is a little more complicated so let me explain. On the tenth anniversary, the government gets to decide whether to pay you your $100K back and owe you nothing, or keep your $100K for another twenty years and keep paying you the interest. The point being, the choice is up to the government.

But either way, with Bond C, you’re getting MORE interest every year, whether they decide to keep your money for 10 years or 30.

So, it’s stupid to buy bonds A or B, right? Bond C is obviously the better choice.

Right?

Hints: assume that you’re a left-handed avocado farmer and therefore don’t pay any tax. So ignore the tax implications. Then, assume that all the bonds are equally sound, backed by the “full faith and credit of the United States” which, in the financial world, means there is absolutely no chance of default. This is not a trick question.

Before I get started, let’s clear off some distractions from the table.

  1. It has nothing to do with compounding. The bonds I described do not compound. They just pay you interest once a year. What you do with that interest is your problem. You might buy some different bonds, but who knows what interest rate you’re going to get? Interest rates change all the time and are unpredictable.
  2. This is not a question about whether you’re going to need the money in 10 years, or whether you’re going to die in 20 years, or whether the government deficits are going to grow or shrink. I’m just asking, all else being equal, which bond is a better deal?

Now on to the answer.

Bond C looks, to almost everybody, to be a better deal. It’s for a higher interest rate, so your broker can tell you little lies like, “You’re getting 4.75% on that thing… tax free! What do you get at your bank? Maybe 2%?” Most people just look at the interest rate and try to invest in the thing with the highest interest rate. So for that reason alone, most people would just choose Bond C.

But a few brave intelligent investors actually try to work out the problem in their head and they reason as follows:

  • If I take bond C, there are two possibilities.
    1. The government pays me back in 10 years, or
    2. The government pays me back in 30 years.
  • In case #1 I make more money than I would have made with bond A… for ten years I’m making $4750 a year instead of $4150, and that’s more!
  • In case #2 I make more money than I would have made with bond B… for thirty years I’m making $4750 a year instead of $4500, and that’s more money too!
  • So in either case, I make more money!
  • Mo money, mo money, mo money, give me the more money! What could be wrong with taking a risk where in EITHER situation I’m better off?

And these people buy bond C, too.

And they are getting ripped off.

I’ll try to explain why.

The secret lies on that day, 10 years in the future, where the government decides whether to pay you back right now or keep your money for an extra 20 years.

How are they going to decide?

Some people think, “Well, it depends on whether we have a bad president who keeps running deficits, so they need more money, or if we have a good president who is paying off the national debt, and they don’t need the money.”

No. That’s not what it depends on, actually. What it depends on is only one thing: what are the interest rates on that day in the future.

Here’s why.

Lets say interest rates go down. So in ten years, the interest rate goes down below 4.75%.

In that case, the government is going to pay you back on the spot, because they can borrow the same money somewhere else for cheaper. For example, if the interest rate in ten years is only 4%, the government can save $750 a year by borrowing elsewhere and paying off your loan, so they’d be crazy not to do it.

But what if interest rates go up?

In that case, the government is not going to pay you back, even if they have extra money burning a hole in their government pockets! If they don’t need the money, they can just loan it to someone else. For example, if the interest rates in ten years are 6%, then the government can loan your $100,000 to someone else at 6%, making $6000 a year, and paying you your $4750 a year, and they’re making a $1250 profit which they can use to give tax refunds to rich Texasshire oilmen and bomb third world countries, or whatever they want to do.

But the important point is:

  • If rates go down they will prepay.
  • If rates go up they will not prepay.

Notice it has nothing to do with whether the government “needs” your money or not in ten years. The only thing it depends on is interest rates.

Now, let’s reexamine Bond C from your perspective.

You buy the bond.

Rates go up.

Now you’re screwed, because you’re stuck with your crappy low-interest rate bond even though everybody else is getting high interest rates.

Or let’s say rates go down.

Now you’re screwed, because the government pays you back, and you have all this money, and you can only invest it at the new, sucky, lower rate.

You’re screwed either way. The only way you’re not screwed is if rates stay approximately the same.

When you buy Bond C, you’re giving up something valuable. You’re giving the government an option to call your bond.

This option has monetary value, because you’re agreeing to take a hit if interest rates change. And people on Wall Street can actually calculate this value.

In fact, Bond C, as of today, “at today’s rates and implied volatility,” was worth about $2000 less than Bonds A or B, which were of equal value “at today’s rates and implied volatility.”

“Implied volatility” is a fancy word to mean “the bond market’s collective opinion, as reflected in the prices they are willing to pay for various things, as to how likely interest rates are to move up or down in the future.

Let me translate. Bond C, which looks like a better deal to almost everyone who doesn’t know how to trade bonds, is worth $2000 less.

If you took all three of these bonds and tried to sell them today on the bond market, you’d get $100,000 for Bond A and Bond B, while you would only get $98,000 for Bond C.

How did I calculate that?

Well, I didn’t. I have no idea how to calculate it. Jared calculated it for me (he works on Wall Street) by going into his office, closing the door, and banging away for the computer for a while. He was doing weird and complicated things with his Bloomberg terminal ($1500/month), spreadsheets (free with Microsoft Office), and the latest Green Day album ($8/month paid to Rhapsody). He needed a yield curve, he needed to know the implied volatility, and all kinds of math things. Smoke came out from under the door to his office. After a while he came out, rumpled of hair and shirt, and announced that Bond C was worth $2000 less.

So the retail brokers selling these bonds have snuck a sneaky derivative into the bond, and you can’t calculate what it’s worth unless you do this for a living, but I assure you that they have calculated it to the nearest penny and are helping themselves to a couple of percent of your money there, and the long and the short of it is that you’re getting ripped off in a way you can’t possibly be sophisticated enough to calculate.

This is the scam.

Most bonds sold to investors at retail by full service brokers are disguised to look like they pay a lot of interest when the truth is that the high interest rates never really compensate you for the option you’re implicitly selling.

This is not a coincidence. It’s by design. All else being equal, the people who sell these bonds love to sneak in prepayment options because it enables them to make it look like you’re getting a higher rate of return. These bonds are designed to rip off retail investors.

But then again, full service brokers, in this day and age of low cost mutual funds and discount brokers, are really nothing more than machines for ripping off retail investors, so don’t be surprised!

Documentary Filmmaker Wanted

We are looking for a talented filmmaker, student or experienced, to make a documentary about the software development process this summer. If you think you’re interested, read on for more details!

This summer, Fog Creek Software has hired four summer interns from Yale, Duke, and Rose-Hulman. Our selection process was extremely competitive, with over 800 kids applying for only four positions.

Instead of wasting their talents giving them the usual dull and unimportant tasks of a typical summer internship, we decided to let the interns create a complete new software product, from beginning to end, over the course of one summer. With experienced software developers as mentors, the team will design, program, test, and roll out a complete software product over the course of one hectic summer, going from concept to paying customers in about ten weeks.

This project would be an excellent opportunity for a documentary filmmaker to create a film about the software development process, in the spirit of Code Rush, Triumph of the Nerds, Startup.com, etc. We’ve been thinking of it as a kind of serious Apprentice-meets-Real-World. Unlike The Apprentice, our business challenge will be a serious ten week project that becomes a real product, not four hours of wrapping chocolate bars disguised as a “business challenge.” And we don’t have a crazy boss with a bad combover firing people every week.

Why would I want to make this film?

We think it will be extremely interesting to everyone who is curious about the process of developing software. It is a unique opportunity to film the complete software development process over the course of one summer in New York City. And it might serve as an excellent demo for your portfolio if you are interested in making documentaries, industrials, or working on a reality TV show.

How will the film be financed?

This is not an industrial; it’s a documentary. We’re looking for a filmmaker who will finance and make the film themselves and own the rights. We want someone who can promote the film to typical documentary outlets (TV, IFilm, cable, video, etc, or just as a student project for a class).

Producing the film on digital video is probably good enough, but again, it’s your film, so it’s up to you.

Fog Creek Software will make available a cash grant of $5000 plus up to $5000 in expense reimbursement to the filmmaker we chose to work with, conditional on producing a finished film. If you’re a student and are willing to scrounge a little bit, this may be enough to actually make the film in DV. If you’re a professional documentary filmmaker, you probably know how to raise money to make films better than we do.

Again, this is not intended to finance the film, and we do not expect to own the rights to the film; it’s just a subsidy that may make it easier.

Who will see the film?

The founder of Fog Creek Software, Joel Spolsky, has his own website called Joel on Software which is extremely popular among software developers around the world. This site gets over 7,000,000 hits a month, has 350,000 unique visitors every month, and usually ranks first among personal web sites read by software developers. The Joel on Software audience is extremely interested in the software development process and will likely be very interested in this film. The interns will be maintaining a weblog all summer describing the development process, and building up a natural audience for a documentary about their experience.

When the documentary is ready, Fog Creek Software will make the DVD available on its website, serving as a traditional retailer and promoting the film extensively to Fog Creek customers and the Joel on Software audience. This will get your film a fairly decent audience, as documentaries of this type go, but we can’t make any promises. You will probably want to cut a trailer as well which we will use to promote the film on our websites.

What’s in the documentary?

You’re the filmmaker. But here are some of the things you might want to film:

  • The project kick-off meeting and progress meetings
  • Interviews with the developers and the interns
  • Candid interviews with interns about their experience, before, during, and after the project
  • Installing new servers at our data center
  • We have a very photogenic, architect-designed loft office in the New York City fashion district
  • Other summer activities that our interns do (moving into summer housing, night on the town, New York touristy activities like visits to museums and the Circle Line which we will organize for the interns)
  • Fog Creek founder Joel Spolsky will be giving a speech to a large group of developers at a conference in Washington DC

The goal is to tell the story of a the summer internship and the new product development simultaneously, in the form of a fairly level-headed documentary. We’re not making a reality TV program, and the participants’ personal lives are their own.

What’s the Schedule?

Filming will begin no later than May 15th. The interns will be moving in around May 29th. The development project will be finished by August 15th.

I’m Interested! Sign Me Up!

The only way to apply if you’re interested is by sending a hard copy resume/CV, cover letter, and DVD portfolio of your work to Fog Creek Software as soon as possible:

Fog Creek Software
Project Aardvark
535 8th Ave., 18th Floor North
New York, NY 10018

Nothing you send us can be returned, and we cannot accept applications sent via email or any other method other than plain old mail. The application deadline is April 15th, 2005, which is really soon, so get on it!

Any Questions?

Email questions to Joel Spolsky, spolsky [at] fogcreek [dot] com.

Contents of Joel on Software, the Book

Contents of the Joel on Software book:

  • Introduction (all new!)
  • Choosing a Language
  • Back to Basics
  • The Joel Test: 12 Steps to Better Code
  • The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)
  • Painless Functional Specifications
  • Painless Software Schedules
  • Daily Builds Are Your Friend
  • Hard-assed Bug Fixin’
  • Five Worlds
  • Paper Prototyping
  • Don’t Let Architecture Astronauts Scare You
  • Fire And Motion
  • Craftsmanship
  • Three Wrong Ideas From Computer Science
  • Biculturalism
  • Every Crash, Everywhere
  • Interviewing
  • Incentive Pay Considered Harmful
  • Top Five (Wrong) Reasons You Don’t Have Testers
  • Human Task Switches Considered Harmful
  • Things You Should Never Do, Part I
  • The Iceberg Secret, Revealed
  • The Law of Leaky Abstractions
  • Lord Palmerston on Programming
  • Measurement
  • Foreword to “In Search of Stupidity”
  • What is the Work of Dogs in this Country?
  • Getting Things Done When You’re Only a Grunt
  • Two Stories
  • Big Macs vs. The Naked Chef
  • Nothing is as Simple as it Seems
  • In Defense of Not-Invented-Here Syndrome
  • Strategy Letter I: Ben and Jerry’s vs. Amazon
  • Strategy Letter II: Chicken and Egg Problems
  • Strategy Letter III: Let Me Go Back!
  • Strategy Letter IV: Bloatware and the 80/20 Myth
  • Strategy Letter V: The Economics of Open Source
  • Murphy’s Law Gone Wild
  • How Microsoft Lost the API War
  • Microsoft Goes Bonkers
  • Our .NET Strategy
  • Please Sir May I Have a Linker?
  • The Best of Ask Joel

 

Please Sir May I Have a Linker?

For some reason, Microsoft’s brilliant and cutting-edge .NET development environment left out one crucial tool… a tool that has been common in software development environments since, oh, about 1950, and taken so much for granted that it’s incredibly strange that nobody noticed that .NET doesn’t really have one.

The tool in question? A linker. Here’s what a linker does. It combines the compiled version of your program with the compiled versions of all the library functions that your program uses. Then, it removes any library functions that your program does not use. Finally, it produces a single executable binary program which people can run on their computers.

Instead, .NET has this idea of a “runtime” … a big 22 MB steaming heap of code that is linked dynamically and which everybody has to have on their computers before they can use .NET applications.

Runtimes are a problem, much like DLLs, because you can get into trouble when application version 1 was designed to work with runtime version 1, and then runtime version 2 comes out, and suddenly application version 1 doesn’t work right for some unpredictable reason. For example right now for some reason our internal company control panel is rounding sales figures to four decimal points, as a result of upgrading from 1.0 to 1.1 of the runtime. Usually the incompatibilities are worse.

In fact .NET includes an extensive technology system called “manifests” which are manifestly complicated and intended to insure that somehow only the right runtime will be used with a given application, but nobody I know can figure out how to use them.

This calls for a story. At the Fog Creek New Year’s Eve party, we wanted a bunch of computer screens in the main room to display a countdown until midnight. Michael wrote an application to do this in C# with WinForms in about 60 seconds. It’s a great development environment.

My job was getting countdown.exe to run on three computers. Sounds easy.

Nope. Double click the EXE, and I got a ridiculously user-hostile error message about mscoree.dll or something, followed by a gratuitous dump of my path. No mention of the fact that the problem was simply that the .NET runtime was not installed. Luckily I’m a programmer and I figured that must be the problem.

How do you install the runtime? The “easiest” way is through Windows Update. But Windows Update really wanted me to get all the critical updates first before I installed the runtime. That’s reasonable, right? Two of the “critical” updates were a Windows service pack and a new version of Internet Explorer, both of which required a reboot.

All told, for each computer I needed to run this little .NET application on, I had to download something like 70 or 80 MB (good thing we have a fast net connection) and reboot three or four times. And this is at a software company! I know how long it took, because the first time it started downloading, I put Office Space on the big screen TV, and by the time the movie was over, the installation process was almost finished. Every ten minutes during the movie I had to jump up, go to each computer, and hit OK to some stupid dialog box.

This is frustrating enough for our in-house apps. But think about our product CityDesk. Almost all of our users download a free trial version before buying the product. The download is around 9 MB and has no additional requirements. Almost none of these users has the .NET runtime yet.

If we asked our trial users, usually small organizations and home users, to go through a movie-length installation hell just to try our app, I think we’d probably lose 95% of them. These are not customers yet, they’re prospects, and I can’t afford to give up 95% of my prospects just to use a nicer development environment.

“But Joel,” people say, “eventually enough people will have the runtime and this problem will go away.”

I thought that too, then I realized that every six or twelve months Microsoft ships a new version of the runtime, and the gradually increasing number of people that have it deflates again to zero. And I’ll be damned if I’m going to struggle to test my app on three different versions of the runtime just so I can get the benefit of the 1.2% of the installed base that has one of the three.

I just want to link everything I need in a single static EXE that runs without any installation prerequisites. I don’t mind if it’s a bit bigger. All I would need is the functions that I actually use, the byte code interpreter, and little bit of runtime stuff. I don’t need the entire C# compiler which is a part of the runtime. I promise CityDesk doesn’t need to compile any C# source code. I don’t need all 22 MB. What I need is probably five or six MB, at most.

I know of companies that have the technology to do this, but they can’t do it without permission from Microsoft to redistribute bits and pieces of the runtime like the byte code interpreter. So Microsoft, wake up, get us some nice 1950s-era linker technology, and let me make a single EXE that runs on any computer with Win 98 or later and no other external dependencies. Otherwise .NET is fatally flawed for consumer, downloaded software.

Craftsmanship

Making software is not a manufacturing process. In the 1980s everyone was running around terrified that Japanese software companies were setting up “software factories” that could churn out high quality code on an assembly line. It didn’t make any sense then and it doesn’t make sense now. Shoving a lot of programmers into a room and lining them up in neat rows did not really help get the bug counts down.

If writing code is not assembly-line style production, what is it? Some have proposed the label craftsmanship. That’s not quite right, either, because I don’t care what you say: that dialog box in Windows that asks you how you want your help file indexed does not in any way, shape, or form resemble what any normal English speaker would refer to as “craftsmanship.”

iPodWriting code is not production, it’s not always craftsmanship (though it can be), it’s design. Design is that nebulous area where you can add value faster than you add cost. The New York Times magazine has been raving about the iPod and how Apple is one of the few companies that knows how to use good design to add value. But I’ve talked enough about design, I want to talk about craftsmanship for a minute: what it is and how you recognize it.

I’d like to tell you about at a piece of code I’ve been rewriting for CityDesk 3.0: the file import code. (Advertainment: CityDesk is my company’s easy-to-use content management product.)

The spec seems about as simple as any snippet of code can be. The user chooses a file using a standard dialog box, and the program copies that file into the CityDesk database.

This turned out to be a great example of one of those places where “the last 1% of the code takes 90% of the time.” The first draft of the code looked like this:

  1. Open the file
  2. Read it all into a big byte array
  3. Store the byte array in a record

Worked great. For reasonable sized existing files it was practically instantaneous. It had a few little bugs, which I worked through one at a time.

The big bug surfaced when I stress-tested it by dragging a 120 MB file into CityDesk. Now, it is not common by any means for people to post 120 MB files on their web sites. In fact, it’s quite rare. But it’s not impossible, either. The code worked but took almost a minute and provided no visual feedback — the app just froze and appeared to be completely locked up. This is obviously not ideal.

From a UI perspective, what I really wanted was for long operations to bring up a progress bar of some sort, along with a Cancel button. In the ideal world you would be able to continue doing other operations with CityDesk while the file copy proceeded in the background. There were three obvious ways to do this:

  1. From a single thread, polling frequently for input events
  2. By launching a second thread and synchronizing it carefully
  3. By launching a second process and synchronizing it less carefully

My experience with #1 is that it never quite works. It is too hard to ensure that all the code throughout your application can be run safely while a file copy operation is in progress. And Eric S. Raymond has convinced me that threads are usually not as good a solution as separate processes: indeed years of experience have shown me that programming with multiple threads creates much additional complexity and introduces whole new categories of dangerously frightful heisenbugs. #3 seemed like a good solution, especially since our underlying database is multiuser and doesn’t mind lots of processes banging on it at the same time. So that’s what I’m planning to do when I get back from Thanksgiving vacation.

Notice, though, the big picture. We’ve gone from read the file/save it in the database to something significantly more complicated: launch a child process, tell it to read the file and save it in the database, add a progress bar and cancel button to the child process, and then some kind of mechanism so the child can notify the parent when the file has arrived so it can be displayed. There will also be some work passing command line arguments to the child process, and making sure the window focus behaves in an expected manner, and handling the case of the user shutting down their system while a file copy is in progress. I would guesstimate that when all is said and done I’ll have ten times as much code to handle large files gracefully, code that maybe 1% of our users will ever see.

And of course, a certain type of programmer will argue that my new child-process architecture is inferior to the original. It’s “bloated” (because of all the extra lines of code). It has more potential for bugs, because of all the extra lines of code. It’s overkill. It’s somehow emblematic of why Windows is an inferior operating system, they will say. What’s all this about progress indicators? they sneer. Just hit Ctrl+Z and then “ls -l” repeatedly and watch to see if the file size is growing!

A picture of the new door

The moral of the story is sometimes fixing a 1% defect takes 500% effort. This is not unique to software, no sirree, now that I’m managing all these construction projects I can tell you that. Last week, finally, our contractor finally put the finishing touches on the new Fog Creek offices. This consisted of installing shiny blue acrylic on the front doors, surrounded by aluminium trim with a screw every 20 cm. If you look closely at the picture, the aluminium trim goes all the way around each door. Where the doors meet, there are two pieces of vertical trim right next to each other. You can’t tell this from the picture, but the screws in the middle strips are almost but not exactly lined up. They are, maybe, 2 millimeters off. The carpenter working on this measured carefully, but he was installing the trim while the doors were on the ground, not mounted in place, and when the doors were mounted, “oops,” it became clear that the screws were not exactly lined up.

This is probably not that uncommon; there are lots of screws in our office that don’t line up perfectly. The problem is that fixing this once the holes are drilled would be ridiculously expensive. Since the correct placement for the screws is only a couple of millimeters away, you can’t just drill new holes in the door; you’d probably have to replace the whole door. It’s just not worth it. Another case where fixing a 1% defect takes 500% effort, and it explains why so many artifacts in our world are 99% good, not 100% good. (Our architect never stops raving about some really, really expensive house in Arizona where every screw lined up.)

It comes down to an attribute of software that most people think of as craftsmanship. When software is built by a true craftsman, all the screws line up. When you do something rare, the application behaves intelligently. More effort went into getting rare cases exactly right than getting the main code working. Even if it took an extra 500% effort to handle 1% of the cases.

Craftsmanship is, of course, incredibly expensive. The only way you can afford it is when you are developing software for a mass audience. Sorry, but internal HR applications developed at insurance companies are never going to reach this level of craftsmanship because there simply aren’t enough users to spread the extra cost out. For a shrinkwrapped software company, though, this level of craftsmanship is precisely what delights users and provides longstanding competitive advantage, so I’ll take the time and do it right. Bear with me.

CityDesk Entity Classes

Comment: This article is Fog Creek Internal documentation, likely to be of interest only to hard core VB 6.0 geeks working with databases through DAO. Rather than use the DAO classes directly and scattering SQL syntax throughout our code, we try to isolate database access by using a thin layer which provides simple objects that map to underlying data. These objects are called “entities.”

About 99% of what we do with databases consists of:

  • select one record
  • enumerate records, possibly matching a query
  • change a record
  • add a new record

The goal of entity classes is to make these things easy and consistent.

To isolate and simplify all access to the CityDesk database, all database access code goes through a set of classes called entity classes. These are VB user-defined classes, which are easily identified because the class names always start with the capital letter E followed by the name of the table. For example, you would use an entity class called EArticle to access the table tblArticle. There does not have to be an underlying physical table for each entity class. If you need to do a sophisticated query that joins multiple tables, this can be done inside an entity which still starts with E and has some sensible name but which does not correspond to a physical table.

Golden Rule: Never let entity classes cause you to do something inefficient with the database.

Comment: A lot of times people using the entity classes forget about what’s involved in the underlying query, and end up writing code that’s ridiculously inefficient. For example, updating a table by looping through it and committing each change, rather than using a SQL UPDATE statement. This is explicitly forbidden.

Each entity class has public members for each of the columns in the table with the exact same names. For example, for tblArticle, EArticle has ixArticle, ixSet, ixLanguage, etc. These are not implement as property get/lets, that just makes the code ugly; they are implemented as plain members. There is ONE EXCEPTION: any large or blob type fields are implemented as property get/lets which defer the actual database access until you access that particular field.

Comment: CityDesk has a lot of tables containing a large blob field, for example, the table of articles. If you’re enumerating the articles in order to list them, you don’t want to spend any cycles extracting the blobs.

When you create a new entity object, name it with the same name as the table in lower case, for example:

Dim article As EArticle : Set article = New EArticle

Creating a new entity object initializes all the fields to something sensible and initializes the primary key ix field (e.g. ixArticle) to -1.

Entity objects usually have these methods:

.Load(ix)

Loads the object from a single record, where the primary key is ix.

Dim article As EArticle : Set article = New EArticle
article.Load 1 ‘ load article where ixArticle = 1
Debug.Print article.ixLanguage

.Save

Commits any changes. If the primary key is still -1, this does an INSERT and always returns the just inserted primary key as a return value. If the primary key is not -1, this does an Update…Where and still returns the primary key.

‘ Add new article:
Dim article As EArticle : Set article = New EArticle
article.ixLanguage = 1
‘ set all the other fields
Dim ixArticleNew: ixArticleNew = article.Save

‘ Now edit it:
article.Load ixArticleNew
article.ixLanguage = 2
article.Save

Comment: We always use autonumber/identity fields for primary keys. One of the most maddening things about SQL is the numerous, inconsistent and unreliable ways of getting the identity field you just inserted.

.List
.GetNext

Used for enumerating through the records. The plain version of List starts enumerating them all. If there are zero matching records, that is, if EOF is true immediately, List returns False, in case you want to have different code for that case.

If you need to return any other kind of list, for example, all non-deleted, you use a variation of List like ListNonDeleted, which can take arguments.

Then .GetNext loads the first record and returns true if not EOF.

‘ List templates:
Dim template As ETemplate: Set template = New ETemplate
If template.List Then
    While template.GetNext
         Debug.Print template.Name
         ‘ No need to move next!
    Wend
Else
    Debug.Print “Nothing!”
End If

Comment: A common mistake in writing DAO code is forgetting the MoveNext at the bottom of your loop. This eliminates the need for it.

 

Worst Project Ever?

Hi Joel,

I’m sure that this is perplexing. As Napster’s vice president of engineering and CTO during much of this time, please let me share an alternative perspective. Building a paid music site based on peer-to-peer technology involved more work than you might think at first glance. We had to:

  1. Integrate a user management and billing system.
  2. Build new Windows and Mac client applications that handle the new system, plus add significant improvements to usability and reliability, including better search tools.
  3. Create a system that identifies every MP3 file our users share with 100% accuracy, regardless of the names given to the files. Determine with 100% reliability if any of these files is on a black list of disallowed files, provided by recording companies and music publishers in many formats and with varying degrees of accuracy. This is not even theoretically possible, but major strides were made with the help of Relatable’s acoustic fingerprinting technology and a large amount of “expert system” analysis code developed in-house.
  4. Build a payment system to compensate the various rights-holders of the media content that understands fully the vagaries of the music royalty systems and the concomitant dispute resolution processes they involve.
  5. Build a system that would enable owners of music to automatically submit both content and ownership and royalty rates to the system. Digitally encode, fingerprint, and catalog all submitted content. Seed the system with the licensed content.
  6. Build a Digital Rights Management (DRM) system from the ground up that allows controlled usage and secure distribution of content over a peer-to-peer network, including content that was introduced by users.
  7. Build a web-based user registration system.

All of this was accomplished and ready to go. I think you would be very surprised at the tiny size of the “whole team of developers” that was involved. Versions of the system were in beta test from December 2001. The only holdup to full deployment was the record companies’ and publishers’ refusal to license their content to Napster.

Please keep in mind that simultaneous to much of this work being done, significant development of the existing Napster system was required to be made under court order. Most of the engineering management and many of the key Napster developers were involved in this work. I was also directly involved in the legal process, requiring me to spend massive amounts of time doing legal work, including briefs, declarations, depositions, and very frequent meetings with the RIAA’s and the Court’s Technical Experts.

As an aside, I hope you don’t think that the “3 months” Shawn took to write the original version (not true anyway) was the reliable system that scaled to handle 2 million simultaneous users! Although the Windows client was largely Shawn’s, the server code was completely rewritten.

Having managed many large-scale engineering projects, I am well aware of the second-version syndrome. But in this case, we were really building a new first version.

Best,

Eddie Kessler

Product Vision

Editor’s Note: As I constantly reiterate on these pages, good design requires making often difficult choices between imperfect options. Making those decisions consistently requires a touchstone — some kind of guiding principle — often called a vision statement. For my company’s product CityDesk, for example, a part of the vision was “content management when you can’t run code on your web server.” That meant that features which required code on the server, like membership or discussion boards, were left out.

Last week at the Cutter Summit Jim Highsmith described a useful way to figure out your vision statement, which he calls Design the Box. Here’s his article, which originally appeared in the 23 August 2001 issue of Cutter Consortium’s Agile Project Management E-Mail Advisor and was reprinted with permission. Register here for a free, four-week trial to the E-mail Advisor. – Joel Spolsky


Picture of Jim HighsmithA sample product vision statement:

“For a mid-sized company’s marketing and sales departments who need basic CRM functionality, the CRM-Innovator is a Web-based service that provides sales tracking, lead generation, and sales representative support features that improve customer relationships at critical touch points. Unlike other services or package software products, our product provides very capable services at a moderate cost.”

This product vision model helps team members pass the elevator test — the ability to explain the project to someone within two minutes. It comes from Geoffrey Moore’s book Crossing the Chasm. It follows the form:

  • For (target customer)
  • Who (statement of the need or opportunity)
  • The (product name) is a (product category)
  • That (key benefit, compelling reason to buy)
  • Unlike (primary competitive alternative)
  • Our product (statement of primary differentiation)

For any project, but particularly those with high uncertainty for which significant requirements changes are anticipated, creating a product vision statement helps teams remain focused on the critical aspects of the product, even when details are changing rapidly. It is very easy to get focused on the short-term issues associated with a 2-4 week development iteration and lose track of the overall product vision.

Even within an IT organization, I think every project should be considered to produce a “product.” Whether the project results involve enhancements to an internal accounting system or a new e-commerce site, product-oriented thinking pays back benefits.

One practice that I’ve found effective in getting teams to think about a product vision is the Design-the-Box exercise (developed originally by colleague Bill Shakelford). This exercise is great to open up a session to initiate a project. In this exercise the entire team, including users, breaks up into groups of four to six (this works best with cross-functional participation). The team makes the assumption that the product will be sold in a shrink-wrapped box, and their task is to design the product box front and back. This involves coming up with a product name, a graphic, three to four key bullet points on the front to “sell” the product, a detailed feature description on the back, and operating requirements.

Coming up with 15 or 20 product features proves to be easy. It’s figuring out which 3 or 4 would cause someone to buy the product that is difficult. One thing that usually happens is an intense discussion about who the customers really are. Using a very simple product example, it always amazes me that the product vision boxes end up so disparate among three to five teams. Presentations by each of the groups are then followed by discussing how the multiple focal points can be reduced to a few that everyone agrees upon. A lot of good information gets generated by this exercise — and it’s fun. Once this exercise has been completed, the group can then work on constructing their own version of Moore’s product vision statement. Finally, with several hours of active discussion recorded on flip-chart paper, the group can construct a good outline for a complete one- to three-page product vision document that might include: the mission statement, a project profile overview (scope, schedule, cost, defects) with priorities, pictures of the “boxes,” target customers and each of their needs, customer satisfaction measures, key technology and operational requirements, critical product constraints (performance, ease of use, volumes), and key financial indicators.

For a four- to six-month project, this visioning exercise might take from a half-day to a full day, but it will pay big dividends. The more critical the delivery schedule and the more volatile the project, the more important it is that the team have a good vision of the final desired outcome. Minus this vision, iterative development projects are likely to become oscillating projects — going round and round in circles because everyone is looking at the minutia rather than the big picture.

Jim Highsmith is the Practice Director, Agile Project Management,  Cutter Consortium.

If you’d like to comment on today’s Agile Project Management E-Mail Advisor, send e-mail to apm@cutter.com, or send a letter by fax to +1 781 648 8707 or by mail to The Agile Project Management E-Mail Advisor, Cutter Consortium, 37 Broadway, Suite 1, Arlington, MA 02474-5552, USA.

© Copyright 2001 Cutter Consortium. To sample Cutter’s other weekly e-mail advisors, register here.