Posts by Joel Spolsky

In 2000 I co-founded Fog Creek Software, where we created lots of cool things like the FogBugz bug tracker, Trello, and Glitch. I also worked with Jeff Atwood to create Stack Overflow and served as CEO of Stack Overflow from 2010-2019. Today I serve as the chairman of the board for Stack Overflow, Glitch, and HASH.

Kinda a big announcement

The other day I was talking to a young developer working on a code base with tons of COM code, and I told him that even before he was born, everyone knew that COM was already so deeply obsolete that it was impossible to find anyone who knew enough to work on it. And yet they still have this old COM code base, and they still have one old programmer holding onto their job by being the only human left on the planet with a brain big enough to manually manage multithreaded objects. I remember that COM was like Gödels Theorem: it seemed important, and you could understand it all long enough to pass an exam, but ultimately it is mostly just a demonstration of how far human intelligence can be made to stretch under extreme duress.

And, bubbeleh, if there is one thing we have learned, it’s that the things that make it easier on your brain are the things that matter.

Programming changes slowly. Really slowly.

Since I learned to code forty years ago, one thing that has mostly, mostly, changed about programming is that most developers no longer have to manage their own memory. Even getting that going took a long long time.

I took a few stupid years trying to be the CEO of a growing company during which I didn’t have time to code, and when I came back to web programming, after a break of about 10 years, I found Node, React, and other goodies, which are, don’t get me wrong, amazing? Really really great? But I also found that it took approximately the same amount of work to make a CRUD web app as it always has, and that there were some things (like handing a file upload, or centering) that were, shockingly, still just as randomly difficult as they were in VBScript twenty years ago.

Where are the flying cars?

The biggest problem is that developers of programming tools love to add things and hate to take things away. So things get harder and harder and more and more complex because there are more and more ways to do the same thing, each has pros and cons, and you are likely to spend as much time just figuring out which “rich text editor” to use as you are to implement it.

(Bill Gates, 1990: “How many f*cking programmers in this company are working on rich text editors?!”)

So, in this world of slow, gradual change and alleged improvement, one thing did change literally overnight, or, to be precise, on September 15, 2008, which was when Stack Overflow launched.

Six-to-eight weeks before that, Stack Overflow was only an idea. (*Actually Jeff started development in April). Six-to-eight weeks after that, it was a standard part of every developer’s toolkit: something they used every day. Something had changed about programming, and changed very fast: the way developers learned and got help and taught each other.

For many years, I was able to coast by telling delightful little stories about our incredible growth numbers, about the pay web site we made obsolete, and even about that one time when a Major Computer Book Publisher threatened to BURY US and launched their own Q&A platform, which turned out to be more of a Scoff Generator than a Q&A platform, but actually now almost anyone I talk to is too young to imagine The Days Before Stack Overflow, when the bookstore had an entire wall of Java and the way you picked a Rich Text Editor was going to Barnes and Noble and browsing through printed books for an hour, in the Rich Text Editor Component shelf.

Stack Overflow got to be pretty big as a business. The company grew faster than any individual’s skills at managing companies, especially mine, so a lot of the business team has changed over, and we now have a really world-class, experienced team that is doing much better than us founders. We’ve done incredible work building a recruiting platform for great developers, a “reach and relevance” platform for getting developers excited about your products, and, most importantly, Stack Overflow for Teams, which is growing so quickly that soon every developer in the world will be using the power of Stack Overflow to get help with their own code base, not just common languages and libraries.

And yeah, the one thing I made sure of was that everyone that came into the company understood exactly why Stack Overflow works, and what is important to the developers that it is by and for. So while we haven’t always been perfect, we have kept true to our mission, and the current leadership is just as committed to the vision of Stack Overflow as the founders are.

Today we’re pleased to announce that Stack Overflow is joining Prosus. Prosus is an investment and holding company, which means that the most important part of this announcement is that Stack Overflow will continue to operate independently, with the exact same team in place that has been operating it, according to the exact same plan and the exact same business practices. Don’t expect to see major changes or awkward “synergies”. The business of Stack Overflow will continue to focus on Reach and Relevance, and Stack Overflow for Teams. The entire company is staying in place: we just have different owners now.

This is, in some ways, the best possible outcome. Stack Overflow stays independent. The company has plenty of cash on hand to expand and deliver more features and fix the old broken ones. Right now, the biggest gating factor to how fast we can do this is just how fast we can hire excellent people.

I’ve been out of the day-to-day for a while now. Together with David Wilkinson, I’m helping to build HASH. HASH makes it easy to build powerful simulations and make better decisions. As we worked on that, we discovered that too much of the data that you might need to run simulations needs to be fixed up before you can use it. That’s because data is often published on the web, using page description languages that are more concerned with formatting and consumption by humans. They lack the structure to make the data they contain readily accessed programatically, so step one is miserable screen scraping and data cleanup. That’s where a lot of people give up.

We think we have an interesting way to fix this. If it works, we’ll change the web as quickly and completely as Stack Overflow changed programming. But it’s kind of ambitious and maybe a little too GRAND. If you are interested in joining that crazy journey, do reach out. The whole thing is going to be open source, so just hang on, and we’ll have something up on GitHub for you to play with.

See you soon!

HASH: a free, online platform for modeling the world

Sometimes when you’re trying to figure out the way the world works, basic math is enough to get you going. If we increase the hot water flow by x, the temperature of the mixture goes up by y.

Sometimes you’re working on something that’s just too complicated for that, and you can’t even begin to guess how the inputs affect the outputs. At the warehouse, everything seems to go fine when you have less than four employees, but when you hit five employees, they get in each others’ way so much that the fifth employee effectively does no additional work.

You may not understand the relationship between the number of employees and the throughput of the warehouse, but you definitely know what everybody is doing. If you can imagine writing a little bit of JavaScript code to simulate the behavior of each of your workers, you can run a simulation and see what actually happens. You can tweak the parameters and the rules the employees follow to see how it would help, and you can really gain some traction understanding, and then solving, very complex problems.

That’s what hash.ai is all about. Read David’s launch blog post, then try building your own simulations!

So, how’s that retirement thing going, anyway?

For the last couple of months, Prashanth Chandrasekar has been getting settled in as the new CEO of Stack Overflow. I’m still going on some customer calls and have a weekly meeting with him, but I have freed up a lot of time. I’m also really enjoying discovering just how little I knew about running medium-sized companies, as I watch Prashanth rearrange everything—for the better. It’s really satisfying to realize that the best possible outcome for me is if he proves what a bad CEO I was by doing a much better job running the company.

Even though I live in Manhattan’s premier NORC (“Naturally Occurring Retirement Community,”) I’m thinking of this time as a sabbatical, not retirement. And in fact I’m really, really busy, and, in the interest of deflecting a million questions about what I’m doing nowadays, thought I’d update my long-suffering readers here.

This adorable little fella, Cooper, is two. If your web app needs a mascot, apply within.

I’m chairman of three companies. You probably know all about Stack Overflow so I’ll skip ahead.

Fog Creek Software has been renamed Glitch, “the friendly community for building the web.” Under CEO Anil Dash, they have grown to millions of apps and raised a decent round of money to accelerate that growth. I think that in every era there has to be some kind of simplified programming environment for the quiet majority of developers who don’t need fancy administration features for their code, like git branches or multistep deployment processes; they just want to write code and have it run. Glitch is aimed at those developers.

The third company, HASH, is still kind of under the radar right now, although today they put a whole bunch of words up on their website so I guess I can give you a preview. HASH is building an open source platform for doing simulations. It’s a great way to model problems where you have some idea of how every agent is supposed to behave, but you don’t really know what all that is going to add up to.

For example, suppose you’re a city planner and you want to model traffic so that you can make a case for a new bus line. You can, sort of, pretend that every bus takes 50 cars off the road, but that’s not going to work unless you can find 50 commuters who will all decide to take your new bus line… and the way they decide is that they check if the bus is actually going to save them time and money over just driving. This is a case where you can actually simulate the behavior of every “agent” in your model, like Cities: Skylines does, and figure out the results. Then you can try thousands or millions of different potential bus routes and see which ones actually reduce traffic.

This kind of modeling is incredibly computationally intensive, but it works even when you don’t have a closed-form formula for how bus lines impact traffic, or, in general, how individual agents’ behavior affects overall outcomes. This kind of tool will be incredibly useful in far-ranging problems, like epidemiology, econometrics, urban planning, finance, political science, and a lot of other areas which are not really amenable to closed-form modeling or common “AI” techniques. (I love putting AI in “scare” “quotes”. There are a lot of startups out there trying to train machine learning models with way too little data. Sometimes the models they create just reproduce the bad decision making of the humans they are trained on. In many cases a model with simulated agents running a white box algorithm is going to be superior).

Ok, so those are the three companies I’m still working on in some way or another. That still leaves me with a couple of free days every week which I’m actually using to work on some electronics projects.

In particular, I’m really into pixel-addressable RGB LEDs, like those WS2812b and APA102-type things. Right now I’m working on designing a circuit board that connects a Teensy 3.2 controller, which can drive up to 4416 LEDs at a high frame rate, to a WizNET Ethernet adapter, and then creating some software which can be used to distribute 4416 pixels worth of data to each Teensy over a TCP-IP network in hopes of creating huge installations with hundreds of thousands of pixels. If that made any sense at all, you’re probably already a member of the LEDs ARE AWESOME Facebook group and you probably think I’m dumb. If that doesn’t make any sense, rest assured that I am probably not going to burn down the apartment because I am very careful with the soldering iron almost every time.

Welcome, Prashanth!

Last March, I shared that we were starting to look for a new CEO for Stack Overflow. We were looking for that rare combination of someone who could foster the community while accelerating the growth of our businesses, especially Teams, where we are starting to close many huge deals and becoming a hyper-growth enterprise software company very quickly. This is not something I’m particularly good at, and I thought it was time to bring on more experienced leadership.

The Board of Directors nominated a search committee and we went through almost 200 candidates. It speaks to how well respected  a company Stack Overflow is that we found ourselves in the rare position of having plenty of highly qualified executives who were excited about the opportunity. Nevertheless, one of them really stood out, and we are pleased to let you know that we have selected Prashanth Chandrasekar as our next CEO. His first day will be October 1st.

Prashanth was born in Bangalore, India, the city with the highest number of Stack Overflow users in the world, one of the global capitals for software developers writing the script for the future. He started out as a software engineer before moving over to management. He has a BS in Computer Engineering from the University of Maine, a Masters in Engineering Management from Cornell, and an MBA from Harvard. He worked at Capgemini as a management consultant and Barclays as an investment banker in their technology group before joining Rackspace in San Antonio, Texas.

At Rackspace, Prashanth really proved his mettle, creating from scratch a completely new business unit inside the company, the Global Managed Public Clouds Business. This group serves companies around the world who need help running on AWS, Azure, Google, and so on. Under his leadership, Rackspace successfully pivoted from a leading managed hosting company to a cloud services company. And he did this while working with developers both inside Rackspace and outside, so he understands our vision of “writing the script for the future” better than anyone I’ve met.

This is an exciting time for Stack Overflow, and we have some big goals for the year ahead. We want to make Stack Overflow more diverse, inclusive, and welcoming. And we want to make it possible for knowledge workers everywhere to use Stack Overflow to get answers to the proprietary questions that are specific to their organizations and teams. We’re doing great work and making great progress in these areas, and I’m confident that Prashanth has some great ideas about how to move forward faster on all our goals.

As you know, I’m keeping my job as Chairman of the Board, so I’ll continue to be closely involved. Being Stack Overflow’s CEO has been an honor, and I can’t wait to see the things the team accomplishes in the year ahead. This will be a great new chapter for Stack Overflow.

The next CEO of Stack Overflow

Big news! We’re looking for a new CEO for Stack Overflow. I’m stepping out of the day-to-day and up to the role of Chairman of the Board.

Stack Overflow has been around for more than a decade. As I look back, it’s really amazing how far it has come.  

Only six months after we had launched Stack Overflow, my co-founder Jeff Atwood and I were invited to speak at a Microsoft conference for developers in Las Vegas. We were there, I think, to demonstrate that you could use their latest ASP.NET MVC technology on a real website without too much of a disaster. (In fact .NET has been a huge, unmitigated success for us, but you kids go ahead and have fun with whatever platform you want mkay? They’re all great, or, at least, above-average).

It was a giant conference, held at the Venetian Hotel. This hotel was so big that other hotels stay there when they go on vacation. The main ballroom was the size of, approximately, Ireland. I later learned there were 5,000 developers in that room.

I thought it would be a fun thing to ask the developers in the room how many of them had visited Stack Overflow. As I remember, Jeff was very much against this idea. “Joel,” he said, “That is going to be embarrassing and humiliating. Nobody is going to raise their hand.”

Well, I asked it anyway. And we were both surprised to see about one-third of the hands go up. We were really making an impact! That felt really good.

Anyway, I tried that trick again whenever I spoke to a large audience. It doesn’t work anymore. Today, audiences just laugh. It’s like asking, “Does anyone use gravity? Raise your hand if you use gravity.”

Where are we at after 11 years? Practically every developer in the world uses Stack Overflow. Including the Stack Exchange network of 174 sites, we have over 100 million monthly visitors. Every month, over 125,000 wonderful people write answers. According to Alexa, stackoverflow.com is one of the top 50 websites in the world. (That’s without even counting the Stack Exchange network, which is almost as big.) And every time I see a developer write code, they’ve got Stack Overflow open in one of their browser windows. Oh and—hey!—we do not make you sign up or pay to see the answers.

The company has been growing, too. Today we are profitable. We have almost 300 amazing employees worldwide and booked $70m in revenue last year. We have talent, advertising, and software products. The SaaS products (Stack Overflow for Teams and Enterprise) are growing at 200% a year. That speaks to the fact that we’ve recruited an incredibly talented team that has produced such fantastic results.

But, we have a lot of work ahead of us, and it’s going to take a different type of leader to get us through that work.

The type of people Stack Overflow serves has changed, and now, as a part of the developer ecosystem, we have a responsibility to create an online community that is far more diverse, inclusive, and welcoming of newcomers.

In the decade or so since Stack Overflow started, the number of people employed as software developers grew by 64% in the US alone. The field is going to keep growing everywhere in the world, and the demand for great software developers far outstrips supply. So a big challenge for Stack Overflow is welcoming those new developers into the fold. As I’ve written:

One thing I’m very concerned about, as we try to educate the next generation of developers, and, importantly, get more diversity and inclusiveness in that new generation, is what obstacles we’re putting up for people as they try to learn programming. In many ways Stack Overflow’s specific rules for what is permitted and what is not are obstacles, but an even bigger problem is rudeness, snark, or condescension that newcomers often see.

I care a lot about this. Being a developer gives you an unparalleled opportunity to write the script for the future. All the flak that Stack Overflow throws in the face of newbies trying to become developers is actively harmful to people, to society, and to Stack Overflow itself, by driving away potential future contributors. And programming is hard enough; we should see our mission as making it easier.

The world has started taking a closer look at tech, and understanding that software and the internet are not just tools; they are shaping the future of society. Big tech companies are struggling with their place in the world. Stack Overflow is situated at the right place to be influential in how that future develops, and that is going to take a new type of leader.

new dog, too

It will not be easy to find a CEO who is the right person to lead that mission. We will, no doubt, hire one of those fancy executive headhunters to help us in the search. But, hey, this is Stack Overflow. If there’s one thing I have learned by now, it’s that there’s always someone in the community who can answer the questions I can’t.

So we decided to put this announcement out there in hopes of finding great candidates that might have been under the radar. We’re especially focused on identifying candidates from under-represented groups, and making sure that every candidate we consider is deeply committed to making our company and community more welcoming, diverse, and inclusive.

Over the years, Fog Creek Software created several incredible hits and many wonderful memories along the way. It is great to watch Trello (under Michael Pryor) and Glitch (under Anil Dash) growing into enormously valuable, successful, and influential products with dedicated leaders who took these products much further than I ever could have, and personally I’m excited to see where Stack Overflow can go and turn my attention to the next thing.

Announcing Stack Overflow for Teams

Hey, we have a new thing for you today!

Today’s new thing is called Stack Overflow for Teams. It lets you set up a private place on Stack Overflow where you can ask questions that will only be visible to members of your team, company, or organization. It is a paid service, but it’s not expensive.

I meet people who use Stack Overflow every single day, but a lot of them tell me they have never needed to post their own question. “All the questions are already answered!” they say. Mission accomplished, I guess!

Still, when I think about what questions developers have every day, only the ones that have to do with public stuff can be asked on Stack Overflow. Maybe you don’t have a question about Python or Android… maybe you want to ask something about your team’s own code base!

That’s the idea behind Stack Overflow Teams.

Helicopters

Quick background: every development team since the beginning of time has been trying to figure out how to get institutional knowledge out of people’s heads and into written, searchable form where everyone can find it. Like new members of the team. And old members of the team working on new parts of the code. And people who forgot what they did three years ago and now have questions about their own code.

For a while developers thought wikis might be the solution. Anyone who has used a wiki for this purpose has probably discovered that not very much knowledge actually makes it into the wiki, and what does is not particularly useful, doesn’t get updated, and honestly it just feels like a bunch of homework to write a bunch of wiki documentation about your code when you don’t know if it will ever help anyone.

Another solution being sold today is the idea of having some kind of online IRC-style chat rooms, and hoping that by searching those chat archives, you can find “institutional knowledge.” Ha ha ha! Even if that works, all you really find is the history of some conversation people had. It might have clues but it’s not knowledge.

But you know what does work? A Q&A system. Like Stack Overflow.

Why? Because unlike wikis, you don’t write documentation in the hopes that one day it might help someone. You answer questions that are going to help someone immediately. And you can stop answering the minute you get the green checkmark that shows that you solved their problem.

And unlike chatrooms, searching actually works. It finds you a question and its answers, not a conversation-captured-in-amber.

This is why Stack Overflow worked so much better on the public internet than the previous generation of discussion forums, and we think that it will work for all the same reasons with teams’ proprietary questions and answers.

When you join a team, you’ll see your team’s private questions right on stackoverflow.com (although they actually live in a separate database for security). Your teams are listed in the left hand navbar.

Screen Shot

Everything else works pretty much … like you would expect. When you ask a question, you can direct it to your team or to the whole world. The UI makes it very clear whether you are posting publicly or privately. If you are asking a question of your team, there’s a Notify field so you can type the names of some people who might be able to answer the question, and they’ll hear about it right away.

Screen shot

When you search, you can search everywhere, or just within your team. You can set up tags that are specific to your team, too.

The pricing is designed to be “no-brainer” pricing, starting at just $10 per month for the first ten users.

I think Stack Overflow for Teams is going to be almost as important to developers’ daily work as public Stack Overflow. It brings Stack Overflow’s uniquely powerful system to every developer question, not just the things that can be discussed in public. You can stop asking your teammates questions in email (where they help nobody else) or in chatrooms (where they are impossible to find) and start building your own private knowledge base to document your code and answer future teammates’ questions before they have them.

Strange and maddening rules

There’s this popular idea among developers that when you face a problem with code, you should get out a rubber duck and explain, to the duck, exactly how your code was supposed to work, line by line, what you expected to see, what you saw instead, etc. Developers who try this report that the very act of explaining the problem in detail to an inanimate object often helps them find the solution.

Stack Overflow April Fools Joke 2018This is one of many tricks to solving programming problems on your own. Another trick is divide and conquer debugging. You can’t study a thousand lines of code to find the one bug. But you can divide them in half and quickly figure out if the problem happens in the first half or the second half. Keep doing this five or six times and you’ll pinpoint the single line of code with the problem.

It’s interesting, with this in mind, to read Jon Skeet’s checklist for writing the perfect question. One of the questions Jon asks is “Have you read the whole question to yourself carefully, to make sure it makes sense and contains enough information for someone coming to it without any of the context that you already know?” That is essentially the Rubber Duck Test. Another question is “If your question includes code, have you written it as a short but complete program?” Emphasis on the short—that is essentially a test of whether or not you tried divide and conquer.

What Jon’s checklist can do, in the best of worlds, is to help people try the things that experienced programmers may have already tried, before they ask for help.

Sadly, not everybody finds his checklist. Maybe they found it and they don’t care. They’re having an urgent problem with code; they heard that Stack Overflow could help them; and they don’t have time to read some nerd’s complicated protocol for requesting help.

One of the frequent debates about Stack Overflow is whether the site needs to be open to questions from programming novices.

When Jeff and I were talking about the initial design of Stack Overflow, I told him about this popular Usenet group for the C programming language in the 1980s. It was called comp.lang.c.

C is a simple and limited programming language. You can get a C compiler that fits in 100K. So, when you make a discussion group about C, you quickly run out of things to talk about.

Also. In the 1990s, C was a common language for undergraduates who were learning programming. And, in fact, said undergraduates would have very basic problems in C. And they might show up on comp.lang.c asking their questions.

And the old-timers on comp.lang.c were bored. So bored. Bored of the undergraduates showing up every September wondering why they can’t return a local char array from a function et cetera, et cetera, ad nauseum. Every damn September.

The old timers invented the concept of FAQs. They used them to say “please don’t ask things that have been asked before, ever, in the history of Usenet” which honestly meant that the only questions they really wanted to see were so bizarre and so esoteric that they were really enormously boring to 99% of working C programmers. The newsgroup languished because it catered only to the few people that had been there for a decade.

Jeff and I talked about this. What did we think of newbie questions?

We decided that newbies had to be welcome. Nothing was too “beginner” to be a reasonable question on Stack Overflow… as long as you did some homework before asking the question.

We understood that this might mean that some of the more advanced people might grow bored with duplicate, simple questions, and move on. We thought that was fine: Stack Overflow doesn’t have to be a lifetime commitment. You’re welcome to get bored and move on if you think that the newbies keep asking why they can’t return local char arrays (“but it works for me!”) and you would rather devote the remaining short years of your life to something more productive, like sorting your record albums.

The mere fact that you are a newbie doesn’t mean that your question doesn’t belong on Stack Overflow. To prove the point, I asked “How do you move the turtle in Logo,” hoping to leave behind evidence that the site designers wanted to allow absolute beginners.

Thanks to the law of unintended consequences, this caused a lot of brouhaha, but not because the question was too easy. The real problem there was that I was asking the question in bad faith. Jeff Atwood explained it: “Simple is fine. No effort and research is not.” (Also this.)

To novices, the long bureaucratic rigmarole associated with asking your first question on Stack Overflow can feel either completely unnecessary, or just plain weird. It’s like Burning Man. You just want to go to a nice glittery dance party in the desert, but the Burning People are yammering on about their goddamn 10 principles, and “radical self-expression” and so on and so forth, and therefore after washing your dishes you must carefully save the dirty dishwater like a cherished relic and remove every drop of it from the Playa, bringing it home with you, in your check-in luggage if necessary. Every community has lots of rules and when you join the community they either seem strange and delightful or, if you’re just desperately trying to get some code to work, they are strange and maddening.

A lot of the rules that are important to make Burning Man successful are seemingly arbitrary, but they’re still necessary. The US Bureau of Land Management which makes the desert available for Burning Man requires that no contaminated water be poured out on the ground because the clay dirt doesn’t really absorb it so well and it can introduce all kinds of disease and whatnot, but who cares because Burning Man simply will not be allowed to continue if the participants don’t pack out their used water.

Similarly for Stack Overflow. We don’t allow, say, questions that are too broad (“How do I make a program?”). Our general rule is that if the correct length of an answer is a whole book you are asking too much. These questions feel like showing up on a medical website and saying something like “I think my kidney has been hurting. How can I remove it?” It’s crazy—and incidentally, insulting to the people who spent ten years in training learning to be surgeons.

One thing I’m very concerned about, as we try to educate the next generation of developers, and, importantly, get more diversity and inclusiveness in that new generation, is what obstacles we’re putting up for people as they try to learn programming. In many ways Stack Overflow’s specific rules for what is permitted and what is not are obstacles, but an even bigger problem is rudeness, snark, or condescension that newcomers often see.

I care a lot about this. Being a developer gives you an unparalleled opportunity to write the script for the future. All the flak that Stack Overflow throws in the face of newbies trying to become developers is actively harmful to people, to society, and to Stack Overflow itself, by driving away potential future contributors. And programming is hard enough; we should see our mission as making it easier.

We’re planning a lot of work in this area for the next year. We can’t change everybody and we can’t force people to be nice. But I think we can improve some aspects of the Stack Overflow user interface to encourage better behavior, for example, we could improve the prompts we provide on the “Ask Question” page, and we could provide more tools for community moderation of comments where the snark currently runs unchecked.

We’re also working on new features that will let you direct your questions to a private, smaller group of people on your own team, which may bring some of the friendly neighborhood feel to the big city of Stack Overflow.

Even as we try to make Stack Overflow more friendly, our primary consideration at Stack Overflow has been to build the world’s greatest resource for software developers. The average programmer, in the world, has been helped by Stack Overflow 340 times. That’s the real end-game here. There are other resources for learning to program and getting help, but there’s only one site in the world that developers trust this much, and that is worth preserving—the programming equivalent to the Library of Congress.

A Dusting of Gamification

[This is the second in a series of posts about Stack Overflow. The first one is The Stack Overflow Age.]

Around 2010 the success of Stack Overflow had led us into some conversations with VCs, who wanted to invest.

at the Getty

The firm that eventually invested in us, Union Square Ventures, told us that they were so excited by the power of gamification that they were only investing in companies that incorporated some kind of game play.

For example, Foursquare. Remember Foursquare? It was all about making your normal post-NYU life of going to ramen noodle places and dive bars into a fun game that incidentally generated wads of data for marketers. Or Duolingo, which is a fun app with flash cards that teaches you foreign languages. Those were other USV investments from that time period.

At the time, I had to think for a minute to realize that Stack Overflow has “gamification” too. Not a ton. Maybe a dusting of gamification, most of it around reputation.

Stack Overflow reputation started as a very simple score. The original idea was just that you would get 10 points when your answers were upvoted. Upvotes do two things. They get the most useful answers to the top, signaling that other developers who saw this answer thought it was good. They also send the person who wrote the answer a real signal that their efforts helped someone. This can be incredibly motivating.

You would lose points if your questions were downvoted, but you actually only lose 2 points. We didn’t want to punish you so much as we wanted to show other people that your answer was wrong. And to avoid abuse, we actually make you pay one reputation point to downvote somebody, so you better really mean it. That was pretty much the whole system.

Now, this wasn’t an original idea. It was originally inspired by Reddit Karma, which started out as an integer that appeared in parentheses after your handle. If you posted something that got upvoted, your karma went up as a “reward.” That was it. Karma didn’t do a single thing but still served as a system for reward and punishment.

What reputation and karma do is send a message that this is a community with norms, it’s not just a place to type words onto the internet. (That would be 4chan.) We don’t really exist for the purpose of letting you exercise your freedom of speech. You can get your freedom of speech somewhere else. Our goal is to get the best answers to questions. All the voting makes it clear that we have standards, that some posts are better than others, and that the community itself has some norms about what’s good and bad that they express through the vote.

It’s not a perfect system (more on the problems in a minute), but it’s a reasonable first approximation.

By the way, Alexis Ohanian and Steve Huffman, the creators of Reddit, were themselves inspired by a more primitive karma system, on Slashdot. This system had real-world implications. You didn’t get karma so that other people could see your karma; you got karma so that the system knew you weren’t a spammer. If a lot of your posts had been flagged for abuse, your karma would go down and you might lose posting or moderation privileges. But you weren’t really supposed to show off your high karma. “Don’t worry too much about it; it’s just an integer in a database,” Slashdot told us.

To be honest, it was initially surprising to me that you could just print a number after people’s handles and they would feel rewarded. Look at me! Look at my four digit number! But it does drive a tremendous amount of good behavior. Even people who aren’t participating in the system (by working to earn reputation) buy into it (e.g., by respecting high-reputation users for their demonstrated knowledge and helpfulness).

But there’s still something of a mystery here, which is why earning “magic internet points” is appealing to anyone.

I think the answer is that it’s nice to know that you’ve made a difference. You toil away in the hot kitchen all day and when you serve dinner it’s nice to hear a compliment or two. If somebody compliments you on the extra effort you put into making radish roses, you’re going to be very happy.

This is a part of a greater human need: to make an impact on the world, and to know that you’re contributing and being appreciated for it. Stack Overflow’s reputation system serves to recognize that you’re a human being and we are super thankful for your contribution.

in Utah

That said, there is a dark side to gamification. It’s not 100% awesome.

The first problem we noticed is that it’s very nice to get an upvote, but getting a downvote feels like a slap in the face. Especially if you don’t understand why you got the downvote, or if you don’t agree. Stack Overflow’s voting has made many people unhappy over the years, and there are probably loads of people who felt unwelcome and who don’t participate in Stack Overflow as a result. (Here’s an old blog post explaining why we didn’t just eliminate downvotes).

There’s another problem, which is that, to the extent that the gamification in Stack Overflow makes the site feel less inclusive and welcoming to many people, it is disproportionately off-putting to programmers from underrepresented groups. While Stack Overflow does have many amazing, high reputation contributors who are women or minorities, we’ve also heard from too many who were apprehensive about participating.

These are big problems. There’s a lot more we can and will say about that over the next few months, and we’ve got a lot of work ahead of us trying to make Stack Overflow a more inclusive and diverse place so we can improve the important service that it provides to developers everywhere.

Gamification can shape behavior. It can guide you to do certain things in certain ways, and it can encourage certain behaviors. But it’s a very weak force. You can’t do that much with gamification. You certainly can’t get people to do something that they’re not interested in doing, anyway. I’ve heard a lot of crazy business plans that are pinning rather too high hopes on gamification as a way of getting people to go along with some crazy scheme that the people won’t want to go along with. Nobody’s going to learn French just to get the Duolingo points. But if you are learning French, and you are using Duolingo, you might make an effort to open the app every day just to keep your streak going.

I’ve got more posts coming! The next one will be about the obsessive way Stack Overflow was designed for the artifact, in other words, we optimized everything to create amazing results for developers with problems arriving from Google, not just to answer questions that people typed into our site.

 

The Stack Overflow Age

Hi, everyone! A lot of stuff has happened since I was writing all those blog posts about Aeron chairs 18 years ago. Some of those blog posts are old enough to go to college.

And, also: Stack Overflow will be ten years old soon! Wow! So I thought it would be cool to get the old band back together for a little reunion tour over the next few weeks. I want to catch you all up on some stuff but mostly I want to tell the story of Stack Overflow in a not-completely-disorganized way. With some perspective, it’s clearer now what we did right and what we messed up, so I’ll try to cover the good and the bad over a series of blog posts.

And, also: we’re just a few weeks away from launching Stack Overflow Teams, the biggest upgrade to Stack Overflow ever, so that’s going to be really cool. I’ll get to that in a future blog post!

Today is chapter one. I want to talk a little bit about what it was like for developers before Stack Overflow, the problem that Stack Overflow tried to solve, and early origins.

In the early days of the Internet, before the Web, there was a system called Usenet which created primitive online discussion forums. When programmers had problems with their code, they could ask a question on a Usenet forum. (They were technically called newsgroups, not forums (even though they had nothing to do with news. (You couldn’t even get news on Usenet.)))

As soon as the world wide web became a thing, Usenet was immediately technically obsolete. We programmers started asking about our problems on various web-based forums, of which there were thousands.

One of the biggest such forums was called Experts Exchange. The first version of Experts Exchange was not successful financially. Apparently they went bankrupt in 2001. Eventually new owners bought the assets and resurrected the site with a clever business model: charging money to read answers.

This actually fixed the business, which started making money, but it caused some problems.

The first problem was that programmers with problems would search on Google, not on Experts Exchange. And Google only knows about free, open websites, not websites that you have to pay to access. So EE did a bamboozle: when the Google Robot came by, they showed it the full question and its answers. But when regular people went to the same page, they saw the answers were scrambled, with instructions to pay (I think it was about $250 a year) to see the results. Most programmers couldn’t be bothered.

The second problem was that EE let you get a free membership if you answered a certain number of questions. As it turned out, the people who were most desperate for free memberships were not exactly the best programmers in the world, and they wrote low quality answers to questions just to get those free memberships. And the quality of answers on the site went down.

For a long time (at least five years, I think) programmers would constantly come across EE in the Google search results, try to click on them, discover that it was a pay site, grumble, and just go back to Google and try to find an answer for free.

And I kept thinking, how hard is it to run a discussion forum on the Internet? For fudge sake, I had written one in Visual Basic in a weekend. (Not kidding, actually. Yeah I know that I am always saying “I could do that in a weekend in Visual Basic” when developers tell me some feature is going to take a year. This is why). So I was confident that it was only a matter of time before one of the 9,000,000 smart programmers in the world decided to route around this EE damage and make a free forum.

You know what? Nobody ever did. I kept waiting.

Another thing I wrote in a weekend (well, to be precise: a fortnight (shut up, I’m telling this lie)) was a job listing board for this blog. And in the first month of running that job board I think we sold about $90,000 of job listings. Huzzah! And then I thought, wow, if we smashed these ideas together—replace Experts Exchange with a free site, and pay for it with job listings—we could undo the damage to the internet and let developers get work done again.

I kept thinking “Man, this is so obvious, somebody is going to do it.”

And they never did.

And I went to one of the programmers at Fog Creek, and explained my idea, and he was like “yeah yeah sounds like a great idea, but I really like working on FogBugz.”

And more time went by.

And eventually, early in 2008, a developer/blogger named Jeff Atwood called me up, and said, “Hey Joel, I’m thinking of quitting my day job to be a Pro Blogger; you’re a blogger: what do you think?”

And I said, “Jeff, I’ve got a better idea” and I told him about the idea to combine the job listings with the Q&A site for developers, and, it took more than a weekend, but eventually I convinced him. We started talking about all the ways our Q&A site would be amazing. Jeff started working on the code in April 2008, recruited two other programmers to join him (Geoff and Jarrod, who are still here), and the three of them heroically launched what became Stack Overflow in September 2008.

 

The original Stack Overflow

 

And thus began the Stack Overflow Age.

Stack Overflow was better because it was free, but it had a ton of other “innovations” (which I put in quotes because we stole them from other Internet pioneers) which made it a much, much better site for getting answers to programming questions.

We wanted the whole thing to be a fun game, with incentives to answer questions, so we had a reputation system. The more you answer, the more reputation you earn. The reputation idea had been seen before on sites like Slashdot and Reddit.

As you earn reputation, you also earn moderation privileges on the site. So the site actually moderates itself, which is pretty cool.

Instead of putting all the Java programmers in one little forum and all the C++ programmers in another, we dumped everyone together and just let them tag their questions. This idea was stolen from flickr (remember flickr?) who, I think, stole it from del.icio.us (now gone)—who knows, anyway, the point is, tags were the new hotness and made Stack Overflow work great.

Most importantly, we realized that each question is asked by one person but the answers are seen by thousands of people who found it through a search. So we decided to optimize everything to be useful for the thousands, not the individual. We literally have 1000 visitors for every person who asks a question. That’s why we sort the answers by votes. It’s also why we optimize for questions and answers that will be helpful to other people, later.

Interestingly, when Jeff and I started Stack Overflow, we didn’t really care if it was a business and we didn’t need it to be a big profitable success. We created it because the internet sucked for programmers and we needed to make it better. We thought the job listings would pay the bills, and we’d fix the internet, and that was all we cared about and it’s what motivated us to work so hard.

Of course, it turned out a lot bigger than we thought it would. The company today has 250 employees, is profitable, and has made it possible for millions of people to learn how to code and to deal with the new, super-complicated world of APIs and frameworks that we live in. But we just wanted to fix the internet.

I have met a lot of people who started businesses because they wanted to start a business. Paul Graham calls this “Playing House.” And they didn’t really care what the business did; they just wanted to “be entrepreneurs.” Which is weird, because being an entrepreneur really sucks. It’s really hard to get through all the extraordinary difficulty, pain, and stress of starting a company if you’re not super, super motivated to solve a problem for the world.

The entrepreneurs who succeed do so because it is incredibly important to them a thing exist in the world, and it does not exist, so they work like crazy until it does. When we started Stack Overflow we didn’t expect it to be a big business; we just wanted there to be someplace where developers could get help to daily problems, while showing off how smart they were helping other developers.

Ok, that’s chapter one. I’ve got a lot more to talk about. In the next installment, I’ll talk more about how Stack Overflow’s light dusting of gamification made it really take off.

Birdcage liners

My new year’s resolution was to give up on reading Twitter and Facebook.

I gave up on the feeds because they were making me angry. A lot of times I was angry because of politics, but even on non-political things, the feeds seemed like they were full of conflict and stress.

I can’t tell you how much happier I am without them. Am I the only one that hated reading feeds? Do they make everybody unhappy? And if they make people unhappy why are they so popular?

Since I design social software for a living I feel like I should have a professional opinion on why Twitter and Facebook made me unhappy.

Let’s start with Twitter. I used Twitter to keep in touch with friends and colleagues because I cared about them. Unfortunately, those friends mostly didn’t use Twitter to share happy news and tell me how things were going. They used Twitter for bumper sticker flame wars. These were not the thoughtful long essays on blogs of yesteryear. 140 characters is too short for that.

Here’s what happened with the 140 characters. You would start out having some kind of complicated thought. “Ya know, dogs are great and all? I love dogs! But sometimes they can be a little bit too friendly. They can get excited and jump on little kids and scare the bejesus out of them. They wag their tails so hard they knock things over. (PS not Huskies! Huskies are the cats of the dog world!)”

Ok, so now you try to post that on Twitter. And you edit and edit and you finally get it down to something that fits: “Dogs can be too friendly!”

All the nuance is lost. And this is where things go wrong. “@spolsky what about huskies? #dontforgethuskies”

Ten minutes later, “Boycott @stackoverflow. @spolsky proves again that tech bros hate huskies. #shame”

By the time you get off the plane in Africa you’re on the international pariah list and your @replies are full of people accusing you of throwing puppies out of moving cars for profit.

Yeah, I get it, this 140 character limitation was just a historical accident, and now it’s 280 characters anyway, and you can always make a Twitter Story, but the flame wars on Twitter emerged from the fact that we’ve taken a medium, text, which is already bad at conveying emotion and sentiment and high-bandwidth nuance, and made it even worse, and the net result is a lot of outrage and indignation.

The outrage and indignation, of course, are what makes it work. That’s what keeps you coming back. Oooh shade. Oooh flamewar. We rubberneckers can’t keep our eyes off of it. I don’t know what the original idea of Twitter was, but it succeeded because of natural selection. In a world where the tech industry was cranking out millions of dumb little social applications, this one happens to limit messages to 140 characters and that happens to create, unintentionally, a subtlety-free indignation machine, which is addictive as heck, so this is the one that survives and thrives and becomes a huge new engine of polarization and anger. It’s not a coincidence that we got a president who came to power through bumper-sticker slogans, outrageous false statements chosen to make people’s blood boil, and of course Twitter. This is all a part of a contagious disease that is spreading like crazy because we as a society have not figured out how to fight back yet.

But Twitter is small potatoes. Facebook is where the action is. Facebook quickly copied Twitter’s idea of the “feed” as a mechanism to keep you coming back compulsively. But whereas Twitter sort of stumbled upon addictiveness through the weird 140-character limit, Facebook mixed a new, super-potent active ingredient into their feed called Machine Learning. They basically said, “look, we are not going to show everybody every post,” and they used the new Midas-style power of machine learning and set it in the direction of getting people even more hyper-addicted to the feed. The only thing the ML algorithm was told to care about was addiction, or, as they called it, engagement. They had a big ol’ growth team that was trying different experiments and a raw algorithm that was deciding what to show everybody and the only thing it cared about was getting you to come back constantly.

Now, this algorithm, accidentally, learned something interesting—something that dog trainers have always known.

Dog trainers give dogs a treat when they get something right. When they say “come,” and the dog comes, he gets a treat. Woof. I can train any arbitrary dog to do that with some reliability. But here’s what happens. Once, just once, I forget to give the dog a treat. And then the dog thinks, well, heck this, I guess “come” doesn’t always mean “treat.” So the trained behavior goes away. It’s technically called extinction: the trained behavior goes extinct.

How do we prevent extinction? By only giving treats some of the time. So the dog learns something more subtle. When my master says come and I obey, I might get a treat. Sometimes I do, sometimes I don’t. That way, if I obey and don’t get the treat, I shouldn’t panic. I should still always come when he says come because that’s still the best way to get the most treats. Intermittent reinforcement works better.

This sounds like what Facebook was doing to me.

Rather than providing a constant stream of satisfying news and engagement with friends, Facebook’s algorithm had learned to give me a bunch of junk I didn’t need to hear, and only gave me intermittent rewards through the occasional useful nugget of information about friends. Once in a blue moon I would hear about a friend’s accomplishment or I would find out that someone I like is going to be in town. The rest of the time I would just get the kind of garbage newspaper clippings circulated by someone who had too much coffee and is misattributing the kick from the caffeine to something they just read online and now MUST share IMMEDIATELY with EVERYONE because this news story about something that happened to a baby bear is SOOOOO important to THE ENTIRE WORLD. And so 9 of out 10 things in my feed are complete garbage—last week’s newspaper lining the birdcage with the droppings already on it—but then once every two weeks I find out my niece is engaged or my best friend got a great new job or my oldest friend is in town and I should make plans to hang out. And now no matter how full the Facebook feed is of bird droppings I still have to keep going back.

Both Twitter and Facebook’s selfish algorithms, optimized solely for increasing the number of hours I spend on their services, are kind of destroying civil society at the same time. Researchers also discovered that the algorithms served to divide up the world into partisan groups. So even though I was following hundreds of people on social networks, I noticed that the political pieces which I saw were nevertheless directionally aligned with my own political beliefs. But to be honest they were much… shriller. Every day the Twitter told me about something that The Other Side did that was Outrageous and Awful (or, at least, this was reported), and everyone was screeching in sync and self-organizing in a lynch mob, and I would have to click LIKE or RETWEET just to feel like I had done something about it, but I hadn’t actually done anything about it. I had just slacktivated.

What is the lesson? The lesson here is that when you design software, you create the future.

If you’re designing software for a social network, the decision to limit message lengths, or the decision to use ML to maximize engagement, will have vast social impact which is often very hard to predict.

As software developers and designers, we have a responsibility to the world to think these things through carefully and design software that makes the world better, or, at least, no worse than it started out. And when our inventions spin out of control, we have a responsibility to understand why and to try to fix them.

This blog post has a surprise piece of good news. The good news is that Facebook suddenly realized what they had done, and today they announced a pretty major change of direction. They want the feed to leave people feeling “more connected and less lonely,” so they have actually decided to sacrifice “engagement.” Mark Zuckerberg posted, “By making these changes, I expect the time people spend on Facebook and some measures of engagement will go down. But I also expect the time you do spend on Facebook will be more valuable.” That’s amazing, but it’s amazing because it demonstrates that Facebook has finally grown up and joined the rest of us in understanding that software developers are designing the future.