Building a Playground for Ideas

Is mediocrity enough?

Sometimes it feels like software engineering doesn’t matter very much. I’ve worked on projects cancelled 8 months in. The code I consider to be my best work is basically collecting dust, because it’s part of a feature that is never really used.

Meanwhile, the software tools I usually interact with are overwhelmingly poor quality, at least in my judgement (and I tend radically towards optimism). They don’t seem to be engineered according to any coherent principles, yet somehow these tools are wildly successful. People build careers around them. Conferences are devoted to them. Billions of dollars have been made.

This is evidence, I suppose, that software engineering doesn’t really matter that much. If mediocrity seems to be good enough for a product to be successful, and success in the market seems to be overwhelmingly dominated by other factors, why bother actually trying to be good at what I do?

Excellence is its own reward, of course – but I’d like to believe that excellence has some relation to market success. Otherwise, why should companies seek excellent programmers and compensate them well?

Higher-order benefits of excellence

What I’m beginning to suspect is this: I am right. The important (market) reason to care about writing excellent code is not to ship excellent products – excellent products, alas, don’t have that much market advantage over mediocre projects. The reason to write excellent code is because it increases the ability of an organization to adapt and learn.

Excellent code is general. It can be changed, re-used, applied in different circumstances. You can ask questions about it. It is an asset. If I know the relevant keywords and hit the search button in the version control system at an organization with a tradition of excellent code, chances are good that, if the problem I’m working on has been solved before by a colleague, I won’t have to solve it again myself a different way. Even if the code I found was never shipped, or belongs to a product that nobody uses.

Mediocre code is rigid, it cannot be adapted. It cannot be understood, and it is a liability. Chances are, if I hit the search button in the VCS of an organization with a tradition of mediocre code, I’m not going to get any closer to finding a solution to my problem – although I might get closer to flipping my desk over in rage.

This is why I think once a software company reaches a certain point of maturity, engineering needs to take priority over product. This seems perverse. After all, isn’t the whole point of engineering to build products for customers?

Ultimately, yes. But it doesn’t follow that the whole point of engineering is to build the very specific set of products that the organization is interested in building right now. The product roadmap is subject to the whimsy of the market, and the products you are building today are probably mostly a mistake.

The Playground for Ideas

You don’t just want your engineers to build products. You want them to build you a platform for many products, a playground for ideas, so you can experiment with many things, learn and adapt faster.

As engineers, we need to stop thinking that the most important thing we do is ship our product roadmap. The most important thing we do is building the playground for ideas. That means

• Breaking down communication silos. Build forums for engineers to learn from each other.
• Building tools, so your colleagues (and not just the engineers) can get feedback on their ideas faster. Document what you learn. Index the collective knowledge.
• Keeping code consistent and adaptable. If a good idea comes along, but all of the code is too rigid to be adapted to take advantage of it, what’s the point?

Again, I imagine this applies only to companies that have reached a level of maturity. If you are an early-stage startup, you will piggyback off of external playgrounds for ideas. Open-source software communities, communities of entrepreneurs, and whatnot.

“Technical Debt” is a bankrupt metaphor

“Product debt” would be better. Naïvely build 10 products with 10 features each, and you have earned yourself 100 features’ worth of maintenance and operations. It is the product that put you into debt, not the technical decisions.

But if you take the time to engineer things well, there will be code re-use, abstraction, observability and the like. Because you have built a great foundation of tools, the next product’s time-to-market will be smaller, and you will be able to get better feedback about it faster. It should be called “technical investment”. Technical investment is how you pay back product debt.