W-Shadow.com

Today I was going to post a new WordPress plugin. I had the screenshots prepared, the .zip archive uploaded, and a halfway decent headline already in mind. The customary anxiousness of being faced with a blank text field was also setting in, and with it the usual doubts – “Is it worth it? Does anyone actually need this plugin? Should I write an outline first? Maybe put this off until tomorrow, or, say, next February?” (Generously sprinkled with mental screams of anguish and metaphorical wringing of hands, all yielding nothing but a confused stare from my dog.)

And thus, as I was pacing back and forth in a dimly lit bedroom and engaging in the aforementioned internal monologue, I happened upon a realization that, I think, will make a much better post topic than here’s-yet-another-damn-plugin. Specifically, I realized that the plugin I had been building and was about to release was one of a particular breed, of a certain pattern. A pattern that presents a recurring temptation to every developer. It can be concisely described with a single word: meta.

For the purposes of this post, meta is where instead of building a better X, you build something that oversees, manages, aggregates or otherwise manipulates X’s. Site monitors are meta. Feed aggregators are meta. Most system utilities are meta. Heck, a significant portion of social media is meta – aggregating and discussing others’ content instead of creating their own. As for the plugin that I built – instead of solving some practical task or catering to a specific vertical, it’s purpose was to collect and display information about other plugins. That’s as meta as it gets.

Which is all well and good, except for one thing – we already have too much meta in our lives, and yet we are tempted to create more. Does anyone really need another aggregator? Another blog about blogging? A dozen analytics services for every tiny thing? Updater updaters? Not really. So why do we keep building more? What is the temptation of meta?

Abstraction Attraction

In a way, software development is all about “seeing the bigger picture” and coming up with useful generalizations. Abstract thinking is a core skill for developers.  The (unfortunate?) side-effect of practising that skill is that you start unconsciously applying it everywhere. And since most of your day as a developer is spent interacting with various applications and APIs, you start noticing common issues and overarching trends in various pieces of software. That’s how meta-applications are born.

We are biased to seeing disproportionately many opportunities for improvement in our own fields of expertise. That’s perfectly fine. The problem is that we also tend to overestimate how important those potential improvements would be to everyone else. This leads to a proliferation of obscure and not-that-useful projects that are soon abandoned by the disappointed authors.

Of course, all of the above also applies to other fields and not just programming. Bloggers mostly think about blogging, so many of them get the bright idea to start another blog about blogging. Internet marketing experts try to sell stuff to other internet marketing experts, and so on. I think, though, that this effect is especially pronounced in the software development niche because we’re all specifically trained for meta-thinking.

Scope Maximalism

Suppose you have two applications (or algorithms, or business tasks). Both are currently implemented somewhat inefficiently, and you only have enough time to fix one of them. You could either improve application A by 20%, or application B by 25% (both apps are equally important). Alternatively, you could spend your time implementing a more general optimization that improves the efficiency of all applications by 15%. What do you do?

Chances are you’d pick the third option. 15% for A + 15% for B = 30%, which is more than you’d get if you only worked on one of them.

I believe this is a general heuristic that we all possess. System-wide optimizations tend to benefit all parts of the system, so we’re tempted to solve local inefficiencies by jumping to a different level of abstraction and looking for a general solution. When this heuristic misfires – e.g. when we mistakenly assume that the more general optimization is worth 30% when it’s actually only a paltry 1% improvement – we get meta.

One way this manifests is aggregators. Someone thinks – “Hey, visiting all those blogs every day is inefficient! Lets build a new site that collects all interesting content in one place!” And then thousands more get the same idea, and soon people are building meta-aggregators that collate and filter content from first-order aggregators.

“It’s Safer Here”

Finally, sometimes going meta is simply a way to avoid dealing with the hard problems of your niche. Building a successful website is hard. Building a directory site that hosts links and reviews of other websites can be a whole lot easier. Creating a popular social networking site is a monumental undertaking that requires a huge amount of skill, time, and resources. And even if you have all that, your chances of success are still very slim. Conversely, building some sort of meta-app that utilizes the public APIs of existing social networks is simpler by an order of magnitude. And even if you fail, well – it’s not like you invested years of work and thousands of dollars in it.

Originality hard, meta easy.

Final Words

Meta ain’t bad. I just think there’s too much of it, and that we waste too much time on creating and consuming it (delicious though it may be). Perhaps we should resist the siren’s call of abstraction a little more often.