Most Programming Is Not Core
Coding Horror started a discussion on the relative merits of writing your own code versus using 3rd-party libraries. The main argument for writing your own is borrowed from Joel’s In Defense of the Not-Invented-Here Syndrome:
If it’s a core business function — do it yourself, no matter what.
Pick your core business competencies and goals, and do those in house. If you’re a software company, writing excellent code is how you’re going to succeed. Go ahead and outsource the company cafeteria and the CD-ROM duplication. If you’re a pharmaceutical company, write software for drug research, but don’t write your own accounting package. If you’re a web accounting service, write your own accounting package, but don’t try to create your own magazine ads. If you have customers, never outsource customer service.
Righty. But then why did Fog Creek, Joel’s company, build their Copilot product based on the open source Tight VNC? I mean, that is the very core of their product, and they decided to use third party code. It’s hard to think of a bigger counter-example to his claim (there are plenty of others, like Flickr using ImageMagick and MacOS using BSD code). So why the contradiction?
It turns out the quote above is naive and simplistic. Writing excellent code is not how you’re going to succeed. You’ll succeed by delivering value and making users happy, hopefully fast. Given that you have limited resources, you need to prioritize your precious programmer firepower and funnel it into those areas that will help you differentiate and build really useful stuff. Rebuilding compilers and libraries is not how you do that. Which is why Joel, who is anything but naive, borrowed the core engine of his product from VNC and concentrated on delivering smooth usability.
Thinking of “programming” as your core business function is too broad. Only a subset of your programming is truly core. You won’t do better than Prototype, jQuery, or extjs. Rails really is great, and so are CakePHP, lex/yacc, and HTML Tidy. The instinct to use third-party libraries is absolutely right. Only when the choices are truly unsuitable or inexistent should you roll your own. This is especially true now that we have so many high-quality open source libraries. Or at least you do when you’re developing in the open source ecosystem, which is something that needs to be taken into account when deciding on the right platform.
People often forget to factor in opportunity cost when thinking of tradeoffs. When you re-write stuff, not only you spent the time and money, but you also did not build something else that might have been valuable. So pick the right libraries and concentrate on giving us the goodness only you can build.
October 17, 2008 at 4:54 pm | Filed Under Business, Programming
Subscribe to blog
Comments
8 Responses to “Most Programming Is Not Core”
Leave a Reply
Too right. You have to make ends meet at the end of the day and spending a month re-inventing the wheel isn’t necessarilly conducive to that.
However I also think that as programmers we have an instinct to understand and control things as thoroughly as possible, and libraries take as further away from that in some ways.
I don’t normally comment on blogs, but bravo; most excellently written post and I cannot agree more. Adding your blog’s feed to my small collection of quality blogs. Keep it up.
I dissent: the vast majority of the sea of open source is crap. Once you tread outside the critical path of high-user-base server-side software, like (to take three examples completely at random) Linux, Hibernate and MySQL, you very quickly get stuck in the mire; even then, much of the “good” open-source software is only good in parts.
For example, if you’re looking for a mature, well-designed and stable interface for writing drivers for Linux - exactly what is needed to make supplying binary drivers viable for hardware vendors - tough luck, buddy. The incentives in open source are almost always not aligned with the consumers’, but rather with the companies which hire the developers: and those guys are usually consultant / service shops, i.e. aligned with *corporate* customers, not consumer customers. Firefox is an exception because of Google advertising revenue, which is correlated with consumer adoption, thus their incentives are aligned.
Rails is a solid concept and has a good API, but its execution is horrific. I wouldn’t touch PHP with a barge pole - I have zero technical respect for the language. Lex & yacc (flex & byacc/bison) are 70s era tools with limited performance, limited utility and limited flexibility, and are best suited to academic/ad-hoc languages - you’ll have a hard time finding a commercial compiler that doesn’t use hand-written recursive descent.
To compete, you need differentiation. Open source can only give you what everyone else has; you need to add in the magic that makes you stand out. Private forks of open-source software are one way of doing that, but is vulnerable to all the usual problems that forking has. Creating your differentiating software yourself means your codebase will be smaller and more perfectly adapted to your problem domain, and thus easier to maintain and keep up to date than a private fork of an ongoing public open-source project.
The only real question is which software in your organization is going to help give you a sustainable competitive advantage, in the language of strategic management. Open source software and third-party libraries can never give you that (unless you own the founding team & brand), because they are equally available to your competitors. Choosing the balance of build vs buy means trading off risk and cost versus lack of differentiation. You have to measure and compare the resources available to you and the risk of non-completion of differentiated software, with the lack of differentiation that would come from the buy / co-opt decision.
Where the balance lies thus depends on both sides: how much [human+monetary] capital you have, versus the competitive landscape on the other.
Whether a third-party library exists or not, in and of itself, is not a sufficient reason to always use it. Making users happy isn’t sufficient; you must make them *happier* than the competition, and that’s not going to happen if you don’t do something the competition isn’t already doing. The only thing that holds you back is risk & capital versus the reward in differentiation. If Fog Creek thought they could have reimplemented RDP and come out ahead, they would have.
Please stop using the term “ecosystem” with respect to open source. An ecosystem implies an economic model that is self sustaining, but the factual evidence of open source reveal it to be flawed both economically and in terms of quality. There is no open source ecosystem.
Actually I don’t think the term “ecosystem” implies anything of the sort; the eco in ecosystem comes from ecology rather than economics, and a functioning ecosystem might very well contain both crappy, diseased, worthless organisms, as well as shining with health and vitality, paragons of virtue, long lived organisms.
In the flora and fauna of open source software there are undoubtedly examples of both of these types, as well as a myriad others. This says nothing about the economic viability or not of the open source model, but given that there is a whole bunch of people out there who continue to use open source software in commercial enterprises, and another whole bunch of people who for some reason commit their time to work on open source projects, it’s probably safe to say that establishing the true economic worth of OS software is more complicated than you’ve stated.
I understand your argument but I think your premises behind are incomplete and don’t tell the full story as it plays out within the corporate IT shops.
Sometimes less is more, and (particularly in the Java space) researching, understanding and picking frameworks at various tiers, can take more time than had you simply solved the problem bottom-up in a KISS fashion.
Perhaps I’ve simply been burned on too many JEE and WS* stacks, but I definitely feel this aspect is often neglected. REST webservices over WSDL is a good example of this, it’s not an actual framework as much as a simplistic architecture (which Flickr, Amazon, Google, Facebook etc. now all favor).
So tell me Barry Kelly what do you use on your website? If you hate php, rails and who knows what else, please tell me.
“Which is why Joel, who is anything but naive, borrowed the core engine of his product from VNC and concentrated on delivering smooth usability.”
You’ve almost said it, but Fog Creek’s product is not remote windowing software — that product already existed in several VNC products as well as Microsoft’s own Remote Desktop that comes built into Windows.
The real feature of Fog Creek Copilot is in its simple usability, without difficult installation and network configuration to get it working. It turns the underlying remote windowing software into a commodity component, just the way a relational database is a commodity component of most business applications - the user probably doesn’t know whether you use MySQL, SQL Server or text files, and if the product works they don’t care.
For a commodity component, whether it’s an engine for your family car, hard disk for your computer, database system or another software component, sometimes good enough is good enough, the value is added elsewhere, and you probably won’t add any more value to the product as a whole by building the component yourself.