Reality-Driven Development
Holy mango! Talk about unexpected. When I wrote my last entry on Feynman and engineering, I was aiming for my 5-strong subscriber base. After one-time deductions of friends and family, that’s a negative number of readers. Not in a million years I could have guessed it would be on Slashdot. But now a decent respect for my newfound readership compels me to explain myself a bit better (or try, anyway).
The biggest controversy was around the "bottom-up" idea. A number of people, including NASA engineers, wrote me about the need for top/down balance. I agree with this view. Feynman’s "bottom-up" is not a dismissal of top-down analysis. As he talks about the lack of a "preliminary study of materials and components" in relation to the engine, it’s clear that such a study would be guided by a plan and exploratory design. After all, engineers can’t randomly test materials until a space shuttle engine crystallizes in front of them. The problem Feynman points out is the lack of essential information about reality in the design. Analysis is important, but it must not overrule or disregard reality. And reality is best exposed by the utmost bottom-up affair: experimentation. Feynman’s bottom-up is empiricism plus the "attitude of highest quality".
He came from the same island as Martin Fowler
I’m not going to dwell on philosophy lest this degenerate into postmodern blabber. For those interested, I think Feynman’s flavor of science is best shown in the last chapter in The Character of Physical Law and in the electromagnetism and quantum mechanics bits of The Feynman Lectures on Physics. The brilliant empirical mind behind Appendix F is laid bare in these wonderful, fun books. But how does this apply to software? Empiricism in a project context is described well in the business literature. Here’s what In Search Of Excellence has to say in the chapter "A Bias For Action":
The problem we’re addressing (…) is the all-too-reasonable and rational response to complexity in big companies: coordinate things, study them, form committees, ask for more data(…). Indeed, when the world is complex, as it is in big companies, a complex system often does seem in order. But this process is usually greatly overdone. Complexity causes the lethargy and inertia that make too many companies unresponsive.
The important lesson from the excellent companies is that life doesn’t have to be that way. Their mechanism comprises a wide range of action devices especially in the area of management systems, organizational fluidity, and experiments. (…)
There is no more important trait among excellent companies than an action orientation. (…) They don’t indulge in long reports. Nor do they install formal matrixes. They live in accord with the basic human limitations we described earlier: people can only handle a little bit of information at one time.
Finally, and most important, is the user connection. The customer, especially the sophisticated customer, is a key participant in most successful experimenting processes.
Action and experimentation are the cornerstones of empiricism. No attempt is made to subdue reality by extensive analysis and copious documentation. Reality is invited in via experiments. Instead of agonizing over market research, an empirical company hires interns and develops a product in one summer. A non-empirical company has 43 people planning an off-button design for one year. Empirical companies still rely on analysis. P&G has memos, they’re just limited to one page. But software projects are not after "empirical reality", we just want working products. Built to Last deftly relates experiments to process in a chapter entitled "Try a Lot of Stuff and Keep What Works":
What looks in hindsight like a brilliant strategy was often the residual result of opportunistic experimentation and "purposeful accidents".
Bill Hewlett told us that HP "never planned more than two or three years out". (…) We could go on with examples from Citicorp, Philip Morris, GE, Sony, and others. (…) We were surprised to find so many examples of key moves by the visionary companies that came about by some process other than planning. Nor do these examples merely represent random luck. No, we found something else at work (…): evolutionary progress. Evolutionary progress begins with small incremental steps
After dubbing 3M the "Mutation Machine From Minnesota" the authors say:
If we had to bet our lives on the continued success and adaptability of any single company (…), we would place that bet on 3M. Using 3M as a blueprint for evolutionary progress at its best, here are five basic lessons (…).
- Give it a try – and quick!
- Accept that mistakes will be made.
- Take small steps.
- Give people the room they need.
- Mechanisms–build that ticking clock
Built to Last makes the inescapable link to biological evolution, the epitome of bottom-up experimental development. Top companies experiment vigorously with products and processes, driven by the market and organizational metrics. Nature experiments with genetic variation, driven by natural selection. The common theme is that successful systems are driven by reality through experimentation. That’s dandy, but how about software? The best discussion I know of software-as-evolution is the famous LKML thread where Linus shuns top-down design in favor of experimentation. I think of it this way:
A good software development process should optimize experimentation and improve feedback from reality. This is what I mean by reality-driven development. And in software the most important realities are user experience and technical quality, while the primary experiments are working software and code. This isn’t a formal model (heh), it’s simply my favorite analogy for software development. I like the name "reality-driven" because when you mention reality people think of users. And I like the model because it helps me focus on important stuff and on effective ideas, like Paul Graham’s advice to release early and let the market design the product. It also has good explanatory power. Firefox is such a great browser due to intense experimentation in the form of add-ons. Waterfall is so awful because reality is ignored: when the time for feedback comes, the project is over.
There is no specific reality-driven methodology. The Agile principles have a lot in common with these ideas (and certainly influenced them), but the devil is in the details. I prefer to think of software engineering in terms of a toolbox, full of techniques we pick and choose for the right situation. Process tools for optimizing experimentation include iterative development, executable architecture, continuous integration, and unit testing.
Based on this model, the two realities we care about are user experience (including the software’s utility) and technical quality. User experience is often neglected in agile and waterfall alike. The measurement tools come from the usability people and from plain old business sense. Techniques include usability testing, observing users, spending time with users (preferably in their habitat), talking to users, and hugging users. Technical quality revolves around the code base and third party tools. Here we’re looking for the ol’ bit of ultraviolence plus generality, clarity, simplicity, security, etc. Tools include code inspections, code reviews, and metric reports as part of the build. The elusive hiring of good programmers is crucial, but it’s not measurement, so it falls within the "software project" box.
When I think about pre-requisites (requirements and top-down design) I do so in the context of this reality-driven model. Pre-requisites can optimize experimentation by minimizing cost and risk. I have seen how well-written requirements can quickly take a team from zero to working software that’s close to users’ wishes. Likewise, good top-down design can help achieve technical quality faster. But I think of prerequisites as sketches, not blueprints. I prefer minimal specs that produce working software to be molded by the users. And rigid upfront design is a sure way to a crappy code base or engineering disasters. Alistair Cockburn put it best: "With design I can think very fast, but my thinking is full of little holes."
In the end, feedback from reality helps you avoid Ivory Tower Development and pass the Ultimate Unit Test. You make your users happy. A reality-driven process with management buy-in purges faulty o-rings and gets the right materials in a shuttle engine. It avoids abominable applications. It brings money and fame and huge obelisks in your honor. So now you know my idea of bottom-up:
- Have a bias for experiment over analysis, though both have their place.
- Optimize experiments: make them as early, fast, cheap, and broad as you can. Analysis can help here.
- Experiment vigorously.
- Be smart and proactive about measuring reality: user experience and technical quality.
- React to feedback. Let reality drive.
Of course, you can turn the empirical machine towards the process itself, and try to improve the way you build rather than what you build ("It’s fractal, dude!"). That’s the whole point of Built to Last. Also, I’ve found that Built to Last and In Search Of Excellence work well for explaining evolutionary/agile ideas to senior management.
I hope I didn’t kill the aforementioned newfound readership by boredom. Thanks for reading and see you next time. The new server arrives on Friday.
February 27, 2008 at 9:15 am | Filed Under Programming
Subscribe to blog
Comments
22 Responses to “Reality-Driven Development”
Leave a Reply
The interesting thing about Agile is that in most ways it is Bottom-Up, especially on a macro level, however, its main design tenet, TDD, is very much Top-Down. TDD evangelists are big on Mocking/Stubbing objects because the idea behind TDD is to build the test before the SUT (System Under Test) has been built. Test Supported Development (TSD) is NOT Test Driven Development (TDD): http://blog.troyd.net/Test%2bSupported%2bDevelopment%2bTSD%2bIs%2bNot%2bTest%2bDriven%2bDevelopment%2bTDD.aspx
Hi Troy, I think you make a great point in your post. The prescriptive nature of methodologies draws a lot of people in. I see pre-packaged methodologies as fad diets. They cater to people’s longing for a magic pill. I think the "exercise more, eat less" equivalent in software is learning about different techniques, trying them out, and using what works at the _right time_. The idea of writing an interface by starting with the client is old as dirt. It’s something I’ve done many times and I see value in it _at the right time_. But the orthodoxy is bad, I think.
Bottom-up design/development works well when you don’t know what you’re doing, and when you are exploring the problem space. Top-down design/development works well when you already understand the problem (likely because you’ve solved it before). There’s little value in getting religious about one approach or the other. Both have their place.
@Louis: Agreed, pragmatism above all.
Pingback from braindump.dk Tech Life of Recht » Blog Archive » X Driven Development
Pingback from foojam.com Reality Driven Development | foojam.com
Pingback from newsmavens.wordpress.com links for 2008-02-28 « Brent Sordyl’s Blog
Pingback from spinu.ro spinu.ro » Blog Archive » Reality-Driven Development
Pingback from chipsquips.com links for 2008-02-29 — Chip’s Quips
Gustavo: As I’ve mentioned to you in person, I’ve spent the majority of the last couple of months interviewing companies in various industries about how they make decisions. As it turns out, the same agile – to – rigid metaphor we see in software development and – as you correctly point out – in business as a whole – applies specifically to strategic decision making processes everywhere from global warming to telecom to the "which movie do I make" decision faced in Hollywood. I’m in New York this week. On a subway train with the kids today (from Ground Zero to the Car Show – another story in itself…) I came up with a metaphor that I think extends your "top-down" to "bottom-up" distinction. Have you seen the YouTube video showing a four-legged jointed robot (called “bigdog”, see http://www.youtube.com/watch?v=YUoW43Xq7ts) that navigates complex terrain and can respond to being pushed by righting itself? I wonder if this creature can serve as a model for thinking about organizational structure in a global and competitive environment. Too centralized and top-down an organization, and we can’t respond to a changing environment, don’t have adequate feedback from our “feet on the ground” front line. Too decentralized, and every joint of the robot is moving in its own empowered visionary direction, and the creature flails. A misalignment of shareholders, staff, and customer interests can, in particular, create a very slow moving organism. Note there are deep parallels here with Deming, TQM, and the demise of the US automobile industry in response to Japan. Here in New York, I met a friend who is on the stage crew for a Broadway production. He told me that a common mantra is to “serve the script” – that although artistic interpretation is encouraged on the part of actors, lighting design, Wardrobe, and so forth, serving the script is a unifying theme. I wonder if we can think of an organization faced with new competition in a global environment in this way. That in the past, a fairly rigid organization was functional, because we were on “even terrain”, where we had huge latitude for pricing, products offered, and other strategic decisions, because the environment in which we moved simply wasn’t that complex. The robot dog here could have a simple, top-down, design because it was faced with a simple task. In contrast, we now must navigate rocky terrain, so every “joint” of our organization must be both coordinated, but also reactive, and we need a unifying theme – the “script” – that guides our movements, but not in too constrained a way. Top-down *plus* bottom-up, meeting in a well-coordinated middle, accepting the organic nature of the organizational beast. I think of our approach at Quantellia to fitting into this vision – giving us a formalism for making smart decisions requiring large amounts of data and also human intuition, and also for coordinating those decisions around a large organization. Early days, though, on how this vision knits software tools together with process and people. And of course the devil is in the details: how do pricing decisions impact overall yield? How can costing information be efficiently maintained? Just my 2:30 in the morning $0.02. Interested in your thoughts.
How does a business "serve the script" in this economic climate? An interesting question. "Serve the script" works well in the entertainment industry (what I prefer to call the Entertainment Industrial Complex) because, well, we have literal stories to tell and a top/down & bottom/up integration that every member of the production team is accustomed to utilizing to create the product. The Japanese companies realized that creating a "village model" for their companies’ employees, i.e. cradle to grave care, strong community identification, post work day socialization essentially required, lots of family oriented activities, etc. The corporation became the employee’s village, with the requisite loyalty to company and purpose (profit). Does that translate in 21st cent. Western corps? Not so much in the USA as the "cradle to grave" model has become unsustainable (if indeed it ever was) and increased lack of long term job stability has created an entirely new climate of job jumping, outsourcing, and very random career paths. There are some companies who have adopted "Serve the client/user" as a workable version of "serve the script" – off the top of my head I think of Southwest Airlines, Harley Davidson, both of which are noted for their employee’s devotion, quality products and lack of turnover. Certainly the community / village meme is paramount in the military, loyalty to your unit, unity of purpose, a common goal are all essential for cohesiveness. I suspect that software companies have a built in advantage in their quest for a script to serve as the evolutionary nature of software development lends itself handily to the meme of a village striving for a common goal. Many individuals working together to create a more functional/efficient software for roll out, then the endless tweaking when/if the product catches on is tailor made for creative types to bond together around the common goal of the best software, best support, and how to improve the customer experience. It appears to me that the challenge for traditional companies in the 21st cent is how to find a "script" and get all employees, top to bottom, engaged in the goal of "serving the script", especially as the economic foundations of our world are in this great state of flux.
Thanks for the thoughtful comments. I think the necessity for an overall top-down direction is clear. This is one of the biggest holes in the evolutionary metaphor. Evolution has no overarching direction, whereas businesses and projects surely do. Every single example in this entry is bound to overall direction. The Linux kernel, regardless of its developers’ fondness of the evolutionary metaphor, has had strong direction in a number of ways. The technical goal of building a fast Unix kernel is very strong guidance. The technical approaches of top hackers like Linus, Alan Cox, Al Viro, etc, are somewhat aligned and well-known, which also provides direction. Just these two things alone – a narrow ultimate goal and shared understanding of how things should be built – provide a clear script for Linux kernel development. 3M, in all their beautiful madness, obviously has an overall direction. You don’t see them trying products in any random field. They have a core direction, and core values, that do guide the whole even as they don’t dictate what the parts will do. So yes, I think the analogy of the creature is great. You need overall plan _and_ evolutionary experimentation. Now, the creature freaks me out, so I don’t know if I can really use it
I’d sooner have Orwell’s boot on my face for ever than be hunted down by one of those things! Man, that is one freaky video hehe.
I see the misalignment of shareholders, management, employees, and customers differently. That stuff is more of the traditional agency problem, http://www.auburn.edu/~johnspm/gloss/agency_problem, than organizational/process design. In a project it’s reasonable to assume that ultimately goals are shared (even if in ill environments that is not the case). Nobody wants the shuttle to explode and we all want a fast bug-free kernel running everywhere. When it comes to shareholders vs. management (or employees or customers), then it’s reasonable to think that goals are _not_ actually shared and we have a whole host of other issues. But apart from that your ideas regarding overall direction match what I think. Maybe more at Donut Hut?
This metaphor of serving the script is great by the way, thanks for that. @Andrew: The question of finding the script for a business is crucial. I think "Built To Last" does a good job of describing a viable way for companies to find (or make) their script. Without wanting to sound like a pop-business drone, I highly recommend Built to Last in this regard (I also recommend The Halo Effect, http://www.amazon.com/gp/product/0743291255/, for a dose of skepticism when it comes to these ‘case study’ books). Basically, I think the script should be a small set of values and goals for the organization. Of course, every company out there has their "mission statement", "core values" or what have you. Except they’re bullshit. They’re flouted constantly by upper management and the foot soldiers sure as hell notice that, cynicism sets in, and there’s no script. If, and only if, and a big if, the set of core values and goals is respected by upper management and taken seriously, then it does become a powerful script. If you hire the right people, they’ll follow the script, and the organization has a powerful drive and direction. Since the core is small, made up of basic values and goals, it does not cripple innovation and experimentation. It’s the balanced whole Lorien talks about. Few companies pull this off, short of startups, but some have and some do. I also think the US has had a very powerful script and that’s partly why it’s such a great place. But alas, this is definitely beer territory and blog comment abuse
Pingback from spin.atomicobject.com atomic_spin » Blog Archive » Reality-Driven Development
Gustavo, Andy: I’m reeling a bit tonight, as I just started reading http://www.deloitte.com/dtt/cda/doc/content/DTT_DR_StratFlex_Comms.pdf . This formalizes, and systematizes, the walking-dog-serving the script model. My former colleague at Frost & Sullivan, Pete Dailey, published on this coincidentally three days ago as well: http://www.ad-hoc-news.de/drucken.html?art_id=16126658§ion=Aktie . The thesis, as best I can paraphrase from a few pages’ read of the Deloitte piece, is that there are ways of running a business and compensating managers when things are stable, but that we are now in a metastable situation in many indutries (this one is about telecom). This means that we have to create completely different organizational and compensation structures: loose-jointed, constraint-based systems where managers are compensated on their ability to *adapt* rather than to reach certain executive-mandated annual business targets: a fundamentally nonreactive – and therefore stunningly obsolete – framework. This is the next stage in the evolution of the creature we call a company. This puppy may well walk one day, even learn to turn left and right without an 18 month lead time. Given that we’re talking about multi-billion dollar transnational corporations, that’s quite a trick. Applied to software, this is XP squared, Extreme Agile, the three-legged stool standing up and walking away on its own. We’re not in Kansas any more, and you’re not Toto. Yeah, I think I need a donut. -Lorien
[...] takes a look at the guts of the kernel to see how an operating system starts life. Since I have an empirical bent I’ll link heavily to the sources for Linux kernel 2.6.25.6 at the Linux Cross Reference. The [...]
[...] sum, do what it takes to obtain good data and rely on it. I’m big on empiricism overall, but in performance it is everything. Don’t trust hearsay, don’t assume that [...]
I have been reading all your posts and I must conclude, you have an excellent blog Gustavo. Please keep the entries coming!
@Jose: thanks mate
Good technical blog…
I stumbled on Gustavo Duarte’s blog this week via this post about how lucky we are to be programmers. The post that led me to his blog is good stuff and has had lots of linkage this week. The rest……
You are professional indeed
[...] takes a look at the guts of the kernel to see how an operating system starts life. Since I have an empirical bent I’ll link heavily to the sources for Linux kernel 2.6.25.6 at the Linux Cross Reference. The [...]
RDD means you actually not only like what you do, but you are willing to be good at it for the client as well.