Building a House
Building a house is still one of the most common analogies I hear and use for discussing software development with customers. The problem is that the architecture of the house is left out; and unless you’re a developer, it’s hard to understand why a stack of wireframes isn’t the comprehensive description of the system needed to provide accurate cost and schedule estimates.
These wireframes are like the diagrams a first-time home owner might sketch on the back of a napkin. In order to know what must be built, and to determine a realistic sense of scope and cost, those rough ideas need to be developed into something much more substantial by a professional. For our house building scenario, this would involve hiring an architect to transform those ideas into a set of drawings which include such things as plumbing, heat and air conditioning ducts, electrical access panels, drains, the appropriate foundation and structural support elements, and so on. The customer asks for a “nice room with a window looking to the back yard”, but all of these other hidden structural elements must be taken into consideration as well.
Unfortunately, determining the actual cost and size of a large project with any accuracy requires having this detailed design; and worse, doing this detailed design can take weeks or even months. By the time development is underway, the customer will inevitably change many of the requirements anyway. In fact, the very process of evolving such a large and detailed design brings many issues to the surface to discuss, debate, and make decisions about. It’s easy to see how this process could carry on for quite a while, and there are still changes in the market to consider and adapt to, both during and after requirements definition.
The house building analogy is useful, but it is also misused. To be fair, a typical waterfall software project put into house building terms would go like this.
- Customer sends sketches to Architect and asks for plan including schedule and cost.
- Architect works with Customer, being paid for his services to arrive at detailed design, which provides the project schedule and cost.
- Builder works with the Architect’s plan to build exactly what’s in the requirements.
- Days or weeks later, Customer makes changes to requirements. This requires the Architect to get involved again, update the plan, work with the Builder to make physical adjustments, which may include tearing down existing structures and modifying others. Plan changes need to be approved by government (verified with tests), documentation updated, and Customer agrees to pay for the changes.
- * Customer requests to now connect their house to other houses near by, the shapes and layouts of which may change over time.
- Repeat #4.
However, this is completely at odds with how homes are normally built. People typically choose a previously-implemented design, and only customize superficial features like countertops, cabinets, floors, and railings. Houses from this plan have been built before, and the labor and materials cost are known from previous experience.
Building software is usually more like constructing something that’s never been built before: the first sky scraper, the Golden Gate Bridge, or the Hoover Dam. The requirements are unique, the pieces have never been assembled in such a way before, and there’s an inherent level of risk in creating something new. When this is the case, the Customer needs the services of an Architect, not just a Builder who stamps out deliverables in a cookie-cutter style.
Waterfall methodologies have been dramatically in decline over the years in favor of so-called Agile methods. There are several problems with a Waterfall approach:
- Because of requirements churn, attempting to write all the requirements up-front is incredibly wasteful. Your team will end up doing a lot of design work that is eventually thrown away. When I hear about churn numbers as high as 80%, I shudder to think of all the time and money spent refining details that will never make it into the production system.
- It tends to take a blind approach to requirements change, working under the assumption that change is bad and should be prevented. Many waterfall methodologies erect barriers to change, making Customers jump through hoops to push changes through.
- Customers typically don’t want to pay for this architecture phase. They want to pay for actual work on their system, and the cost of defining the actual scope and cost of their system seems steep. “Why should I pay you $<large number> just for you to tell me how much you’ll charge me for the final system?”
An Agile Solution: Efficiency and Uncertainty
The Agile community emerged largely as a reaction to the disappointing results of Waterfall projects. Customers were sick of schedules and budgets that were consistently and grossly overrun (despite all the up-front design), and wanted to more easily change requirements without being put through the wringer to do so. Developers were tired of spending their valuable time writing detailed design documents that kept changing throughout the project, and the burden to update that documentation as the project goals changed focus or priorities shifted made development a very inefficient and slow process.
Essentially, as is often the case, a trade-off was made. Some stakeholders realized that the predictability they were attempting to achieve by spelling out everything in detail was delusional: they were aiming at a moving target regardless of their efforts to pin things down. Project costs rose precipitously because the scope of the system grew. They needed a process that acknowledged and accepted these dynamic project forces instead of one that denied and suppressed them. They needed a process that was efficient, and they gave up the illusion of certainty to get it.
The cone of uncertainty is one of the most valuable tools for visualizing and understanding uncertainty in development projects. From that article:
Research has found that the accuracy of the software estimate depends on the level of refinement of the software’s definition. The more refined the definition, the more accurate the estimate. The reason the estimate contains variability is that the software project itself contains variability. The only way to reduce the variability in the estimate is to reduce the variability in the project itself.
An important—and difficult—concept is that the Cone of Uncertainty represents the best case accuracy it’s possible to have in software estimates at different points in a project. The Cone represents the error in estimates created by skilled estimators [which is up to 16x error at the beginning of a project]. It’s easily possible to do worse. It isn’t possible to be more accurate; it’s only possible to be more lucky.
Commitments made too early in a project undermine predictability, increase risk, increase project inefficiencies, and impair the ability to manage a project to a successful conclusion.
Meaningful commitments are not possible in the early, wide part of the Cone. Effective organizations delay their commitments until they have done the work to force the Cone to narrow. Meaningful commitments in the early-middle of the project (about 30% of the way into the project) are possible and appropriate.
Or as noted by the MSDN documentation for Composite Client Application Guidance:
Application requirements can change over time. New business opportunities and challenges may present themselves, new technologies may become available, or even ongoing customer feedback during the development cycle may significantly affect the requirements of the application. Therefore, it is important to build the application so that it is flexible and can be easily modified or extended over time.
Agile methods have always dealt with small iterations and early, frequent builds to evolve product functionality, but the Lean Development method especially has an explicit focus on making meaningful commitments at the latest possible responsible moment. From Wikipedia:
As software development is always associated with some uncertainty, better results should be achieved with an options-based approach, delaying decisions as much as possible until they can be made based on facts and not on uncertain assumptions and predictions. The more complex a system is, the more capacity for change should be built into it, thus enabling the delay of important and crucial commitments. The iterative approach promotes this principle – the ability to adapt to changes and correct mistakes, which might be very costly if discovered after the release of the system.
While the methodology pundits debate the relative merits of development processes, Tom DeMarco, author of the fantastic books Peopleware and Controlling Software Projects, challenges us to consider the value of our projects and to re-evaluate elaborate control measures in light of the bigger picture. From his article:
To understand control’s real role, you need to distinguish between two drastically different kinds of projects:
- Project A will eventually cost about a million dollars and produce value of around $1.1 million.
- Project B will eventually cost about a million dollars and produce value of more than $50 million.
What’s immediately apparent is that control is really important for Project A but almost not at all important for Project B. This leads us to the odd conclusion that strict control is something that matters a lot on relatively useless projects and much less on useful projects. It suggests that the more you focus on control, the more likely you’re working on a project that’s striving to deliver something of relatively minor value.
This adds additional credence to the idea of agile development, since the Big Design Up-Front (BDUF) of Waterfall is obsessed with detailed definitions to make accurate schedule promises. If the business model is sound, lots of control isn’t needed. It’s better to navigate with a good compass (the product vision) than an incorrect and constantly changing map.
So What Will I Get?
At the end of the day, customers want to know what they’re going to get. There are tradeshow and conference dates to meet, beta and v1.0 launch targets to hit, release cycles to determine, and so on. Ultimately, some up-front architecture and design is usually required. But instead of trying to pin down every detail, it’s actually best in terms of efficiency to share the vision, provide direction, and define the technical scope only in a loose way. This functional scope definition should reflect the reality of the cone of uncertainty at that point in the development process.
At the beginning of the project, items to include in the project should be broadly defined, having a granularity appropriate for the actual uncertainty. As the project progresses and work is done, the scope becomes better defined, risks are determined benign or neutralized, and the cadence of that specific team working on that customer’s specific product can be measured.
This is successfully done with Lean and Kanban approaches, providing an empirical method for measuring productivity. Planning can then be adjusted to match the regular rhythm of production that develops. This creates a semblance of real predictability. You can’t predict any one thing with absolute certainty, but if you can measure the general rhythm at which work is done, you will very quickly learn to recognize how long different kinds of tasks actually take. Because you’re focusing on measured results rather than predictions, it’s easier to normalize task size and estimate larger functional units.
If you’re interested in learning more about Lean development, Mary and Tom Poppendieck’s book Implementing Lean Software Development is a fantastic background and explanation of the Lean process. They also have a new book coming out this November, exploring the same topics more deeply.
When a good process is followed by hard-working, talented developers, the customer will end up with what they need to satisfy their business goals, often in surprising ways, ways that will be radically different from what they initially asked for.
What Can Be Promised?
The customer has their vision, beginning with the end in mind as they do; and they are asked to break it apart, prioritize individual features, and stretch their end-goal image across a long and expensive tunnel that separates them from their goal. They usually want some kind of promise of what they’re going to get at the end, even though they’re going to keep adding and changing features throughout its development.
What can be promised? It would be dishonest to promise customers what they’re asking for, unless they promise not to change their minds as the project progresses, but the only certainty in software development is that requirements will change.
Development isn’t about creating flawless, predictable plans that eliminate business risk. Development is an unpredictable journey to create tremendous value. To achieve great reward, risk is a necessary part of the equation. Instead of coming up with creative ways to hide real project risk, it’s best to have a solid process for detecting and managing that risk.
Critical Development 