From the Editor
IEEE Software, May/June 2000
Sitting on the Suitcase
Suppose you’re preparing for a trip and deciding which suitcase to take.
You have a small suitcase that you like because it’s easy to carry and will
fit into an airplane’s overhead storage bin. You also have a large suitcase,
which you do not prefer because you’ll have to check it in, lengthening your
trip. You lay your clothes beside the suitcase, and it appears that they’ll
almost fit into the small suitcase. What do you do? You might try packing
them very carefully, not wasting any space, and hoping they all fit. If that
approach doesn’t work, you might try stuffing them into the suitcase with
brute force, sitting on the top and trying to squeeze the latches closed. If
that still doesn’t work, you’re faced with a difficult choice: leave a few
clothes at home or take the larger suitcase.
Software projects face a similar dilemma. Project planners often find a gap
between a project’s business targets and its estimated schedule and cost. If
the gap is small, the planner might be able to control the project to a
successful conclusion by preparing extra carefully or by squeezing the
project’s schedule, budget, or feature set. If the gap is large, the
project’s targets must be reconsidered.
Software Estimation
Some industry experts imply that the goal of estimation is to achieve
pinpoint accuracy. They claim that effort estimates created using automated
estimation tools can be accurate to within about 10% (see, for instance,
Capers Jones’s Assessment and Control of Software Risks, Yourdon
Press, 1994). Meanwhile, various reports about software industry practices suggest that
real-world estimation accuracy falls far short of this ideal. Effort
estimates that are accurate to within 50% are found in less than half of all
projects (see The Standish Group’s 1994 report "Charting the Seas of
Information Technology").
Estimation accuracy is probably worse than it at first appears. The
Standish Group’s 1994 "Chaos Report" found that "challenged projects" (those
that experienced schedule and budget overruns) routinely threw out
significant amounts of functionality in order to deliver the schedules and
budgets they eventually did. Of course, their estimates weren’t for the
abbreviated versions they eventually delivered; they were for the originally
specified, full-featured versions. If these late projects had delivered all
of their originally specified functionality, they would have overrun their
plans even more.
These cost and schedule overruns are partly attributable to software
developer optimism (see, for example, Michiel van Genuchten, "Why Is
Software Late? An Empirical Study of Reasons for Delay in Software
Development," IEEE Transactions on Software Engineering, Vol. 17, No.
6, June 1991, pp. 582–590). They are partly attributable to the use of
inefficient practices that fall short of expectations. And they are partly
attributable to unrealistic target setting—targets are established, the
targets become commitments, and the commitments are later reported as
"estimates."
Target Setting and Control
The purpose of a software estimate is not to make a perfect prediction
about a project’s eventual cost or schedule. Changes occur throughout a
project that invalidate many of the assumptions that went into early
estimates, and even if we eventually achieve results within 10% of our
original assessment, the results are usually a fluke. The project we end up
with is rarely the same project we originally estimated, so the "accurate"
estimate was really for some other project.
I propose a different goal for software estimation: An estimate should
achieve accuracy sufficient to let the project manager control the project
to meet its business targets. As Tom Gilb points out in Principles of
Software Engineering Management (Addison-Wesley, 1988), estimates by
themselves merely give us the ability to predict project outcomes, but we
don’t need just prediction; we need control.
Estimation on software projects interplays with both target setting and
control. Businesses have many important reasons to establish targets
independent of software estimates, just as you might have important reasons
to prefer a small suitcase to a large one. Sometimes software must be ready
for a trade show, or for a holiday sales season, or for tax preparation
season, or for some other externally imposed date. Sometimes a business is
operating under tight cost constraints and will be penalized for exceeding
cost targets. The business environment dictates these targets, and a company
has little ability to influence them. What a business can influence are the
parameters of its software projects. In this context, a pinpoint-accurate
estimate that "we will deliver our software four weeks after the end of the
holiday sales season" is of little use. The value to the business arises
from being able to control the project enough to deliver desirable
functionality within the business timeframe and cost desired.
Thus, the primary purpose of software estimation is not to predict a
project’s outcome; it is to determine whether a project’s targets are
realistic enough to allow the project to be controlled to meet its targets.
Will the clothes I want to take on my trip fit into the small suitcase or
will I be forced to take the large one? Can I take the small suitcase if I
make minor adjustments? Business managers want the same kinds of answers.
They often don’t want an accurate estimate that tells them that the desired
clothes won’t fit into the suitcase; they want a plan for making as many of
the clothes fit as possible.
Excess Baggage
Problems arise when the gap between the business targets and the schedule
and effort needed to achieve those targets becomes too large. I assert (with
no real proof other than my personal experience) that if the initial target
and initial estimate are within about 20% of each other, the project manager
will have enough maneuvering room to control the feature set, schedule, team
size, and other parameters to meet the project’s business goals. If the gap
between the target and what is actually needed is too wide, the manager will
not be able to control the project to a successful conclusion by making
minor adjustments to project parameters. No amount of careful packing or
sitting on the suitcase will squeeze all my clothes into that smaller
suitcase, and I’ll have to take the larger one even if it isn’t my first
choice. The project targets will need to be brought into better alignment
with reality before the manager can control the project to meet its targets.
Estimates don’t need to be perfectly accurate as much as they need to be
useful. We should not let the pursuit of ever more accurate estimates blind
us to the important but limited role that estimation plays. Control is the
all-important suitcase. The business targets are the suitcase’s valuable
contents. Accurate estimates just provide the wheels that make the suitcase
a little bit easier to maneuver.
Editor: Steve McConnell, Construx Software, 11820 Northup
Way #E200, Bellevue, WA 98005.
E-mail: steve.mcconnell@construx.com
- WWW:
http://www.construx.com/stevemcc/
