Software Project Survival Guide: How to Be Sure Your First Important Project Isn't Your Last. Redmond, Wa.: Microsoft Press, 288 pages, 1997. Retail price: $24.99. ISBN: 1-57231-621-7. Available from Microsoft Press 1-800-MS-PRESS (1-800-677-7377). Buy SPSG from Amazon.com.

Survival Skills 

Small projects can succeed through sheer force of will and a bit of luck. Medium and large projects require a more systematic approach. This chapter outlines some of the skills needed to make a medium-size project succeed.

Planning

Many technical workers would rather do technical work than spend time planning. Many technical managers do not have sufficient training in technical management to feel confident that their planning will improve the project’s outcome. Since neither party really wants to do the planning, it often doesn’t get done.

But failure to plan is one of the most critical mistakes a project can make. Because of the upstream/downstream effect described in the last chapter, effective planning is needed to resolve problems upstream, at low cost, rather than downstream, at high cost. The average project spends about 80 percent of its time on unplanned rework—on fixing mistakes that were made earlier in the project.

Success in software development depends on making a carefully planned series of small mistakes in order to avoid making unplanned large mistakes. Exploring four design alternatives and discarding three of them amounts to making three small mistakes. Not doing enough design work and rewriting the code three times amounts to making three large mistakes. Because fixing upstream defects downstream costs 50 to 200 times as much as fixing them upstream does, carefully orchestrated projects jump at the chance to fix as many mistakes as possible at the bargain price of 1/50 to 1/200 their potential cost.

Where do projects find the time to plan? It’s easy. You take a small percentage of the time that most projects spend on unplanned rework and invest that in the project early to avoid doing extensive rework later. Not all the time spent on upstream activities will actually result in downstream savings, but much of it will. A good rule of thumb for upstream quality assurance activities is that each hour spent on activities such as technical reviews will save 3 to 10 hours of downstream costs. Seen from a slightly different perspective, each day a developer spends reviewing a project requirements or architecture will typically save the project 3 to 10 days later in the project.

Examples of Software Planning

How does a project team plan a project to deliver software on a budget? Here is some of the work used specifically for project planning:

A Software Development Plan maps a course for the project. Committing the plan to writing allows the project’s stakeholders to refer to the plan throughout the project.

Project estimates provide a foundation for project plans. A careful estimate leads to scoping the project appropriately, which in turn leads to budgeting, staffing, and scheduling it appropriately. A shoddy estimate can undercut the project in all these respects, making it difficult to complete the project successfully and impossible to complete it efficiently.

Revised estimates created at the end of each major phase allow for mid-course corrections and help to keep the project on a solid footing.

A quality assurance plan that includes both technical reviews and testing assures that the project will not succumb to a costly, defect-ridden test, debug, correction cycle.

A detailed implementation plan defines the order in which the software will be constructed. It ensures that the software solution is constructed in a way that maximizes the value available to the customer at each stage and minimizes the risks.

In addition to these explicit planning activities, several of a software project’s major activities can be thought of as planning activities.

Requirements development identifies in detail the problem the project team is trying to solve. It amounts to planning to solve the right problem.

Architecture creates a high-level specification for the way in which the problem will be solved. It is a plan to build the right solution to the problem.

Detailed design is a comprehensive plan of what the project is going to build. It is a plan to build the right solution in the right way.

Planning Checkpoint Review

Planning is so critical to the success of a project that some experts have reported that a project’s success or failure is determined as early as 10 percent into the project. Whether the exact number is 10 percent or some other number, very early in the project the project team should have produced a user interface prototype, detailed requirements, and a detailed project plan including careful cost and schedule estimates. That information can be used to make a "go/no go" decision about the project.

Two-Phase Funding Approach

One problem with software project funding in some organizations is that project managers have to ask for funding for the entire project before they have had a chance to complete much exploratory work. Such funding requests inevitably miss the mark because too little is known about the desired software to support creation of meaningful cost and schedule estimates. The software industry’s experience is that estimates created at this stage of a project often miss the mark by as much as a factor of 4 on either the high or low side.

A better approach is for the project manager to request funding in two major phases. The project manager first requests funding for the exploratory phase during which the first 10-20 percent of the project is completed. At the end of that phase, the project team holds a Planning Checkpoint Review. In conjunction with that review, senior management or the customer makes a go/no go decision, and then the project manager requests funding for the remainder of the project. At this point there is still a large potential for variation in project cost, but the exploratory work that has been done will bring it down from a factor of 4 either way to more like ± 50 percent.

Preparing for the Planning Checkpoint Review

Before the planning checkpoint review can be held, the following materials need to be available:

Name of project’s key decision maker

Vision statement

Business case for the software

Preliminary effort and schedule goals

Preliminary effort and schedule estimates

Top 10 Risks List

User Interface Style Guide

Detailed User Interface Prototype

User Manual/Requirements Specification

Software Quality Assurance Plan

Detailed Software Development Plan

Each of these materials is explained in more detail elsewhere in this book.

If these materials are not available, there’s no point in holding the Planning Checkpoint Review because there won’t be enough information available to determine the project’s viability. If the project team continues to be unable to create these materials in order to support the Planning Checkpoint Review, it is safe to assume that the project is not being run effectively and faces a great risk of failure.

The exact amount of time required to create these materials will vary depending on how much work is needed to identify the software’s requirements. In circumstances in which end users know exactly what software they want built, this period might take only 10 percent of the software’s total development schedule. In more typical cases, it might take 10 to 20 percent of the total development schedule. On some projects, the hardest part of the job is helping the end users figure out what they want built, and so occasionally this part of the project can take 25 percent of the total development schedule or more. The initial funding request and plans for the Planning Checkpoint Review should take this variability in requirements work into account.

Agenda of the Planning Checkpoint Review

The Planning Checkpoint Review should focus on the following topics:

Is the original product concept still viable?

Will it be possible to develop a product that matches the project’s vision statement?

Is the business case for the software still justified when the updated, more accurate cost and schedule estimates are considered?

Can the major risks to the project be surmounted?

Have users and developers been able to agree on a detailed user interface prototype?

Is the User Manual/Requirements Specification complete and stable enough to support further development work?

Is the Software Development Plan complete and adequate to support further development work?

What is the estimated cost of completing the project?

What is the estimated schedule for completing the project?

The work done during the first 10-20 percent of the project should be sufficient to answer these questions, and the answers to these questions should give the client or top management enough information to decide whether to fund the second phase of the project.

Major Benefits of the Planning Checkpoint Review

Breaking software project funding into two major phases helps software organizations in at least three ways. First, there is a tendency to view any canceled project as a failure, but a project canceled at the 10-20 percent complete point should be considered a resounding success. Canceling one project that ultimately goes nowhere after it is 10-20 percent instead of 80-90 percent complete can pay for the exploratory phases of a lot of other projects.

Second, by deferring the bulk of the funding request until after the project is 10-20 percent complete, it provides for much more reliable funding requests for the bulk of the project.

Finally, requiring project managers to complete 10-20 percent of their project before requesting funding for the rest of it forces them to focus on the upstream activities that are critical to a project’s success. These activities are often abbreviated or ignored, and the damaging consequences of such neglect don’t otherwise become apparent until late in the project when many defects have to be corrected at great cost downstream. If the project team is required to complete the most important upstream work before proceeding with downstream work, overall project risk can be substantially reduced.

Risk Management

A special kind of planning is risk management. Anyone who has been through a medium or large software project knows first-hand that dozens of things can go wrong. The most successful projects take active steps to avoid succumbing to such problems. You might be an optimist, but with software projects, as the saying goes, you can hope for the best but you should prepare for the worst.

Several of the most serious software project risks are related specifically to planning:

Failure to plan

Failure to follow the plan that has been created

Failure to revise the plan when project circumstances change

Not practicing active risk management on a software project is tantamount to ignoring decades of experience in which the software industry has learned thousands of times that software development is a high-risk activity. Many risks can be prevented or minimized if the project performs active risk management. Many of those same risks can cripple a project if not actively addressed.

The practices contained in this book’s survival strategy (and discussed throughout the rest of the book) have been chosen because they involve less risk than alternative practices or because they promote detection and control of other kinds of project risk. You don’t really have a choice about whether to practice risk management on a software project. As Tom Gilb says, if you don’t actively attack the risks on a software project, they will actively attack you.

Project Control

A theme of this book is that software projects can be controlled to meet their schedule, budget, and other targets. To some people, the idea of "controlling" a project sounds almost inhumane. It conjures up images of an oppressive project manager wielding authority with a whip and brass knuckles. If that’s what you think, realize that the opposite of "controlling" a project is having a project that is literally out of control.

For some reason people think that it is somehow possible to have a project that is under control without anyone or any group of people actually controlling it. Both my brain and my experience tell me that is impossible.

The opposite of "controlling" a project is having a project that is literally out of control.

Some people object to the idea of project control because they think it refers to controlling the people on the project. It doesn’t. It refers to controlling the project itself. Here are some examples of what I mean:

Choosing a software lifecycle model such as the staged delivery model used in this book to provide a framework for the project’s technical work.

Managing changes to requirements so that only necessary changes are accepted.

Setting design and coding standards so that the designs and source code produced are consistent with each other.

Creating a detailed plan for the project so each developer’s work contributes to the goals of the project and doesn’t conflict with other developers’ work.

Control is not something that happens as a by-product of good technical work. Control needs to be built into the project explicitly through active project management and continuous, ongoing control activities. Specific activities are discussed throughout the rest of the book.

Project Visibility

Closely related to project control is the concept of "visibility," which refers to the ability to determine a project’s true status. Is the project performing within 10 percent of its schedule and budget targets, or only within 100 percent? Is the project on track to achieve its quality goals, or is it lagging behind? If the project team can’t answer such questions, it doesn’t have enough visibility to control its project.

Here are examples of activities that contribute to visibility:

Using a vision statement or goals to set broad objectives for the project. If you don’t know where you want the project to go, it probably won’t go there.

Holding a Planning Checkpoint Review after the project is 10 percent complete to determine whether the project is viable and should be completed.

Regularly comparing actual performance against planned performance to determine whether the plan is working, corrective action needs to be taken, or the plan needs to be modified.

Using binary milestones to determine whether tasks are done or not done. Milestones are considered either "100 percent done" or "100 percent not done" because allowing that little step from "100 percent done" to "90 percent done" has historically been found to reduce the quality of the status information from "very good" to "awful."

Driving the product to a releasable state periodically to help determine the product’s true condition and control its quality level.

Revising estimates at the end of each phase to support improved plans based on more information and better understood planning assumptions.

Many project teams have found the hard way that good visibility is not something they get automatically. If you want good visibility, the project team has to plan it into the project from the start, and if you want a successful project, you have to have good visibility.

Peopleware

Software development requires creativity, intelligence, initiative, persistence, and a great degree of internal motivation. Any effective approach to software development must remember that if the development team isn’t on board, the project cannot possibly succeed. It might ship. But it will ship with low quality. Or it will ship without the spark of inspiration that differentiates a merely passable product from a great one. Or a substantial part of the development team will quit within a few weeks of releasing the product.

Tom DeMarco and Timothy Lister popularized the term "Peopleware" to refer to software management practices that recognize the importance of the human role in software development. If you are not a software developer, some of the most important peopleware guidelines might surprise you.

Align developers’ interests and work assignments.
As a general rule, the single greatest motivator for an individual software developer is the alignment between the developer’s interests and the developer’s assigned work. If developers find their work interesting, they are highly motivated. If they find their work boring, they are demotivated. After 15 years of research, Robert Zawacki reported that about 60 percent of a developer’s productivity comes from the match up between the job and the developer. For best productivity, be sure that developers are assigned to jobs they find stimulating.

Do try to show developers that you sincerely appreciate them.
Like everyone else, developers like to be appreciated. If the project sponsors sincerely show they appreciate the developers, the developers’ commitment to the project will tick up a notch. If developers think the display of appreciation is phony or manipulative, their commitment will tick down.

Don’t try to motivate developers with cheerleading campaigns, impossible challenges, or monetary rewards.

Some people are motivated by impossible goals. Because developers pride themselves on being realistic, they tend to be demotivated by goals that are too far out of reach.

Provide thinking-oriented office space.
Software development is process of continuous discovery and invention. The atmosphere most supportive of that process is one that is relaxed and contemplative. Effective software development requires that developers achieve a level of concentration similar to that of a mathematician or physicist. Can you imagine Albert Einstein sitting at his desk while his manager berates him, "Albert, we need that theory of relatively now! Hurry up!" Since software developers aren’t as smart as Albert Einstein, they need an even more supportive work environment.

A handful of software research studies have found that productivity of developers who work in private, quiet, one- or two-person offices can be as much as 2.5 times as great as the productivity levels of developers who work in open work bays or cubicles. Software development is a deeply intellectual activity. It’s hard to be intellectually effective with phones ringing, announcements blaring over the public address system, and people walking up to your cubicle every few minutes to ask questions on various topics.

The job of the average manager requires a shift in focus every few minutes. The job of the average software developer requires that the developer not shift focus more often than every few hours.

Avoid open work bays.
There is a claim the recurs from time to time that open work bays encourage communication on software projects. The problem is that open work bays encourage incidental, unstructured communication that hurts productivity. Beneficial, incidental communication can be encouraged more effectively by installing a soda machine in a place where developers can bump into each other when they’re not trying to concentrate on problems that demand their complete concentration. The claim in favor of the communication benefit arising from open work bays has an intuitive appeal but doesn’t bear close examination, and the software research data indicates clearly that developers are most productive in one- or two-person offices.

Many organizations are restricted in their ability to provide developers with quiet, private offices. Either their general office spaces simply do not have enough private offices for each developer to have one, or private offices are treated as VP-level status symbols, which makes it impossible to provide them to developers. Some organizations have found it effective to locate their software development teams in separate facilities where they can provide developers with the productive environments they need. Others improve their environments incrementally by letting developers work occasionally in private conference rooms, wear head phones to cut down on distractions, or work at home. Organizations that cannot find a way to provide developers with a quiet, private environment free from interruptions will have no choice but to adjust their productivity expectations sharply downward.

User Involvement

User involvement is critical to a software project in several respects. Success in building software hinges on building a product that end users will use and like. Without end-user involvement, software developers are notorious for crafting technically elegant solutions to problems that users don’t care about. Marketers are notorious for overloading a software product with so many features that users can’t find the few features they really need.

There really is little magic involved in developing software products that users love. The project team simply needs to ask the users what they want, show them what they intend to build, and ask them how they like it—then listen carefully until they fully understand both the stated and unstated elements of the users’ responses.

It may take the project team several attempts to fully understand what the user is asking for. Users must also understand that the mock-ups developers show them to solicit their input are nowhere close to being the real product. Even identifying who "the user" is can be difficult. All these issues are discussed more in Chapter 8.

User involvement saves time because it eliminates one large source of requirements changes—feature creep arising from not defining the right product in the first place. If users aren’t involved early on, when they are brought in to review the product late in the project they identify ways in which the product that has been developed falls short of their needs. At that point the development team is faced with a difficult choice: Ignore the user input and adhere to the budget and schedule, or act on the user input in the downstream part of the project and throw the budget and schedule out the window. Typically a compromise is reached in which the easy user-originated changes are adopted and the hard changes are deferred to a later version. Clearly, it is better to involve users early on, while the software is malleable, and before a lot of work has been sunk into software the users don’t want.

Because of this dynamic of late-discovery-of-user-needs followed by compromise-between-user-desires-and-schedule-and-budget, projects that involve users early on tend to deliver products that are thought by users to have higher quality than projects that do not involve users early on. Externally, the software better fits user needs and expectations, which is one kind of quality. Internally, the software contains fewer defects because there have been fewer changes in direction and the product architecture, design, and implementation have been more stable. If the perceived user needs change throughout the project, internal software quality will degrade rapidly as developers are forced to cobble unforeseen functionality onto an existing framework that is ill-suited to support it.

In addition to early involvement, users need to be involved on an ongoing basis. Only rarely do software projects hit upon a really good solution on the first attempt. Typically, a software project must generate several versions of a user interface prototype before the users sit up and say, "Yes, that is the software I want." A good, mature user interface prototype can move a project most of the way toward delivering what end users want, but usability aspects of the software are inevitably refined as the software itself is developed. Building user-oriented checkpoints into the project helps the project to make a set of small, inexpensive mid-course corrections rather than waiting to make one large, expensive correction at the end of the project. (This book’s use of staged deliveries provides for such a series of mid-course corrections.)

User involvement doesn’t have to break the bank. In Usability Engineering, Jakob Nielsen points out that the benefit-to-cost ratio is highest when 3 to 9 usability testers are used.

In 1994, the Standish Group conducted a review of more than 8000 software projects. Their review concluded that end-user involvement was the most significant project success factor. Conversely, among projects that had failed, lack of user input was the most significant failure factor. Experts in rapid development of computer software have stated that ready access to end users is one of the most critical success factors in rapid development projects.

Involving users throughout the project is a critical software project survival skill.

Product Minimalism

Successful development of a software project requires a "less is more" orientation from requirements time through acceptance and release. Because software development work is so mentally taxing, it becomes critical that people working on the project take active steps to make the project as simple as possible rather than needlessly complicated. Feature specifications, designs, and implementation should all emphasize simplicity. Many developers are attracted to complexity, so their tendency is sometimes to make the problem more complicated rather than less. But success depends on finding ways to make the project more simple.

When developers look for the most straightforward ways to accomplish the project’s objectives, they will eliminate huge sources of errors, and it is even possible to influence this at the end-user level. There is often a 2-hour version of a feature, a 2-day version, a 2-week version, and a 2-month version. Developers should start with the 2-hour version. That will tend to be the simplest, most straightforward, and least error-prone version. After that has been implemented, if it isn’t sufficient, they can implement the 2-day version, and see if that’s sufficient. But they should proceed from simple to complex, and not the opposite.

The French writer Voltaire commented that an essay was finished not when there was nothing more to add, but when there was nothing more to take away. The general movement on a software project should be in the direction of taking elements of the software away to make it simpler rather than adding elements to make it more complex.

Focus on Shipping Software

Effective development teams are single-minded in their pursuit of releasing their products. One of the ways in which Microsoft has been particularly effective is explicitly focusing on the act of shipping products. Developers who see a product through to release receive a "Ship-It" award, which acknowledges them for driving a product to completion. Developers who have been with the company for many years typically have large collections of Ship-It awards. This simple practice emphasizes to developers that Microsoft doesn’t make its money by developing software; it makes its money by shipping software. That’s how most other companies that produce software make their money too.

Focus is just as important for business software developers who release software to their internal customers as it is for shrink-wrap developers who release software to the general public. A clear vision helps any kind of software development team focus on the product-release goal. If different developers carry different visions of the product they’re developing to the end of the project, a great deal of time, effort, and money will go into reconciling the differences in their visions.

A clear architecture can also help align the development team on the release goal. If the project team doesn’t create a good architecture, it’s hard to focus technically. If it does create a good architecture, that builds focus into the project at a deep technical level.

The software development team must be relentless in ensuring that every technical decision contributes toward minimally satisfying the system’s required functions. If a project team is working on an academic project for a university class, there might be some reason to make a product arbitrarily more complicated. But if a team is working on a commercial product, their mission is to provide the simplest, cleanest solution that solves the business problem. Any decision that is counter to these characteristics should be rejected.

One message of this book is that software development is inherently a functional activity that serves practical objectives. It has strong aesthetic and scientific components, but it is not itself art or science. The effective software developer realizes that software projects do not exist primarily to provide the developer with a high-tech sandbox, and prioritizes his or her activities accordingly. The Microsoft experience shows that this results orientation can be cultivated in project teams. The developer who does not share this focus on the bottom line is a drag on the project and ultimately of little use to the organization.