This book has been superceded by Professional Software Development. 

After the Gold Rush: Creating a True Profession of Software Engineering. 208 pages, Redmond, Wa: Microsoft Press, 1999. Retail price: $24.99. ISBN: 0735608776.

Software Dinosaurs 

"It looks obvious until you try it."
— IEEE Software

In the state of Texas, professional engineers were originally licensed in 1937 after a boiler explosion caused the deaths of more than 300 school children. The boiler part that failed in 1937 was mechanical. Today, that part’s function is handled by software. Good software has the potential to make our lives much better; bad software has the potential to make it much worse. The practices needed to create good software are well established and readily available, but there is a chasm between the average practice and the best. Consider these statistics:

• An aerospace company develops software for companies on a fixed-price basis. Their managers think they have problems: 3 percent of their projects overrun their budgets; 97 out of 100 are delivered on target. They didn’t know that the typical business systems project overruns its planned budget by about 100 percent and only about a quarter are delivered within 25 percent of their original targets.
• A team developing software for the U.S. Air Force committed to a 1-year schedule and a $2 million budget even though the most credible bid for the project was 2 years and $10 million. By practicing active risk management and sound fundamental development practices, the team delivered the project 1 month early. The software delighted its users and, one year later, only two defects had been found in operation. The project manager pointed out that the project used techniques that had been known for years but were rarely used in practice. In contrast, the average software project does no risk management at all and overruns its schedule by more than 100 percent.
• An organization was determined to shorten its schedules and focused on systematic process improvement. It attained an average of 23 percent schedule reduction per year for 6 years, for a total reduction of 91 percent. Most organizations have no process improvement program in place. In the worst organizations, productivity has been seen to erode from one year to the next.
• An organization that committed to achieving outstanding quality attained an average of 39 percent reduction in its post-release defect rate every year for a period of 9 years—a cumulative reduction of 99 percent. The average organization doesn’t even know what its post-release defect rate is.

The view from the top looks good, but the view from the vantage point of the average project leaves much to be desired, as the victims of many well-known software disasters will attest. Problems with the baggage handling system caused a delay of more than a year in opening Denver International Airport. Estimates of the delay’s cost ranged as high as $1.1 million per day. The FAA’s Advanced Automation System overran its planned budget by about $3 billion. The IRS bumbled an $8 billion software modernization program that cost U.S. taxpayers $50 billion per year in lost revenue. After topping $44 million, a long series of overruns forced California to cancel its motor vehicle registration system. The B-2 bomber wouldn’t fly on its maiden flight because of a software problem. The Ariane 5 rocket blew up on its maiden launch because of a software error. In Seattle, computer-controlled ferries caused more than a dozen dock crashes in the 1980s, resulting in damage worth more than $7 million. The State of Washington recommended spending more than $3 million to revert the ferries to manual controls.

Even software applications that aren’t considered to be mission-critical are being used in important applications. The project lead of Lotus Symphony received a call from a surgeon who was using Symphony to analyze patient data during open heart surgery. Newsweek magazine printed pictures of the Nicaraguan Contras planning operations using Excel on portable PCs . The Microsoft Excel technical support team received calls from the battlefield during Operation Desert Storm.

In the rush to get the current project out the door, software developers sometimes lose sight of the big-picture reasons their software development jobs matter. Whether we like it or not, software is important. It affects people’s lives. And the development practices most commonly used to produce software are not up to the task. As Fred Brooks pointed out more than 10 years ago, "The gap between the best software engineering practice and the average practice is very wide—perhaps wider than in any other engineering discipline."

It’s time for software development to grow up.

The Purpose of This Book

This book is about creating a true profession of software engineering. Significant developments are under way that will affect the careers of practicing programmers, including initiatives in education, professional development, certification, and licensing. Some of these developments are well thought out and positive; others are being forced and need to be improved before they're standardized. Software development is changing, whether programmers recognize it or not. Programmers who aren’t paying attention could easily find themselves working as twenty-first century software janitors. This book describes the occupation of computer programming as it exists today and the profession of software engineering as it can exist in the future.

Who Should Read This Book

If you develop software for a living, important developments are under way that will affect your career and your ability to call yourself a "software engineer." They apply to you whether you are a programmer, tester, requirements analyst, manager, or other software worker.

If you manage a software organization, you should know about the incredible potential of systematic approaches to software development—in other words, from software engineering.

If you are an educator, you should understand industry’s needs for professional software development—what is useful and what is not—and the elements of a complete software engineering curriculum.

If you are a student who wants to work in the software field, you will need to know about the body of knowledge that makes up the field of software engineering and what a career in software engineering will look like.

If you are a public policy maker, you should know how software development got to be the way it is now so that you can make sensible policy decisions related to licensing and regulating software, software developers, and companies that create software.

Key Questions

This book is organized as a set of essays. They can be read individually or together, and they are all related to the theme of creating a true profession of software engineering. They address the following questions:

• What is software engineering?
• Why isn’t regular computer programming good enough?
• Why do we need a profession of software engineering?
• Why is engineering the best model for a software development profession?
• Will software engineers have to be licensed? Will you have to be licensed?
• What does a software engineer have to know?
• What can you do while the software field is getting its act together?

This book focuses on software engineering in the U.S. and Canada. Many of the issues involved in establishing a profession are legal and cultural, and they vary among countries. To keep the narrative straightforward, I have deliberately maintained a North American focus.

Prospecting for Software Engineering

Most software development practices in common use today are seriously outdated and underpowered. This situation arises from lack of initial education, lack of continuing professional development, lack of time for personal improvement, lack of professional standards, treatment of software as a craft rather than as an engineering discipline, and many other reasons.

Practices exist today that make engineering of software possible. The best companies are already using them, and these practices provide the companies with formidable advantages. Industry researchers have long observed 10-to-1 differences in productivity between different organizations competing in the same industries. More recently, researchers have observed differences as large as 600 to 1. The most effective organizations are doing very well indeed.

The benefits of creating a true profession of software engineering are so compelling that, I believe, 25 years from now organizations that today resist the movement toward software engineering will be viewed the same way as farmers who resisted crop rotation or businesses that resisted the telephone.

Practitioners who protest that "software engineering" is not possible are generally ignorant, misinformed, or simply afraid to change. But the fact that software engineering is possible does not mean that it is easy. For software engineering to achieve the status of a true profession, many organizations will have to work together, including the engineering community, the software community, the academic community, accreditation agencies, and state legislatures.

"If it ain’t broke, don’t fix it," the saying goes. Common software development practices are seriously broken, and the cost of not fixing them has become extreme. Traditional thinking would have it that change presents the greatest risk. In software’s case, the greatest risk lies with not changing—staying mired in unhealthy, profligate development practices instead of switching to practices that were proved to be more effective many years ago.

How to change? That is the central topic of the rest of the book.

This material is (c) 1999 by Steven C. McConnell. All Rights Reserved.