|
This book has been superceded by Professional Software Development.
Chapter 1
|
||
|
He that will not apply new remedies must expect new evils In 1975, Fred Brooks compared the development of large software systems to dinosaurs, woolly mammoths, and saber tooth tigers fighting the glutinous grip of the tar pit. Brooks predicted that the software engineering tar pit would continue to be sticky for a long time to come. A quarter century later, the average software project provides proof positive for Brooks’ prediction. The problems that Brooks described twenty-five years ago were not new when he described them, and the software community has now had another quarter century to work on them. How much progress has been made? Many of the problems plaguing the typical software project today haven’t changed much. For example, schedule pressure is a common feature of today’s projects. According to some estimates, excessive schedule pressure occurs in about 75 percent of all medium-size projects and in 90 percent or more of all large projects. Overtime is more the norm than the exception. Internet startups are known for the long hours they expect from their employees, and stories of programmers sleeping under their desks abound But this isn’t a new phenomenon. As far back as the mid-1960s, one report stated that, "In many companies, programmers faced with deadlines have been known to spend nights in their offices." In 1975, Fred Brooks pointed out that "more software projects have gone awry for lack of calendar time than all other causes combined." Schedule overruns have been around for at least 30 years—and, people’s impatience being what it is—probably since time immemorial. Many people today complain about the shortage of qualified software developers. Experts estimate that North America is now experiencing a software personnel shortage of of about 10 percent. This is not a new problem, either. Thirty years ago, total employment was estimated at 100,000 programming jobs in the United States, and experts estimated that 50,000 additional jobs were available—a 33 percent labor shortage that makes today’s 10 percent labor shortage seem almost insignificant. The scope of today’s large software projects seems daunting, and a natural tendency is to think that no one ever attempted projects of the scope we now face. Yet even a huge project such as the initial development of Microsoft Windows NT has historical precedents. The initial Windows NT project required about 1,500 staff-years of effort, but the development of IBM’s OS/360, which was completed in 1966, required more than three times as much effort. Recent surveys have found that the most frequent causes of software project failure have to do with requirements problems—requirements that are too vague to be implemented, that contradict each other, or that change frequently and wreak havoc on the system design. But requirements problems are not new. Back in 1969, Robert Frosch observed that a system could "satisfy the letter of the specification and still not be very satisfactory." Modern developers rack their brains trying to keep up with the frenetic pace of change brought on by Internet development. How do you keep up with new languages, shifting standards, and vendors that release new products every few months? To those of us who were in the software world 15 years ago, this predicament sounds an awful lot like the mid-1980s when the IBM PC began to revolutionize corporate computing. When the Fortran programming language was developed in 1954–58, it was supposed to eliminate the need for computer programming—scientists and engineers could simply enter their formulas into the computer, and the computer would translate the formulas for them, thus the name FORmula TRANslation. Of course, Fortran didn’t eliminate programming; it just reduced the need for machine-language programming. From time to time we still hear about the promise of automatic programming—computers will become so advanced that the need for computer programmers will disappear. But this conjecture was already a well-polished chestnut thirty years ago when Gene Bylinsky reported that, "Predictions of businessmen blithely conversing with their omnipotent machines in plain English still get played up regularly in the press." The reality is that defining problems in painstaking detail is difficult work that can’t be automated. That aspect of computer programming will not go away. New tools are useful, but not a substitute for clear thinking. I made that point in my 1996 book Rapid Development; Robert Frosch had already made the same point in IEEE Spectrum 30 years earlier. Internet developers talk about development in Internet time. The Internet makes it possible for developers to roll out revisions to their programs with unprecedented ease. CDs and DVDs don’t have to be duplicated; users can download upgrades electronically, making, delivery of upgrades quick and inexpensive. This ease of distribution contributes to pressure to release upgrades frequently in response to user requests. Internet developers say that users would rather get the software quickly than have it be perfect. Users will tolerate low reliability. According to Internet developers, "It’s better to be first than right." How unprecedented are these dynamics really? Some Internet developers think these dynamics are unique to web projects, but industry old timers know better: low rollout cost, easy corrections, low cost of failure—this sounds like a good old-fashioned in-house mainframe production environment. The common threads tying together 25 years of software development—schedule pressure, staff shortages, large projects, faulty requirements, volatile technology, and even development in Internet time—are a source of both comfort and despair. The despair arises from the fact that some problems have been with us for a quarter century or more and are still common. We truly have been stuck in the tar pit a long time. But we’ve been staring at the same problems long enough to recognize some patterns, and therein lies the comfort. Problems that once seemed intractable have begun to seem less difficult, and, as I intend to show throughout the rest of this book, we are now on the brink of fixing them. This material is (c) 1999 by Steven C. McConnell. All Rights Reserved. |
Email me at stevemcc@construx.com.