Professional Software Development: Shorter Schedules, Better Projects, Superior Products, Enhanced Careers . 241 pages, Boston, MA: Addison-Wesley, 2004. Retail price: $29.99. ISBN: 0321193679. Buy PSD  from Amazon.com.

Chapter 6
Novum Organum

“A prudent question is one-half of wisdom.”
            — Francis Bacon

In 1620, Francis Bacon published the Novum Organum, a masterwork that challenged Bacon’s contemporaries to discard their ancient reliance on pure deductive reasoning and embrace a scientific method based on observation and experience. He imagined a new world of culture and leisure that could be gained by inquiry into the laws and processes of nature. In describing this world, he anticipated the effects of advances in science, engineering, and technology.

Bacon’s scientific method consisted of three steps:

• Purge your mind of prejudices—what Bacon called “superstition.”

• Collect observations and experiences systematically.

• Stop, survey what you have seen, and draw initial conclusions (the “first vintage”).

Novum Organum was part of a larger work called Instauratio Magna, which set out to organize the sciences, define a method of scientific inquiry, collect observations and facts, present examples of the new method, and define a new philosophy based on the results of this scientific work. This work so influenced modern scientific methods that Bacon is often called the Father of Modern Science.

The title page of Novum Organum contained an image of a ship passing through the Pillars of Hercules. The Pillars of Hercules were generally accepted to have stood on the sides of the Strait of Gibraltar, the sole passage between the Mediterranean Sea and the Atlantic Ocean. For the ancients, the Pillars of Hercules symbolized the limits of human’s possible explorations. Beyond the pillars lay the edge of the earth; the ancients had not been inclined to progress into those outer reaches and leave the old world behind.

Figure 11-1

This is the title page from Bacon’s Instauratio Magna, which contains his Novum Organum. The ancients were reluctant to sail beyond the Pillars of Hercules.

At this time, average software development practices are becalmed in a windless sea of code-and-fix programming—a kind of flat earth approach to software development that was proved ineffective 20 years ago. Leading software engineers have a clear idea of what lies beyond software’s Pillars of Hercules—software engineering has already had its Marco Polo, Vasco da Gama, and Ferdinand Magellan who have explored the new world of better software engineering practices. As Chapter 13 will describe, vast software riches await in waters that have been charted extensively but are traveled infrequently by average software practitioners.

Profession Defined

Considering the disparity between the best software organizations and the worst, the current challenge is not so much to advance the state of the art as it is to raise the average level of practice. The new world has been adequately explored; it’s time to start colonizing. The traditional means of raising the level of practice in a field, especially one that affects the public welfare, is through establishment of a formal profession.

Although the term is often used casually, the notion of a “profession” has a long and well-defined legal standing. The Code of Federal Regulations (CFR) in the United States says that a person employed in a “professional capacity” can be distinguished by several characteristics.

A professional’s work typically requires advanced knowledge in science or a field of learning that is acquired through a prolonged course of specialized study. The CFR distinguishes this from a general academic education. It also distinguishes it from training in routine processes, whether they be mental, manual, or physical. The CFR goes on to note that professional work can be creative and artistic. The work may depend primarily on the inventiveness, imagination, or talent of the person doing the work.

The CFR states that professional work requires the consistent exercise of discretion and judgment in its performance. The work is predominantly intellectual and varied in character. The CFR again differentiates this from routine mental, manual, or physical work.

Most software developers will recognize characteristics of their own work in the CFR’s description of a profession. The work certainly requires advanced knowledge (detailed technical knowledge, anyway), and it benefits from specialized instruction and study. Software development contains a significant creative element, and clearly calls for a great deal of discretion and judgment. In short, the work performed by software developers seems clearly to meet the definition of “professional work” as defined in the CFR.

The CFR contributes part of the legal definition of a profession. The body of legal precedents (court cases) establish a slightly different but complementary definition. According to legal precedents, a profession has:[1]

·   A requirement for extensive learning and training

·   A code of ethics imposing standards higher than those normally tolerated in the marketplace

·   A disciplinary system for professionals who breach the code

·   A primary emphasis on social responsibility over strictly individual gain, and a corresponding duty of its members to behave as members of a disciplined and honorable profession

·   A prerequisite of a license prior to admission to practice

How well does software rate according to these criteria?

In Search of a Software Engineering Profession

Gary Ford and Norman E. Gibbs of the Software Engineering Institute identified eight elements of a mature profession.[2] A professional’s development typically follows the progression shown in Figure 11-2.

Figure 11-2

Professional development follows most or all of these basic steps in all well-established professions.

Initial Professional Education. Professionals generally begin their professional life by completing a university program in their chosen field—medical school, engineering school, law school, and so on.

Accreditation. University programs are accredited by oversight bodies that determine whether each program provides adequate education. This ensures that, as long as professionals graduate from accredited programs, they will start their professional lives with the knowledge they need to perform effectively. The Accreditation Board for Engineering and Technology (ABET) oversees engineering programs in the U.S. and the Canadian Engineering Accreditation Board (CEAB) oversees engineering programs in Canada.

Skills Development. For most professions, education alone is not sufficient to develop full professional capabilities. Nascent professionals need practice applying their knowledge before they are prepared to take primary responsibility for performing work in their field. Physicians have a three-year residency. Certified Public Accountants (CPAs) must work one year for a board-approved organization before receiving their licenses. Professional Engineers must have at least four years of work experience. Requiring some kind of apprenticeship ensures that people who enter a profession have practice performing work at a satisfactory level of competence.

Certification. After completion of education and skills development, a professional is required to pass one or more exams that ensure the person has attained a minimum level of knowledge. Doctors take board exams. Accountants take CPA exams. Professional engineers take a Fundamentals of Engineering exam at college graduation time and then take an engineering specialty exam about four years later. Some professions require recertification from time to time.

Licensing. Licensing is similar to certification except that it is mandatory instead of voluntary and administered by a governmental authority.

Professional Development. Many professionals are required to keep their professional education current. Ongoing professional education maintains or improves workers’ knowledge and skills after they begin professional practice. Professional development requirements tend to be strongest in professions where there is a rapidly changing body of technical knowledge. Medicine is perhaps the most notable because of the constant improvements in drugs, therapies, medical equipment, and diagnosis and treatment procedures. After a professional’s initial education and skills development are complete, this requirement helps to ensure a minimum competency level throughout the professional’s career.

Professional Societies. Professionals see themselves as part of a community of like-minded individuals who put their professional standards above their individual self-interest or their employer’s self-interest. In the beginning, professional societies usually promote the exchange of knowledge, and over time their function evolves to include defining certification criteria, managing certification programs, establishing accreditation standards, and defining codes of ethics and disciplinary action for violations of those codes.

Code of Ethics. Each profession has a code of ethics to ensure that its practitioners behave responsibly. The code states not just what its practitioners actually do, but what they should do. Professionals can be ejected from their professional societies or lose their license to practice for violating the code of ethics. Adherence to a recognized code of conduct helps professionals feel they belong to a well-regarded community, and enforcement of ethics standards helps maintain a minimum level of conduct.

In addition to the eight elements identified by Ford and Gibbs, many professions exhibit a ninth characteristic that applies to organizations rather than specific workers:

Organizational Certification. In many professions, not only must individuals be certified, their organizations must be certified. Accounting firms are peer reviewed. Hospitals are accredited, as are universities. For fields as complex as accounting, education, and medicine, organizational certification is a response to the reality that individual competence is not sufficient to guarantee adequate levels of professional service; organizational characteristics can have as much influence as individuals’ characteristics.

Ford and Gibbs point out that many nonprofessional occupations exhibit some small number of these elements. For example, the state of California requires licenses for custom upholsterers, amateur boxers, private investigators, and mule racing jockeys, but it doesn’t require most of the other elements of a profession for those occupations. For all common professions, nearly all of the elements exist.

Ford and Gibbs defined several levels of maturity for each of the elements of a profession.

Nonexistence. The element simply doesn’t exist.

Ad Hoc. The element exists, but only in isolated, uncoordinated instances.

Established. The element exists and is clearly identified with a specific profession. (Ford and Gibbs use the word “specific,” but I find that confusing.)

Maturing. The element has existed for many years and is actively maintained and improved by some professional body.

A mature profession is one whose elements have reached the maturing stage. Of course, “maturity” is a moving target. Some of the specific elements that seemed mature 30 years ago don’t seem mature today, and some elements that seem mature today will not seem mature 30 years from now.

Table 11-1 describes the maturity of the software engineering profession. The profession is mostly in the Established stage, with some areas lagging and some moving into Maturity. More details about the entries in the table are discussed throughout this book.

Through the Pillars

Software engineering does not fully meet the definition of a profession at this time. Widely available initial education is just beginning to emerge. Certification became available only in 2002. Licensing is available to only a tiny fraction of current software workers. A code of ethics exists, but it isn’t enforced. Much work is being done, however, to accelerate the movement of software engineering into the Established and Maturing stages.

If we apply Bacon’s scientific method to software engineering, we can see the three steps we need to help software engineering reach maturity.

Purge your mind of prejudices. The software industry needs to kick its addiction to code-and-fix development, a prejudice that has a long track record of benefiting no one.

Collect observations and experiences systematically. A few organizations have begun collecting data on the effectiveness of their development practices and evaluating which practices make them most successful. Some have achieved dramatic results. Other organizations need to follow.

Stop, survey what you have seen, and draw initial conclusions. This book presents some initial conclusions.

Software development is at a major decision point. We can stay safe in our code-and-fix harbor, not venturing past the Pillars of Hercules and not achieving the significant gains that have already been discovered by software engineering explorers. Or we can boldly venture toward a new profession of software engineering and begin colonizing a new world of higher productivity, lower costs, shorter schedules, and better quality.

This material is (c) 2004 by Steven C. McConnell. All Rights Reserved.