McConnell: - Steve McConnell
Audience: - (Questions)
McConnell: Its nice to be here to talk today about the Software
Project Survival Guide. I always like talking about software development to software
developers because I think software developers have a really unique way of looking at the
world. As an example of what I mean, you might have heard about, the time a boy was taking
a walk in the woods. He came across a frog. The frog said "Kiss me and Ill turn
into a beautiful princess." The boy picked up the frog and looked at it and smiled
and put it into this pocket. The frog said "Kiss me, Ill turn into a beautiful
princess, and Ill stay with you for a week." The boy pulled the frog out of his
pocket and looked at it and put it back into his pocket. The frog said "Okay,
Ill stay with you for a week and Ill do whatever you want." The boy
patted the frog and smiled and finally the frog said, "Look, whats it going to
take? Ive told you if you kiss me, Ill turn into a beautiful princess,
Ill stay with you for a week, and Ill do whatever you want. Why wont you
kiss me?" The boy said "Look, Im a software developer, I dont have
time for a girlfriend, but a talking frog is cool."
Well... there are a lot of projects that Ive seen lately that havent been
so cool. Ive been trying to figure out how to talk about the Survival Guide.
What Id like to do this afternoon is to describe a project that I was involved with
as an observer over the last couple of years, and use that as a jumping off point for
discussing effective software development practices and ineffective software development
practices. You should have a handout that was on your chair when you sat down that
contains a list of top dos and donts. This is summarized from the last chapter
of the Survival Guide. As I go through the case study, what Id like you to do
is look at that list of dos and donts and bounce that off the case study and
also bounce it off your own experiences. And after I describe this project that I
observed, what I would like to do is open up the floor for discussion and talk about what
could have been done on that project to make it run more smoothly.
The specific project Im thinking of was Version 2 of a successful vertical market
software application. Version 1 had sold quite well and the company that was developing it
decided that they would like to go ahead and release Version 2. Unfortunately, they did
not really have the resources they needed to develop Version 2 in house. So, they decided
that they would like to put the project out to bid, get some proposals from vendors and
then have a vendor do the development work for them. The first thing they did was to
create a statement of requirements. That was a fairly detailed statement of requirements.
It was about 50 pages long, I guess. And then they held a vendor meeting to answer
vendors initial questions and 10 vendors showed up that meeting. After holding the
vendor meeting, a few weeks went by and at the proposal submission deadline, five vendors
submitted proposals for the project. Out of those five proposals, there was one proposal
that came in at around $300,000...
I should back up and say one more thing about the proposal meeting. At the proposal
meeting, the client did not mention what his budget for the project was, but they did say
that they saw as a relatively minor upgrade to Version I or incremental upgrade to Version
I. They said theyd really like to have the software delivered within five months,
and that they had a strong preference for a fixed price bid. Although they would accept
time and material bids if the vendors absolutely felt they had to provide a time and
materials bid rather than fixed price.
So the proposal deadline came and five vendors ultimately submitted proposals. The low
bid was $300,000. There were three bids in the $400,000 range, and then there was a fifth
bid that was at $1.2 million. The client took a look at that set of bids and threw out the
low bid and the high bid with the expectation that since those were on the extreme ends of
the scale, they were statistical outlieroutliers and therefore werent valid. They
took a look at the bids in the middle and selected the least expensive bid, which was
right at $400,000, and then negotiated the vendors price down to $350,000, and they
got started with the project.
I guess I should also mention that of those bids, the four lowest bids, all said that
they could get the project done in five months. The fifth bid, the high bid said the
project would take a year, and all five of the bids turned out to be time and materials
bids, rather than the fixed price bid that the client had asked for.
So the client selected the vendor and got the project underway. Five months came up
pretty quickly on this project and at that point the vendor was nowhere near finishing the
software, and the client really was not all that concerned about it. They felt that they
had a market window of about eight months. The vendor said they were quite certain that
they could deliver the software within eight months, and so they rescheduled for eight
months and worked toward that. Well, they got to eight months and the vendor still was not
very close to shipping the software. So they took a six-week schedule slip, after that
they took another six-week schedule slip, after that they took another six-week schedule
slip, and so on. Finally, after 14 months, the vendor delivered about 25% of the
originally planned functionality at a cost of 1.5 million dollars.
When you consider the combined effect of the cost of schedule overrun with the
functionality under-run, in real terms this project overran its schedule by about 1000%
and overran its budget by about 1400%.
So what Id like to do now is ask you if anything jumped out at you about this
project description, either from your own experience or maybe from that
list of dos and donts that was
handed out at the beginning of the talk about what might have been done to make this
project go better. And if there are details that you need to get a little bit more
explanation about, you could certainly feel free to ask about those too.
So who noticed something?
Audience: Well, if they just used the cost and didnt do
any other homework, didnt they kind of get what they deserved? Isnt that kind
McConnell: Yeah, I agree with that. I think, in this case,
the vendor made the selection based on choosing--mostly on choosing--the second lowest bid
after throwing out the first lowest bid.
Whats really crazy about that is that these were time and material bids. The
difference in the bids was really accounted for by different expectations about how much
work was involved in the project. It wasnt based on the fact that one vendors
rates were 25% lower than another vendors rates, it was based on the fact that they
thought there was 25% less work there. So yeah, I think choosing the winning vendor based
on the dollar value of a time and materials bid was not a very good idea.
Audience: Lack of internal milestones.
McConnell: Yes! "Do" number 5 is "Take
periodic snapshots of project health and progress and re-plan when necessary." In
this case, the client really didnt get any kind of progress tracking other than the
fact that the vendor missed its ultimate delivery of the software. There was really no
tracking of the project on an ongoing basis.
I happen to know that for a fact, but even if I didnt know it for a fact, I think
that I would be willing to bet with almost 100% certainty that that was true for the
following reasons: Suppose Im a client and I have a vendor working on a project and
Im tracking it carefully and I get to the point in the project where I say,
"Okay, it really looks like this vendor is having some problems. Theyre going
to deliver the software 100% over budget. 100% over budget -- thats not very good,
but we dont really have any choice at this point. I guess were just going to
have to live with it." I can conceive of a client going through that line of
thinking. What I cant conceive of is a client that says, "Were tracking
this project carefully and it really looks like its going to go 1400% over budget, I
guess thats the best we can do." I simply cant conceive of a client going
through that thought process, so the conclusion I would have to draw is that they simply
werent tracking the project at all, or at least not in any meaningful way. And we
can see obviously the damage thats caused by that.
In this particular case, in the winning vendors proposal there were some initial
tasks that were described and that were estimated at a fairly low level of detail for some
of them. And the initial work that was done on those tasks within the first six to eight
weeks of the project overran the estimates by roughly a comparable proportion to the
overrun of the entire project. And so, if the client had been paying attention, they
really could have had a pretty good idea six to eight weeks into this project of the
ultimate project overrun magnitude.
If they had simply been paying attention ... I think with my background, I would have
been very confident at eight weeks saying this project is way off the rails, way out of
alignment with the initial estimates. And I think even someone who hasnt gotten into
this stuff as much as I have would have been very confident. Even if they werent
confident at eight weeks, they would have at least put the project on probation, or on
alert, after eight weeks. And I think by the end of 12 weeks, they shouldnt have had
any doubts about where the project was headed.
Audience: I think that the problem was when they described
the high bid at the beginning as though it was an outlier, like there was some reason that
that one vendor would estimate that the project would take 1.2 million dollars. They
should have figured out what the doubts were that that person who submitted the high bid
had, and then asked the other vendors whether they had addressed those doubts.
McConnell: Yeah, in this case, actually there were a couple
things going on. One is outliers, when you throw out outliers, you typically do that
because youre looking at random events. Its a statistical idea that applies to
random events, and these proposals were not random events. So this approach of throwing
out "outliers" was really not a valid.
In this case, the high bid actually had another interesting characteristic, which was
that that particular vendor had hired one of the programmers who worked on Version 1 of
this software, and that developer was involved in creating that high bid. None of the
other vendors had any particular expertise with that software and so I think the client
was really doubly to blame in this case. One for not really understanding what an outlier
was and the other was for ignoring the fact that the best-informed vendor created the
That doesnt mean that they had to accept the highest bid, but I think what it
does mean is, if they want to contain their risk on this project at all, then they really
ought to take a hard look at why the vendor that knows the most about this software is
bidding the highest amount? What is it seeing that the other vendors dont see? And
that should have created an opportunity for the client to go to the other vendors and say,
Have you considered factors x, y and z, and how does that affect your bid? That
wasnt done with this project.
Audience: It is somewhat surprising that all of the vendors
bids came in so close to each other except the really high one.
McConnell: I think
its hard to look at that without being a little bit cynical about the bidding
process. The client had not said, "We have a budget of $350,000," which was in
fact their budget. However, they did say, "We want this done in five months, we see
it as a relatively small upgrade to the product," and I think that there were enough
clues given that the vendors came in with bids between $300,000 and $400,000. So they
actually did end up zeroing in on the clients budget pretty well. And I guess that I
really am pretty cynical about that process.
I think that there almost had to be an attempt to guess the clients budget and
then reverse-engineer the proposals based on what they thought the budget was. If I wanted
to be a little less cynical about it, if I wanted to give it the most charitable possible
explanation, then I think what I would guess is that the vendors all said, "Well, the
client knows this software a lot better than we do. They say it can be done in five
months, who are we to second guess the client when we dont know as much about it as
You cant exactly start work on a new project, staff it up to 25 people instantly
and work effectively. I think that given the description by the client, it was almost
inevitable that the bids would come in that way.
Yeah, so there is some convergence among the estimates. I dont think it was for
technical reasons, I think it was more for sales and business reasons. And I think that is
one of the reasons why the client ended up taking the bid it did. The client had a budget
of $350,000. They hadnt gone through any very careful estimation approach to come up
with that budget number. They just kind of said, "Well, what do you think it will
take to develop Version 2? Well, I think we could get $350,000 from the budget committee.
Okay, well, lets try to do it for that amount." And that was the thought
process they went through. I think that when the vendor bids came in in that same
ballpark, the psychology on the client part was kind of interesting. Which is the people
on the client said "Hey, you know were pretty smart. We didnt even work
that hard on estimating this. It looks like we were right."
The client ended up confusing cause and effect.
Audience: How did the
client pick the feature lists? Was there a lot of feature creep on this project?
McConnell: I dont know that I would say the feature
list was out of bounds. I would say the vendor, who was selected and started out working
under the assumption that the project was a lot smaller than it was, worked very, very
inefficiently. And so the client got only 25% of the functionality for 1 1/2 million
dollars. I think they paid an incredibly high price for the functionality they got. I
wouldnt necessarily say the feature list was outlandish. I think the feature list
might have been excessive, but I think probably the high vendor was in the ballpark. It
might have been off by 25% or 50% for delivering the whole functionality, but I think
theres a big difference between starting out a project with a realistic idea of what
the true scope of the project is versus having to adjust your approach midway through the
project when you realize its bigger than you thought at first.
One of the key ideas I try to get across in the Survival Guide is the idea that
software is a little unusual in that if you dont do it right the first time, a lot
of times you dont get a chance to fix it later. I should make clear what I mean by
that. If you start out developing relatively low quality requirements, low quality designs
and so on, you can get a chance to fix the software. That is, you can rework the
code and bring it up to some level of quality with testing and so on. Sometime you can get
the defects out of the software. So in that sense, yeah you get a chance to fix it.
But what you dont get a chance to fix is the project itself. We have this
phenomenon in software where if you have a requirement defect, lets say, that might
simply be a defect in a one sentence statement of requirements. But if you can change that
at requirements time, all you have to do is change that one sentence. But if you let that
stay in the project, in the flow of the project, that can turn into a handful of design
diagrams, it can turn into several hundred lines of code, dozens of test cases, training
of end user support personnel, online documentation and other kinds of documentation, help
screens, paper documentation, and so on.
Clearly, if you have a chance to correct the error when its a one sentence
statement of requirements or a handful of design diagrams, that is an awful lot cheaper
than if you have to rework hundreds of lines of code and dozens of test cases and dozens
of help screens and so on.
So in a software project, one of the things that we find is if you dont get it
right the first time, that is requirements time or design time, you can fix it later on --
maybe. Maybe you can fix it later on, but youre going to fix it only at very great
cost. So you can fix the product later, but you cant fix the project.
Its going to be very, very expensive, and you could have done it much more
efficiently if you had applied that effort at the right time in the project.
Audience: Were the vendors fully aware of the whole feature
list before they made their bids and sort of a corollary to that, was the customer really
honest about what it was they were expecting? And if not, were they being actively
misleading or merely ignorant?
McConnell: The customer put out a 50 page requirements
document. I think there was actually blame in both camps in this case. There was a 50 page
statement of requirements, and I have a hard time seeing how any of the vendors bid as low
as they did based just on that statement of requirements. Having said that, I think that
the client actually did in this case take many opportunities to try to persuade the
vendors that the project was a small project, an incremental release or an incremental
improvement over Version 1 -- that kind of thing. And I cant tell you whether the
client was being actively dishonest or whether they were just engaging in really active
I think they wanted it to be a small project, they wanted it to cost $350,000, they
wanted a vendor who would give them a fixed price bid for $350,000, and things just
didnt work out the way they wanted.
Audience: Id be interested in knowing why the client
didnt take stronger action after 5 months, when they saw the vendor was having
problems. Did they do any kind of major status review? Why did they accept the
vendors revised estimate of 8 months?
McConnell: I think really the answer is the clients
evaluation of the vendor kind of went like this: "We like the programmers who are
working for the vendor. They seem like theyre honest. They seem like theyre
trying hard. They seem like nice people, and so if they say they can get the project done
in eight months, they probably can."
I think that was the extent of the review on the client side. And I think all those
things were true, at least my observation was that the people working on the vendor side
were sincere, they were trying hard, they were being honest and they were nice -- and they
were in way over their heads. That was the real problem.
Audience: Was there any background check, or references, with
McConnell: I really dont know the answer to that.
Vendors were required to submit references, but I dont know whether the client
actually checked the references.
Audience: Did the vendor do any additional work to try to get
a better idea of the scope of the project -- additional requirements work, or anything
McConnell: One of the interesting thing about this project
was that the winning vendors proposal had suggested an initial prototyping and
further specification phase, and the client crossed that out of the proposal and said,
"We dont want to spend the extra money for that."
We can see how well that idea worked out.
Audience: Wasnt this really some kind of a maintenance
project? Did the vendor take that into account?
McConnell: The answer to this question goes back to the
earlier question about what kind of expectations did the client try to set about this
project. In the proposal materials in this case, the client actually said that there were
certain sections of the old program that would simply be completely thrown away and
rewritten, and the other sections of the program would not be changed at all.
To me, that sort of thing never happens. You can hope that it happens, but unless
youre on the eighth generation of your software and you know that the subsystem
interfaces are extremely pure, that kind of approach is really a pie in the sky wishful
thinking. And to the extent that the client said that, I think that that was, at best, an
irresponsible statement and I think at best it was extremely wishful thinking. At worse,
it was misleading the vendors.
To the extent that the vendor believed it, I think that theres some blame to be
had on the vendors side too because I think anybody who has been through this sort
of activity a few times would have to be pretty skeptical about that kind of a claim. And
one of the interesting facts about this, which goes to the same point, is that a couple of
the vendors requested source code so that they could see what the quality of the code was,
and the client refused to provide that. Which again, I think on the vendor side it should
have been a pretty big warning sign. .
Audience: Are this clients project management skills
any better now than they were?
McConnell: Well, who do you think the client blamed for all
the problems in this case?
Yeah, the client laid 100% of the blame at the feet of the vendor. This was the
clients second attempt to outsource a project. The first attempt had gone about the
same as this one, although it was on a smaller scale. I think the client was very
frustrated with the vendors poor performance in this case, including their poor
management performance and their poor ability to track progress. In some sense, I think
the clients attitude there is justified, but I think this is one of those cases
where the clients attitude throughout the project was, "We didnt have the
technical resources to develop this software, thats why we outsourced it. It
doesnt make sense for us to also practice really active management of this project.
If we have to do that, why do we bother to outsource the project in the first place?"
From kind of a moralistic point of view, I can see that that line of reasoning makes
sense. But from a practical point of view, youre just asking for trouble if you take
that line of reasoning. You can say who should do all this work as much as you want
to, but if your main concern is getting software of a level of quality you want, when you
want it, you have to take on that responsibility yourself, even if it isnt
"right" or it doesnt seem fair that you should have to do that work.
And one of the points I make specifically about third party relationships like this is
that it certainly may reduce the amount of management work involved, but I think it
increases the degree of management sophistication required
to effectively keep tabs on the project.
In this case, I think the client side management really needed to insist at proposal
time on some very tangible project tracking mechanisms so that they didnt have to
just rely on this gut feeling about whether the vendors were good guys or not, but could
actually have tangible evidence of progress all along throughout the project. And that
just wasnt so. So Im not sure the client learned anything about project
management on this project, and, interestingly enough, this project, among other reasons,
basically caused this client to say, "We dont want to be in the software
business anymore." That wasnt their main line of business, so they are in the
process of completely getting out of it.
Audience: What kind of project management team was put
McConnell: What kind of project management team was put
together? Was it experienced developers or was it just a bunch of developers who were told
to go off and make this work?
On the dos and donts list, do #9 is start the project with a small senior
staff. One thing thats kind of interesting about this project is that the two main
developers on the project were ages 35 and 40. So you might think that that would
constitute a small senior staff, but the 40 year old had gotten his Ph.D. in English and
had kind of made a career transition to software, and this was the first project he
actually got paid for writing software for. And the other guy was a more experienced
software developer, but hed been working mainly in scientific systems and so this
was his first shrink wrap application. I dont know whether he was a good scientific
systems programmer or not, but in terms of the unique demands for mass distribution,
end-user-intensive kinds of products, he was kind of a rooky. So they started the project
with a small staff, but I think it really wasnt a senior staff. And that went for
the project manager too. He had managed some large projects, but they had been in-house
client/server kinds of projects and so I think if you didnt really appreciate some
of the differences between in-house software and commercial quality software, you might
think that the team looked okay. But if you understand that there is a pretty big
difference in robustness and compatibility issues and that kind of thing, I think
youd see that this team was pretty green.
Audience: I have a question about what should the vendor have
done at about five months, should they have really taken a hard look and tried to
accurately reschedule or should they have done what they did because the client might not
have accepted a really huge overrun?
McConnell: I think this is one of those cases where you get
to see what the character of the vendor is. Theres a really odd, odd interaction in
this case between dishonesty and incompetence I think.
There is active dishonesty where you get to five months and you say, "we know that
this project is going to take nine months more, were just going to tell the client
that its only going to take three months more." Thats active dishonesty.
And then theres simple incompetence where people take a really hard look at it
and they say after taking a really hard look at it, "we are 100% sure that we can
deliver this in three months."
And theres this kind of gray area where they just dont really want to know
that its going to take, they think they might not like the answer if they find out,
so they just dont try very hard to find out.
And so I think that this case was somewhere in that gray area. I think given the nature
of the initial bid of five months, if somebody was basically competent to start with, they
probably wouldnt have bid that five months in the first place and so they
wouldnt know that they should announce a 200%, well 300% slip when they got to the
five month mark. So I see it as a case where its hard to address the hypothetical
really directly because it asks you to assume competence thats not in evidence.
Audience: I mean if they were competent, what should they
McConnell: I really dont know how to answer that
question because if they were competent, they wouldnt have given the really low bid
in the first place.
If I can restructure your question a little bit. What if they brought in some really
expert consultant who took another look at their requirements and at that point, they got
a really clear idea that they were looking at a really huge project and massively missing
their deadline. Realize its not nine months more at that point, they only delivered
25% of the original functionality. So at that point to come up with a whole project
estimate, I mean we can assume theyre talking maybe 15 to 24 months for 100% of the
So what should they have done at that point? I guess I draw a pretty hard line there. I
think they shouldve had a meeting with the client and said, "We goofed up. We
did a bad job in the proposal. Were to blame, now we see that this project is a lot
bigger." If they really felt the client had actively misled them, I think they could
try to share some of that blame with the client.
I dont see this as a gray area. I think they just need to lay it out
for the client and then try to work together with that. I think in this case and in a lot
of other cases too, the client would have recognized that there wasnt perfect
information at the beginning of the project to estimate the whole project. And of course
the client is not going to like hearing that news, its going to be awful news,
its not going to be well received. But I think in any case, you can convince the
client or at least make the case that its better to get the bad news now than it is
to get it later, and if the client blows up and is completely unreasonable, then you may
be better off without the work. So I think thats what they should have done assuming
they had the expertise to do it.
Audience: What was the user
involvement? What kind of beta test cycles were cycles were involved and were there any
McConnell: The user involvement I dont think was a
really big issue. This was Version 2. I think the client company had a pretty clear idea
of what functionality users really wanted in Version 2, so the bad requirement stuff
wasnt really an issue in this case. As for Beta testing, they went through some kind
of a thing with a handful of in-house users of the product. They didnt do anything
you would normally think of as beta testing. And as far as quality practices are
concerned, as we see so often in cases like this, the system testing cycle got truncated
because they needed to get the software out into the marketplace.
And as a result, I dont know if anybody would be really surprised at this, but my
understanding is that the software that was released was very buggy and unreliable, and
they began working on a maintenance release almost immediately. User satisfaction with the
functionality was pretty good. User satisfaction with the reliability was not good.
Audience: What are some of the things you think killed their
productivity? It sounds like one problem is that if they try to go out with this ambitious
list of features and then five months later, they cant do all that and they throw
away a lot of it, theyve wasted a lot of time on stuff that is eventually going to
be thrown away.
McConnell: What do I think killed their productivity? I think
the staff was probably incapable of doing a good job on this project. Their two people
with the most responsibility were very junior. They had very little Windows experience.
They had very little or no shrink wrap experience. They had basically no experience in the
domain area of this application. If I had been brought in at that early stage of the
project, I would have said the aggregate risk level on this project is simply
insurmountable. And so I think in some sense, the die was cast very early in that project.
Personnel issues were a big, big deal on this project.
In terms of process issues, they didnt really have much of a project
plan there and the project plan that was stated in the proposal was so far out of
alignment with the real scope of the project that they essentially abandoned the plan and
let the project run free form. So they didnt get the benefit of any kind of planning
either, and so they had bad personnel, or ill suited personnel, bad process. There are
lots and lots of reasons why there was basically 100% chance this project wasnt
going to come out anywhere near what they thought it would.
Audience: Companies are
always dealing with make or buy decisions. I assume the client had their own in-house
staff produce Version I. Why didnt they just produce version 2 in house?
McConnell: The answer to that is an interesting case of a
medium size mistake giving rise to a big mistake. On Version 1 of the software, the client
had put together a team that included four contract programmers and one in-house
programmer and after Version 1 was completed, the four contract programmers scattered to
the winds and a few months later the one in-house programmer left the company. So the
client company really didnt have any in-house programmers who had any familiarity
with Version 1. And I think that was a really fundamental mistake they made on Version 1.
This was a fairly key product for them and if you look at the fact that its a key
product, it really doesnt make much sense to have four contractors and one in-house
employee developing all this knowledge about this product. I think they should have done
either of a couple things. Either initially they should have staffed Version 1 with more
of their in-house programmers, or at the end of the project, if they didnt realize
it ahead of time, when they realized, hey weve got this strategic technology,
its been developed with four contract programmers and only one in-house programmer,
this is a big risk for us. They should have tried really hard to convert the key
contractors to employee status so they could keep that expertise in-house. Just from a
risk point of view, there were some big risks that hadnt been addressed in Version 1
that ended up giving rise to other kinds of problems in Version 2.
Any thing else jump off that list of dos and donts at you?
Audience: Is there any evidence that the vendor actually
tried to tell them that they were incompetent?
McConnell: That the vendor was incompetent?
Audience: Yeah, not capable of completing the work, once they
got into the project and understood it better? Did they try to tell the client they were
in over their heads?
McConnell: Ive seen the same thing you have where the
client wants so much for the vendor to be able to do the job at the price the vendor said
that they will tell the vendor, in essence, "you are qualified to do the job even if
you think you arent."
In this case, I think it was actually even worse than that because the vendor had
actually proposed as its main proposal an implementation in Visual Basic, and they had
said "Oh and by the way, if you really want to do it in Visual C ++, then for an
extra $50,000, we can do it in that." And much to their surprise, the client said
"Well we do really want to do it in Visual C ++, well pay the extra $50,000 for
it." Well the vendor had made its proposal based on the strength of some specific
people in Visual Basic, and all of a sudden they found themselves scrambling to try to
find some people who knew Visual C++.
And whats really awful about this project, which is where I really became
intensely aware of this project, is the vendor called me on Thursday afternoon and said,
"We have our first meeting with the client on Monday, do you know where we can get
three experienced C++ programmers?" And I said "No, I dont know where to
get that." The situation was really dicey. I actually ended up talking with my
companys attorney about that because it seemed to me at that point there were pretty
good signs that this was a train wreck waiting to happen and I felt like somebody ought to
say something, but my attorney said otherwise.
Audience: Did the requirements stay pretty stable or was
there a lot of creep and moving around?
McConnell: The requirements were basically stable. If you
were to say, "describe a typical project kind of like this one," I would
probably include requirements creep as one of the typical characteristics of this kind of
project. But in this particular case, they really got lucky on that one and there really
werent any significant requirements changes.
Audience: 50-pages of requirement sounds like a lot for a
McConnell: Yeah, I think that 50 page document included
upwards of 400 individually enumerated requirements, and they werent specified in
amazing detail either. One of my friends talks about "the squint test." The
squint test is squint your eyes and look at something and say "does this even look
like it is possible, its remotely right?" And I dont think this passed
the squint test.
Audience: Given that the vendors submitted time and materials
bids, why did the client even look at the total prices? Isnt that meaningless for
time and materials contracts?
McConnell: Yeah, absolutely. Estimates are meaningless in a
time and materials context because the difference in the price is the difference in the
estimate of the amount of work involved.
There are a couple issues here. One is that the client indicated a strong preference
for fixed price and all the bids came in time and materials. That should have been a
warning sign that caused the client to take a look at whether they really understood the
scope of the project. The vendors are saying in pretty clear language, "Our estimates
involve enough risk that were not willing to take on that risk in the form of a
fixed price contract. In fact, were not even willing to apply a big multiplier to
how big we think the project is and do a fixed price." And that, to me, is pretty
clear in contract-speak that says "We dont have any idea how big this project
is." So that was one warning sign.
Another warning sign was the spread in the estimates. There was a factor of four
difference between the $300,000 estimate and the $1.2 million estimate. Thats
another warning sign the client should have paid attention to and didnt pay
attention to. And there are all kinds of things the client could have done in that case.
One thing the client could do that applies as much to in-house projects as outsourced
projects is to re-estimate system size effort and schedules periodically. The client
essentially is in a position at that time to know that it doesnt have a really clear
estimate. And what it should have done is try to set up the project in such a way that it
could re-estimate fairly quickly and get a better idea of how big the project was. It
couldve gone to something like a two phase acquisition where it gives out a fairly
short project of one or two months, the purpose of which is to develop information on the
real scope of the project. And then they estimate that second part of the project
separately. That would have been one thing they couldve done and I think there were
really clear warning signs in this case that something like that was needed. I guess
thats what I wanted to say about that.
We have time for a couple more questions and then well wrap things up.
Audience: Who wound up
doing the maintenance version?
McConnell: At the end of the project, there were very unhappy
people on both the client and vendor side, there were threats of litigation on both sides,
both the client and vendor ended up walking away with just a bad taste in their mouths,
and nothing really happened. To the best of my knowledge, in-house staff ended up doing
the maintenance. I'm not really sure.
Audience: Did the client do anything to try to build a team
spirit? Im thinking in this context if I screwed up and I know this is strategic,
Im going to try to bring the external vendor in close and be real buddy, buddy with
them. So that way even I dont have a project plan, I at least have a relationship
with the vendor I can turn to if things start to go sour.
McConnell: I think thats an excellent point. That idea
of team spirit can apply to just the developers; or to developers and QA; or developers,
QA, and management; or to client and vendor relationship.
In this case, I think that thats right. If the client lacks the technical
expertise to track the project really carefully, they at least shouldve tried to
stay in close communication with the vendor so that they could have subjective sense of
how well the project was going. I dont think that subjective sense is by any means
the ideal, but as a fallback position, in this case that would have been a lot better than
what they did. In this case, the client and the vendor were in cities about 60 miles apart
so there was a geographic barrier between the client and the vendor that I think
contributed to the problem somewhat. In that case, the client really shouldve been
very active in making sure that they called the vendor frequently, scheduled interactive
meetings and that kind of a thing.
Audience: What kind of staff did the client put on the
project or did they just have the vendor go away and come back in five months?
McConnell: The client had somebody designated as the project
manager, but as I said, the project manager really didnt do much and even if I
didnt know for a fact that the project manager didnt do much, just looking at
the outcome of the project makes it clear that the project manager didnt do much or
didnt have the remotest idea of how to do much on this particular project
Audience: Did the vendor try to educate the client at all
about what the project was going to look like?
McConnell: Keep in mind that the people on the vendor side
were pretty inexperienced as far as this kind of software project went. I guess I
dont know the exact answer to that question from my information about the project.
My expectation would be that, given the experience level of the people on the vendor side,
is that really didnt happen. Number one, because they didnt have the expertise
to share and number two, because if they did, they probably wouldnt have been far
enough down the road with it to know that it was important to share it.
Audience: Some of the key features are probably defined by
the code and if you cant look at the code, what basis do you have for even knowing
whether you should do the work?
McConnell: On the vendors side, if I were bidding
on that project, to me that would have been a deal breaker...basically when I saw hat
answer, I wouldve said, "I cant bid on this project." Either that
or, I think that might have contributed to the time and materials bids rather than the
fixed price bids because youre right, you might end up deciding that the code is
garbage and you have to throw all or some of it away. And so, yeah, I think that that has
to play a big factor. Assuming that for some reason you have to bid on a project, I think
at the very least you have to identify the fact that you dont know anything about
the code base as a big risk. And there are other things they didnt know about too.
Like, is there any high level design documentation? Somebody hands you 150,000 lines of
code with no design documentation, its going to take quite a while to get to a point
where youre going to be able to work with that. So yeah, I would identify it as a
very high level risk item.
Audience: What is the main problem? What is the one thing
that they should have done differently?
McConnell: This gets to a point that I dont mention in
the Survival Guide, but I try to mention emphatically when I talk about Rapid Development,
which I talked about in the Classic Mistakes chapter. A lot of times when people are
trying to develop software effectively, theyre looking for the one or two things
that they have to get right. But there are so many things that can go wrong in a software
project that Im convinced that the key to success in software is not so much getting
a few things right, but getting almost nothing wrong. And in this case, they had mistakes
lined up. And if they had eliminated the first half dozen mistakes, they still
wouldve had a project that was very much at risk. And the outcome shows that. They
struggled to get 25% of the functionality delivered. So I cant answer your question
directly. I dont think there were any one or two things that wouldve made a
Audience: In that case, would it have been easier to start
from scratch or evolve the existing code base? This is a common issue. Is there any
McConnell: Right, this is common issue, do you evolve the
existing code base or do you start over? I have a couple of reactions to that. One is that
there has been an interesting data point that was created by the NASA Software Engineering
Lab. Their rule of thumb is that if you have to modify more than 25% of the code in an
existing code base, that its more efficient to rewrite it from scratch than it is to
modify the existing code base.
The other phenomenon that I see pretty often is that companies get naturally very
attached to their existing code base and theyre very reluctant to throw it out and
start over, and so what very often happens is that they always do one more major release
on any given code base than they should have done. The last release on any given code base
almost always involves very brittle code, code thats hard to extend, code that ends
up being very defect-prone, and people end up throwing it away one version too late. And
so I think I would probably try to tilt the balance toward rewriting rather than keeping,
unless you have code that you know to be very reliable.
Audience: After all is said and done, what has the client
learned? Will they do it with their own staff next time? What will they do differently?
McConnell: The comment is what has the client really learned
in this case? Is Version 3 going to be any different or are they just going to do
something relatively superficial like say well we need to do it in-house next time, but
they end up doing essentially the same thing in-house that they did with the vendor the
Yeah, one of the really agonizing things about this kind of situation is that even if
the client calls in somebody like me, odds are pretty good that theyre going to
ignore what I tell them. A lot of times the changes need to be made at a pretty high level
on the client side. Thats one of the reasons I wrote this Survival Guide. One of my
big goals for that book was to write a short book that would be short enough for upper
management to have the time to read. Upper management isnt going to read any of my
other books because theyre too long and theyre too detailed, and it
doesnt make sense really. So I had the manuscript for the book reviewed by upper
managers at large organizations to make sure that I was considering the right issues and
talking about them in the right way.
I guess I am here to promote the book, so in a way it is a sales pitch for the book,
but I think that there is--regardless of whether you use that book or you use your
discussions or you work with your manager to educate that managers manager--there is
this education in the industry that needs to happen in upper management.
Software makes up an increasingly portion of many companys budgets and upper
management needs to get a lot more sophisticated about their software decisions, and
thats a sophistication theyre not going to attain throughout osmosis.
Its actually going to require some education from somewhere and if it has to come
from the bottom up, then fine. Its probably in your interest if youre working
for that company to get that started.
Audience: Given the amount of QA they had, were there red
flags coming up from QA that they couldve paid more attention to?
McConnell: The warning signs in this case were not that
subtle. They didnt need QA. After five months, the software simply wasnt
there. Nobody was even saying that it was there. It just wasnt done yet. So there
wasnt anything to test or perform other quality assurance on. In some other
projects, the warning signs might be a little harder to distinguish, but in this case,
they werent. They were very, very blatant.
Audience: How did you find out so much about the project?
What was your involvement?
McConnell: My involvement was that I knew people on both the
vendor company and the client company, and had gotten this phone call that my lawyer
advised me not to do anything about. And so my involvement was that I was very frustrated
about not being able to go to the client company and say, "Youre really headed
for trouble here and I have some specific reasons to think that." And I also couldn't
go to the vendor company and say, "I think you have the wrong idea about how big this
project is." So it was a painful experience for me because I felt like I should do
something about it, but I wasnt able to do anything about it. So I kept tabs on it
Audience: Am I seeing project management get better, stay the
same or get worse?
McConnell: I think theres a huge amount of variation in
the software industry right now. One of the issues that were dealing with is that
the industry is growing more quickly than new people can be trained to work in the
industry. There have been front page stories in the New York Times and the Wall Street
Journal in the last couple months that have talked about the labor shortage for
programmers. Estimates in the US range from 200,000 to 400,000 open programming jobs right
now. Were creating new programming jobs at a rate of about 100,000 a year in this
country and were granting about 35,000 computer science and computer related degrees
a year. So most of the people coming into the field are not getting even as much training
as an undergraduate degree in computer science. And so I think that were basically
growing this pyramid and the number of people who are educated and experienced on the top
and the middle of the pyramid is relatively small compared to all the people coming in at
That whole situation is compounded by the fact that were still a relatively young
industry and I think there are really good management practices out there and I talked
about some of them in Rapid Development and the Survival Guide, but I think the knowledge
of those is not very widespread. So in a way, I think the situation is basically stable or
the level of management may be basically stable, but the industry is getting bigger and so
it appears maybe that the problem is getting worse. There are pockets where the management
is very sophisticated, but I think at this point, I really couldnt call it anything
much more than a few pockets.
Ill take two more comments.
Audience: Earlier you commented that if somebody brought you
in as a consultant in a case like this, your advice might be ignored so what hope is there
McConnell: I might have been being a little too flip when I
made that comment earlier. If the project manager of that project brought me in, I think
that theres a good chance that upper management would ignore whatever came about as
a result of that. But if upper management brought me in, theres a very good
chance that action would be taken. Basically, I think its a long, ongoing
educational process and the hope has to be long term rather than short term. Bringing one
guy in for a day or two to talk about some stuff is probably not going to make a
difference unless its part of a long term education strategy. My answer is to think
long term, not short term and things will get better over time.
Audience: What software might be new but there other things
that are similar, what kind of parallels do we draw to other industries?
McConnell: I agree with that point. I think that its
very constructive to draw parallels to other industries as long as you dont take the
parallels too seriously. And this is the key point: as long as you know enough about the
other industry to draw a meaningful parallel.
For example I draw a lot of parallels to building construction based on the idea that
you go through a steady progression of activities that range from not very expensive to
quite expensive. Youre going to draw blueprints before you actually start ordering
materials, sawing boards and pounding them together because its a lot easier to
change lines on a blueprint than it is to actually move a physical wall when thats
been constructed. And I think that parallels to software from that kind of thing are
pretty clear. That is its easier to change things in requirements time or design
time than it is to change hundreds of lines of code even though it doesnt look as
physical as the materials in a building look. The main cost in both cases is actually
people's time rather than the material in most cases, and so the parallel works pretty
There are other parallels that are interesting that are not quite as obvious. Like one
thing you hear people say sometimes is, "Why cant these software guys estimate
their projects more accurately? Look at the building industry." Well, look at
the building industry! For small projects, like if youre going to build a $200,000
house, you can estimate that kind of project pretty accurately, but when you look at
one-of-a-kind projects that are in the millions of dollars, cost overruns on those kinds
of projects can be very, very high, for the same reason that they are in
software--its a new thing. So I think those parallels need to be drawn carefully and
with some knowledge about the other industry that youre comparing them to.
Thanks very much for coming today.