It’s one of the seemingly inexplicable oddities of life in the software business: when you ask developers about their status, they often say “I’m about 80% done with my project” for about 80% of the duration of the project. So they think they are 80% done when they are only about 20% into the project, which makes their estimate wrong by 400%. Not good.
When hearing a story like this, other developers just put on a slightly arrogant look, and respond “bad management” and “it would never have happened where I work because …” (insert here, but in my experience often something such as “we are CMM level enormous certified”).
Why? It’s puzzling because of the magnitude of the mistake in the first case, and secondly because some projects do not seem to be affected by it at all. And, grabbing my own nose here, both seem to happen even to the same teams! (That eliminates CMM certification as the key difference …)
When I realized that both can happen to the same people, and compared the projects in which it happened, I think I have come up with an explanation. I now I think that it depends on the number of layers in the stack that the project is building. If it is a shallow stack, estimates seem to be in the ballpark. If it is a deep stack, they often are off dramatically. Let me try to explain:
A project’s shallow architecture might look like this:
If such a “shallow” project has five major parts, they can be developed largely independently from each other. By the time 4 out of the 5 parts have been written, “80% done” is a highly reliable estimate. Some straightforward database apps are like that: the app has 5 screens (or 50), and we are done with 4 (or 40), so we are 80% done with the project.
A project’s deep architecture with the same number of parts might look like this. Here, before any value can be delivered to the customer, all five layers must work, one on top of the other.
The first fallacy consists of thinking that this is the same diagram, just rotated. It’s a fallacy because the layers depend on each other, unlike in the shallow case, and cross-dependencies introduce complexity. Only once the last 20% of the code are being written does it become clear whether the lower layers work (most often not), or worse, whether they are useful at all in the way they were written. In many of these deep projects, extensive refactoring is required in the lower layers, making the project’s trajectory go more like this:
Here, everything looked like it was going nicely when the project proceeded along the green line. But then, when “80% done”, refactoring of the lower layers became necessary and the project moved into the endless meanderings of the red line that are a sign of multi-level refactorings. The project disappeared into doing gory things that nobody could understand, the customers can’t see any measurable progress for a long time (that’s because all the work is in lower layers that aren’t visible to the customer), the engineers get sick of refactoring and often, projects get canceled.
I drew the trajectory as a recursive tree traversal, which is kind of the worst case, but in no way unheard-of. Of course, each project meanders around differently but it sounds plausible that it visits the different layers in about as many ways as it would if recursively traversing the tree. If we assume that each circle represents the same amount of work, a 5-layer project might have 61 steps (!) instead of the 5 that one might naively think. That would make the naive estimate wrong by a factor of 12, not just 4.
What do we learn from it? First, consider yourself lucky if you think you are 80% done but are only 20% done. You could be just 6% done! (if you have to do a full tree traversal).
Ok, just kidding. But the dangers are real.
Secondly, the amount of work and chance of misestimate goes up exponentially with the number of layers. Humans are well-known to be very bad at predicting things that grow faster than linearly. So, keep the world flat! At least as much as possible. Your predictions will become better with it.
And now compare whether this explanation correlates with any recent major project successes and disasters that you might have seen. Would love to hear some stories that confirm or revise what I wrote.