- Estimates can and should be made for different purposes – there is a difference between, for instance, estimating the likely cost of a piece of work to decide if it is worth doing and estimating how much work can be accomplished in a given timeframe
- People use estimates inappropriately, and then blame the estimates, or the estimator when things go wrong
- A common source of error is an estimate created for one use is then blindly used for something that requires different precision or direction of error.
- Much of the trouble with estimating is not estimation itself, but the communications, or lack thereof, between people. If you don’t know how an estimate is going to be used, it’s likely to be the wrong estimate for the situation.
- In the end, estimation is not about a number and the accuracy or precision of that number. It’s about learning enough that you can guide your software development to a satisfactory completion.
George Dinwiddie has written a book titled Software Estimation without Guessing: Effective Planning in an Imperfect World. The book discusses different approaches to estimation for software products, the ways they can go wrong and be misused, and when to use them.
The book can be purchased here and InfoQ readers can download a sample chapter here.
Dinwiddie spoke to InfoQ about the book:
InfoQ: Why did you write this book?
George Dinwiddie: I started thinking more deeply about estimation a decade ago, when I noticed how much effort a team was putting into estimating stories, and how little value they were getting from it. The more precise they tried to make their estimates, the more expensive and less valuable they seemed to be.
That team was fortunate, though. There was nobody beating them up over “missed estimates.” The primary impetus for the book is how badly people treat each other over the topic of estimates. Too often, people mistreat other people. And I hate seeing people being mistreated.
People asking for estimates aren’t generally articulating their needs well enough to get them met. People providing estimates don’t understand the needs behind the request, and so provide estimates that don’t fulfill that need. People use estimates inappropriately, and then blame the estimates, or the estimator when things go wrong. People adjust their estimates hoping to avoid blame, and make them useless for their intended purpose.
It’s tempting to try to avoid the whole topic. Why do something that brings you pain? Creating software takes as long as it takes. Perhaps that’s true if we don’t change our way of working in response to how things are going. But creating software is not the end goal. Other goals and other people depend on the software that is created.
The value of that software, or the capability it gives, varies over time. Sometimes the sooner we can do something, the sooner we accrue value and the greater value over the useful life of the software. Sometimes being early has no additional value, but being late negates all the value. Understanding the larger context helps us understand the time function of this particular value.
As I looked more deeply at the topic I realized that, even if you can create software without estimation, it’s not a smart way to run a business. While Agile Software Development tries to narrow the gap between “the business” and “software development,” that gap still exists. The pain surrounding estimation is a symptom of that gap. If the gap didn’t exist, the software development team would, in the interests of being successful at business, also be interested in looking forward in time.
InfoQ: Who is this book for?
Dinwiddie: This book is for people who ask for estimates, people who are asked for estimates, and the people caught in between these two. All of them have needs that are inhibited by estimating using habitual patterns learned by rote. And often they have trouble understanding and communicating with people in the other groups.
One of the goals of this book is to help any of these people understand the viewpoint of the others. Much of the trouble with estimating is not estimation itself, but the communications, or lack thereof, between people. If you don’t know how an estimate is going to be used, it’s likely to be the wrong estimate for the situation. If you fear an estimate for one use is going to be misused for another, then you’ll likely develop an estimate that doesn’t satisfy either need.
To some degree, this book is about reducing the gap between software development groups, the business that they serve, and the management structures that intend to make these things work together. While some of the book focuses on the people issues of this gap, it mostly assumes that everyone wants to succeed together. The purpose of the book is to help them do that.
InfoQ: Estimation for software development is notorious for being incorrect, so much so that in many organisations there is a serious lack of trust for any estimations provided by software professionals. Why is it so hard, and what has caused this to happen?
Dinwiddie: As I just mentioned, one source of error is an estimate created for one use is then blindly used for something that requires different precision or direction of error. Let’s look at a possible example.
Suppose you want an estimate to determine if developing a new product might be feasible given the capacity of your development organization. That estimate won’t be appropriate for guaranteeing a release date. And if a development team, based on past experience, thinks that their estimate will be used as a plan and used to judge them and their work, they’ll pad their estimates to put in the contingencies that a plan should have, but that warp the estimate. If a manager, based on past experience, thinks that the estimate includes padding, they may disbelieve it and cut both the plan and the estimate they communicate to others. Instead of an estimate, we’re left with a raw number produced by a mish-mash of different assumptions, some in direct opposition with each other.
Then we confound that by developing software that may have fundamental differences to our past experience, using people whose knowledge, skills, and experience working together are unknown, in contexts that have different tacit expectations. There is a lot of variability in software development, and most of it isn’t readily visible.
InfoQ: The book provides many different approaches to estimation, why are there so many choices? What are some of the approaches, and how do they differ?
Dinwiddie: In one sense, there is only one way to estimate. That’s by comparing the unknown to the known. There are, of course, many ways to do that comparison. You might conceptually break the unknown down into smaller pieces that are easier to compare. You might build a model that encapsulates the comparison based on measurable attributes of the planned work. You might create a stochastic model or Monte Carlo simulation to try a large number of variations. Each of these general strategies also has many variations.
In the book I bring up many of the variations. Some variations are common to all three of these major strategies. Some look a little different from one strategy to another. There are too many combinations to cover them all, so I try to help people reason it out for their specific context. This is not a recipe book you can follow without thinking.
InfoQ: Given the wide range of estimation approaches, how does someone who needs an estimate to make a business decision pick the correct approach to use?
Dinwiddie: The choice of estimation strategy depends in large part on how you think about the work to be estimated. Start with what you know, or have already decided. Break the work down according to how it makes sense from a business point of view rather than from the way you assume it will be accomplished. It also depends on the knowledge you already have. How can you get the proposed work to seem more comparable to your references?
InfoQ: There has been a fair amount of push back against trying to estimate in uncertain environments, to the extent that #NoEstimates has become a catch phrase. How does what you are proposing align with the ideas from #NoEstimates?
Dinwiddie: Uncertain environments are exactly where you need estimates the most. It’s in those environments that you can’t see where you’re going. I was once sailing home when dense fog set in. My friend and I kept sailing by a compass course. I hadn’t taken precise notice of the last landmark we could see passing, but after awhile it seemed that we should have seen the lighthouse that was south of the river entrance. Had I not made that estimate, we would have gone a long way in the wrong direction. I presumed we had followed a course a bit further east than intended, so we turned west. After another period of time, I estimated we might be getting close to land. Sounding the shallow depth of the water suggested we might still be a little south of the river. Turning north until we found the deeper channel and then west again, we found the entrance to the river. In the fog we couldn’t tell our location except by estimating the distance and direction of our travel.
I’ll leave others to define the ideas behind NoEstimates. Over the years, I’ve gotten confusing and contradictory definitions.
If it means “don’t use story points,” then that doesn’t mean very much. Bob Payne and I were advocates for counting stories much earlier, but estimating stories doesn’t seem terribly harmful for choosing how much to bite off at one time. It’s perhaps more work than is valuable.
If you’re using them to make long term predictions, then you’ve got way too much precision for the amount of accuracy. I’ll go further than that and advise that you don’t decompose the work into story-sized pieces too far in advance. As I describe in Software Estimation without Guessing, there are a lot of downsides to doing so much detailed work far in advance, whether you’re estimating or counting. Story points, or counting stories, makes sense for a team trying to decide about how much work fits into the next couple weeks. It seems like an awful lot of work and precision for longer term estimates, though. And it’s the longer term that usually matters to the business.
I’ve also been told that it means “asking questions about estimation,” but I’ve had proponents be hostile to the questions I ask. I don’t think they consider the viewpoints and needs of others. Perhaps it’s an attempt to flip the typical power structure to put the software developers on top, but I don’t think that solves the problems. It just shifts how the problems appear. It still results in people treating other people badly. Using power over other people is not a form of collaboration. Respecting their needs does not mean giving up on your own needs, and standing up for your own needs does not mean dismissing theirs. On top of that, remember the needs of the goal that you, and they, set out to accomplish. This book focuses on all three sets of concerns.
I intend that this book will help people formulate and ask the questions that they need, the questions that get them out of the tarpits and on the way to better organizational success. As Esther Derby noted, this book is not just about how to estimate, but also how you can think about estimates.
InfoQ: Estimation is inherently a people and collaboration based activity, how do we keep them “clean” and not influenced by bias or unrealistic expectations?
Dinwiddie: Of course our estimates are going to be influenced by bias and unrealistic expectations. We need a feedback mechanism to tune these influences out over time.
We make assumptions about how things will go based on our past experience. If we make estimates, those assumptions are baked into the estimates. When the assumptions are wrong, so are the estimates. That seems bad, but it’s actually a feature. By checking the accuracy of our estimates early and often, we can detect when our assumptions are wrong. We can tell this because later reality differs from what we expected.
The one thing you know about an estimate is that it is going to be wrong to some degree. How wrong and wrong in what way are the more important questions. Making an early prediction and then trusting it for a long time seems like a foolish strategy. Estimates have a limited shelf-life. If you’re going to make a long-term estimate, you should also make some shorter-term interim estimates that you can use to check your assumptions. This gives you a means of calibrating your estimation process to your actual conditions.
In the end, estimation is not about a number and the accuracy or precision of that number. It’s about learning enough that you can guide your software development to a satisfactory completion. It’s about noticing growing problems before they sink your effort. It’s about the ability to make reasonable decisions, and then knowing to revisit those decisions when they’re built on shaky ground.
While most discussions of estimates are about the bad behavior that all of us have seen, this book offers advice on how to get out of that bad behavior. And it offers ways to get value, rather than pain, from the practice of software development estimation.
About the Book Author
George Dinwiddie helps organizations develop software more effectively. He brings decades of development experience from electronic hardware and embedded firmware to business information technology. He helps organizations, managers, and teams solve the problems they face by providing consulting, coaching, mentoring and training at the organizational, process, team, interpersonal, and technical levels. Involved in the Agile community since 2000, he has helped organizations ranging from a 6-person startup to a Fortune 100 company and a billion-plus dollar federal program, either directly or in partnership with other companies. He is the author of Software Estimation without Guessing: Effective Planning in an Imperfect World (Pragmatic Bookshelf), Evolutionary Anatomy of Test Automation Code (LeanPub) and co-author of Patterns of Agile Journeys (LeanPub).