Development Estimates are Broken

Our clients frequently come to us with horror stories of past engagements with agencies and developers. Their experiences vary, but by and large the most common pain point we hear is related to estimates and budgets. Tell me if you’ve heard (or uttered) any of these phrases before:

“The estimate was wrong. Way wrong.”

“There was substantial budget overrun.”

“We paid a fixed bid for something that wasn’t worth anywhere near what we ended up paying for it.”

‍

The Terrible Secret

Development estimates suck. Developers and customers have been fundamentally going about this all wrong, and the outcomes are terrible and frustrating for everyone.

We needed a different approach to create better expectations and better outcomes for us and our clients–an approach that allowed us to maintain high-trust, high-performing relationships with our clients. So we came up with one.

Rule #1: Start Blurry, Finish Sharp

When starting a new development project, you’re too far out from the finished product to have a perfectly crisp and clear vision of the effort. Without a perfectly sharp vision of the effort involved, you can’t produce a perfectly reliable estimate.

I can feel management squirming in their chairs as they read this. Welcome to the uncomfortable reality–building complicated and custom digital products is full of ambiguity. Read on, because I promise I’m going to make you feel better.

Teams and clients try to mitigate this ambiguity by delaying development until as much ambiguity as possible is removed from the estimate process. This is vaguely the promise of “waterfall” product development. Few recognize the problems this creates.

First, how much ambiguity is enough to remove before you can reliably estimate the work? Do you need a perfectly written technical specification? Do you need a complete, pixel-perfect design? The only time when all ambiguity is removed is when the build is done.

So, at what point before being completely done is enough ambiguity removed to have a perfect estimate? Put simply, never. 😳

Development estimates suck, remember? This still applies, even under mythical “perfect circumstances.” And it gets worse…

Secondly, all that time and effort you’re going to spend trying to reach “the perfect estimate” is going to be costly, and it will yield no real-world value at all. Remember, you’re blocking your team from building anything that delivers any value to end users in the interest of better development estimates. We’ve all heard of paralysis by analysis, yes?

Uhh… that’s terrifying. What gives?

At Camber, we start with low-resolution, blurry estimates for larger scopes of work (e.g. the up-front estimate for a first major release, starting from project day zero), and later refine to high-resolution, clearer estimates for scopes of work that are directly in front of us (e.g. specific deliverables to be completed in the next two weeks).

This way, we invest just enough effort and budget to know roughly where the long-term budget will end up, and to be confident in our more refined estimates for more immediate work.

Wait, you said that development estimates suck, and there are no perfect circumstances for an estimate. Why are you so confident in your “refined” estimates?

‍

Rule #2: Estimate Using Relative Effort with Constraints

How’s that for an esoteric section headline? Let me explain.

Relative Effort

Relative effort estimates mean you are anchoring the estimates on tasks relative to each other, not relative to time.

✅ DO THIS: Task 1 will require about 2x the effort of Task 2.

⛔ DON’T DO THIS: Task 1 will require 3 hours, and Task 2 will require 7 hours.

Okay, why though?

Because development estimates suck, remember? But do you know what sucks less? The reliability of relative effort estimates over time.

We’ve found that estimators are much better at judging the relative effort of one task vs another than they are at estimating the exact hours required to complete either.

After development has started, task 1 is still going to be roughly 2x the effort of task 2, regardless of how many hours it actually ends up taking to complete either.

If you estimate using relative effort values, original estimates remain relevant later in development. If you estimate using fixed hour values, old estimates quickly become meaningless as the development train keeps rolling.

Constraints

Constraints mean limiting the estimate values available to choose from. I’ll explain why to do this in the next section.

Applying Relative Effort and Constraints

Our constraint of choice for estimate values is the Fibonacci Sequence (1, 2, 3, 5, 8, 13, 21, etc), because it offers a non-linear scale where the relationship of one value to another in the sequence is intuitive, and the magnitude between them becomes greater (blurrier) as the sequence progresses.

We refer to each estimate value as “points” of effort. So “1” means one point, “2” means two points, and so on.

From any given list of tasks to be estimated, the estimator picks the one task that will likely take the least amount of time to complete. They assign this task an estimate of 1 point, and this task becomes the estimator’s “basis point.” All further tasks will be estimated relative to this basis point using only the Fibonacci Sequence values.

An estimate of 2 points means roughly twice the amount of effort of the basis point task. An estimate of 5 points means roughly five times the amount of effort as the basis point task.

You may have noticed that this non-linear scale of estimate values also helps us to reapply Rule #1 in the context of individual tasks. The bigger and more complex a given task, the lower the resolution–or more blurry–its estimate becomes. The higher the estimate point value, the greater the impetus to break up the task into smaller, clearer tasks so that we can sharpen the estimate when the task is closer on the development roadmap.

We also often include more than one estimator in the process of applying point estimates, which acts as an internal quality control mechanism to ensure greater accuracy in selecting these point values.

How the heck am I supposed to budget my project with these points? I need my estimates in dollars and time!

‍

Rule #3: For Budgeting, Use “Expected Hours” Ranges Up-Front, then Historical Performance

When you’re just diving into a brand new project, you really don’t know how these estimated “points” of effort are going to translate into hours and budget. This is when your overall project budget is going to be the fuzziest (refer back to Rule #1 again).

However, we recognize that you need to have some idea of what kind of budget is likely to correspond with your total points estimate. So we provide a blurry range of “expected hours per point,” using the basis point task as our yardstick.

We do this by revisiting the “basis point” task–the first task estimated with “1 point” of effort, which the estimator perceived to be the lowest effort/fastest task to complete on the list. For this one task, we break rule #2 in the interest of attaining a blurry overall budget picture for our clients. We reestimate the lowest effort/fastest to complete task on the list as a range of hard hours (the horror!). Fortunately, relatively small tasks are easier to estimate reliably in hours.

Once satisfied with this range of hours for the basis point task, the estimator will then gut check that range against other tasks to see if the conversion rate seems to hold up. For example, if the basis point task is estimated as a 4 to 6 hour task, do other tasks estimated at 1 point also look like 4 to 6 hours? Does a 2 point task look like about 8 to 12 hours (twice the hours of the 1 point task)? And so on.

You should be able to see at this point how the expected hours range, paired with the non-linear points scale, work together to mitigate some of the risk of sucky estimates before work has ever started.

It’s all uphill from here.

Remember how I said relative effort estimates hold up better over time than hour estimates? While the expected number of hours per point is really fuzzy before any work has been done, it becomes much more clear after actually delivering on some work.

Because we work to deliver small chunks of fully-complete functionality, we can relatively quickly establish an initial “real hours per point” conversion rate for the project. This number has the benefit of being much sharper than the original estimated range of hours per point because it’s based on real historical performance of delivering completed tasks. The further into the project we go, the sharper this number will become, until all ambiguity is removed (because the work is DONE)!

You said you were going to make me feel better at some point. How do we make sure this doesn’t go off the rails?

‍

Rule #4: Fixed Budget, Flexible Scope

This was it. This was the breakthrough that helped more than anything to build the high-trust, high-performing relationships with our clients that we were after. Fixed budgets with flexible scopes was the answer that best balanced needs and aligned expectations.

We stopped asking clients to trust our estimates. Estimates suck. We started asking our clients to trust our principles and our approach.

Nobody can reliably guarantee that [group of tasks] will take [x time] and cost [$y]. If they do, they are marking up their estimates substantially to cover the downside of their estimates sucking (which they do, by the way). That means you’re going to pay way more than the work was actually worth. We think that is a terrible outcome for clients, and not a great way to build high-trust nor high-performing relationships.

We changed the focus from getting initial estimates perfect (an impossible task) to optimizing our approach to actually delivering work. Every product decision is value-focused, using our Value Framework. From what to build to how to build it and when, everything we do is based upon delivering the best possible value, period.

This relentless focus on the optimal way to deliver value in the form of completed product features ensures that we are always delivering efficiently and taking small steps in the right direction instead of big swings in the wrong direction.

The promises we make to our clients are that we won’t exceed the fixed budgets they approve, and that we’ll deliver as efficiently as possible so that as much scope as can be delivered in that fixed budget, will be.