The eternal battle in a services agency world

When you learn about agile practices, one of the most challenging concepts to grasp is Story Point Estimation. This is particularly true for service companies that normally bill clients by the hour. 

On the one hand, they need to submit project proposals that are based on concrete time estimates.

Story points, on the other hand, are an abstract concept by design. They were born out of the needs of product companies with the purpose of having a better assessment of the amount of work involved in developing software units, regardless of how long each unit takes. This makes people struggle between using points and hours. You want to reap the benefits of point estimates, but you still want to keep things tied to time.

In this post I discuss the notion of point estimation, highlighting its advantages over hour estimation. I explain one approach to doing point estimation in a professional services company, while addressing common concern from various stakeholders about the lack of time estimates.

Why should I care about point estimates?

To understand the importance of point estimation, it is necessary to point out the issues of time estimation. It is a well known fact that developers are generally not good at estimating how long it will take to complete a development task. 

Developer Estimation Comic Strip

To be fair, the reasons for this are all justifiable and varied:

  • Unknown development factors: The estimate is created based on what you know. But you don’t know what you don’t know. Unexpected things come up all the time in software development, but they can’t be quantified.
  • Long tasks: Often work is too big in scope or involves actually doing multiple things to achieve the end goal. This adds another layer of abstraction and uncertainty to the estimate and hides the real amount of work that it will take.
  • Overconfidence: Another factor is the optimistic approach that developers often take to estimation. In their minds they calculate how long it will take to just build things, assuming that their sharp skills will allow them to move smoothly over the development task. Often this does not happen that way.
  • Lack of confidence: When developers are at the junior level or when senior developers face tasks and technologies they have never used before, they tend to feel a lack of confidence and this can lead to overestimation. 
  • Not accounting for full development cycle: Finally, when faced with an estimation task, developers only think of the time required to do the ‘implementation’ part of the deliverable. But from a software development cycle perspective, there is a lot more to just making a feature “work”. There is quality assurance time, bug fixing, research, user acceptance testing, deployment etc. All of these additional tasks are not accounted for when doing estimations, but they are actually required to call something “Done”.

With such a long list of things that can go wrong in estimations, it is no surprise that most projects (50% to 80%) have schedule or effort overruns. With some studies like the Standish Group's "Chaos Report" even describing an average cost overrun on 89% of the projects.

Furthermore it has been shown that when a software professional has a 90% confidence (almost sure) that they included all of the effort required in their estimate, in reality the frequency with which they were right about their assessment is 60% to 70%.

Project Management - Expecting Different Outcome Doing the Same Thing

The conclusion is that even at best effort, time estimates are really a subjective and abstract measure of the actual effort required to complete a task. However, because they are expressed in concrete units (hours, days, weeks etc) they are taken literally by everyone else but the developers themselves. 

Enter point estimates…

So, if it is so difficult to estimate how long something is going to take, and it has been proven that these estimates are most of the time wrong, what can we do?

The first step is to acknowledge the subjectivity of these “ideal” time estimates and then find a way to express them in non-concrete units which can help us plan work ahead. But more importantly something that can help us to measure work done in the past. It is this last part about measuring the amount of work done where point estimates prove their value and it is more often the part that is missed when evaluating whether point estimates make sense. 

With time units, measuring work done can only account up to the number of working hours in a day. For example all you can tell is that someone worked 40 hours in a given week. By this measure, unless someone puts in extra time, it is not possible to assume that more work can be done in a given week. 

With an abstract measure like points however, more points can be achieved in a given week when you start optimizing how time is spent. When you remove distractions, when you make work clear to developers, when you break down work into digestible and independent user stories etc. Your increased focus and optimization efforts will all be reflected in the amount of “points” that your team accomplished in a given sprint (the amount of work done). Your time sheets however will continue to show the same 40 hours per developer per week.

But... what exactly is a “story point” and how do I use them?

A point is an abstract measurement of amount of work. You don’t think about how long this work will take, but rather how big of an effort the work entails. It is a measurement of relative size, not time.

Points are subjective and we should be careful about inflating themIt helps to think of an analogy. Imagine you own a moving company and your clients hire you to load a truck with boxes and then unload the truck in the new house. You also have a crew to carry the work. Your crew is composed of people of different ages, different experience moving boxes and with different physical strength. They also come and go because they do this part time and they are not available every day.

As you take on new clients, you always wonder:

How much work your crew can handle per week?

How many moving jobs they can take on over the next few weeks, or months?

So, you want to figure out how much load overall your crew can move on average in a given week.

Your goal is not to understand how much time it will take to move individual boxes on and off the truck by individual people. As you can imagine, you will have multiple boxes of different sizes and different weights which are being moved by people with different abilities who come and go as they are available. So this is a challenge.

To assess that abstract concept of how much load you can move, you start assigning subjective sizes to your boxes (i.e. Large, medium, small, extra small, extra large etc.). This tells you the relative size of the boxes, not the actual concrete measurements in inches or the weight in pounds (i.e. boxes of slightly different sizes and weights may be classified both as “large”). 

Then you let your crew work for a week and you count how many of each size you moved over the course of the week. If you do this for even as little as two or three weeks, you will have a pretty good idea of how much “load” or “work” your team can tackle (already accounting for all the variables and uncertainties described before), even without knowing the detailed measurements and weight of each box or how long it takes to carry a single box. 

Finally as you engage in business with new clients, you must assess how many boxes of each size will be required for the new moving job. This, combined with your understanding of how much load your crew can move will help you understand where in your calendar your team will be ready to help move those boxes for the new client.

Ok, pretty long analogy to say that your moving crew is your development team, the moving jobs are your projects or even your features within a project. Moving boxes is your team accomplishing and releasing user stories, and the amount of work that your team can carry in a given time frame (i.e. two weeks) is your sprint velocity. Finally, that subjective measurement of how much effort a single box entails (your box sizes) becomes your user story points. The advantage of points over box (or t-shirt) sizes is that points are expressed in numbers and unlike the box sizes they can be added up to account for your entire team’s velocity in points for a given sprint. 

As you plan new development work, you break down work into user stories and then you estimate these user stories in points which reflect their relative size. Then sprint planning involves two simple steps. Set your team capacity (in points) for the upcoming sprint based on your historical velocity and then fill your sprint with boxes…. oops!, sorry… user stories that add up to that velocity.

So where did time go?

You may be wondering, OK I start using points and that will help with the planning of upcoming sprints. However, I need to give time based estimates to clients and I need to bill them based on time. Well, the good news is that you don’t have to stop thinking about time completely. You just have to think about it in the proper phases of your project. 

At the start of a project you still need to make a rough estimate of how many hours something is going to take. You do this estimate with your more experienced software engineer(s). And you do it for the high level features of the software project that you will deliver. You don’t break down into user stories at this point, or at least you don’t estimate those smaller pieces. This initial time estimate is necessary to do the math and be able to provide a cost estimate to your client and it is wise to pad it a bit to account for the unknown.

The second part of the project where it is critical to focus on time is, well, on time tracking. As you work on those user stories that were estimated in points, you still need your software developers to track how much time they spend in each user story.

As the project progresses, your project managers should constantly keep an eye on time spent vs the original rough time estimates for each high level feature. This is the way for them to keep tabs on the budget consumption. 

Furthermore, they can always compare the time spent vs the percentage of points finished for a given feature. Say you have a search feature for a custom application that has 10 user stories that describe the functionality. And say that those stories have various point sizes that add up to 100 points. When developers are “half way”, having completed around the 50 point of work, the time spent in that feature should be about half of the original rough time estimate made for the feature.

Have your cake and eat it too.

With this approach you get the best of both worlds. You get a real representation of how much work your team can accomplish every sprint. This allows you to plan sprints ahead and finish when you said you were going to finish. And you also get the time record which helps to keep a close watch on project budget consumption.

While trying to implement this you may get some push back from different people who would be uneasy about stop asking for detailed time estimates from developers. You can remove this mental barrier by pointing out the great advantage of actually knowing how much work your developers can do and reminding them that those time estimates that they are providing today are wrong most of the time anyways.

Still, there are challenges in the implementation of this point estimation practices. In a later post I will go into the details of the tactics and the problems you can find along the way and how to deal with them. In the meantime I hope this post has helped you understand what points really are and how they help software development companies to have a clear picture of how much work can be done by the dev team.