As a consulting company, we don’t have the luxury of working without a defined project scope. For whatever silly reason, our customers always want to have a clear idea of how much something is going to cost and how long the work will take. Go figure.
On more than one occasion I have had the (cynical) thought that agile development methods are a response to developer antipathy to project estimation. While I know that’s false—heck, I’m a certified Scrum Master myself—my experiences with getting realistic estimates from developers have made me somewhat jaded.
I find that developers come in three different varieties when it comes to this.
Mr. Two to Two Hundred Hours: An actual estimate. Very helpful. This kind of developer doesn’t know and doesn’t know how to know.
Ms. Off by a Factor of Two: This kind of developer understands the general principles of project estimation, but is always low by a factor of two.
Mr. Realistic: A percentage of developers—and definitely the smallest percentage—are both accurate and realistic when it comes to development estimating. What it is that they do differently is what separates them from their peers.
A typical project starts out with a general goal and some requirements. Create an estimation tool would be a goal. It has to be Web-based would be a requirement.
Early in a project, the requirements can be problematic for developers. The requirements aren’t stable, so one day the project will be on MySQL and the next day it will be on Oracle. The requirements are often very well-defined in some areas and loosely defined in others. Even the goal can be subject to refinements, from the project’s target consumers to the budget.
So the first element in the art of estimation is to estimate what you know. And ignore the rest. As consultants, we typically kick off a project with a period of discovery and analysis. During this period, we can usually tell a customer what it will take to go from groping in the dark to having a well-defined goal and a set of requirements that, at least at the high level, aren’t likely to shift like sand underneath our feet.
We can predict what we will have to do to get through this period and we can estimate that very accurately. So costs and schedule are known.
If you work in an agile method, this can be the product of one or two sprints, depending upon the complexity of the project and duration of the sprints.
With the goal and requirements stabilized, it becomes possible to do some up front design work which can be broken out into any of a number of categories.
User Interface Design work, for instance, is typically at the information architecture and wire framing level. Top level screens first. Lower level screens might be left for later.
Architectural Design work is done—with groundwork laid for the overall architectural pattern and design patterns of the project.
The data entities and their relationships might be diagrammed.
Along with the design work, whatever it might be, there is often a level of prototyping that needs to be done. Estimation is fraught with peril when there are unknowns. So this is the time to unequivocally address the unknowns. Will a set of controls be adequate or will you have to find new ones or roll your own? Best to find out. Will an architecture be scalable enough? Try it out.
The time spent in proof of concept mode is never wasted, even if the code is wholly throw-away. Because when this phase is done, you know that your design will meet your needs. And you can better predict what your implementation time will be.
The first step is itemizing out the work.
The next step is breaking the work out into granular tasks that can be estimated. How granular is granular? That depends greatly on the estimating skill of the developer. Estimates should have a lot of tasks that are at the two- and four-hour level. But good estimates may also include a few tasks that are at the forty-hour level. Most developers would find it frustrating to take all work down to the two-hour level. But when you see too many forty-hour tasks, you know that the estimates will be way off. That’s more like creating big buckets of time than thinking your way through estimates.
The final step is estimating. How much you estimate is determined by your development method. In an agile method, you might only estimate a sprint at a time.
For us, we have to estimate all of the known and agreed-upon requirements. If there are maybes or nice-to-haves, we typically don’t spend the time estimating them until they firm up as requirements—or until the main work is done.
Good estimates also have to accurately include how much time will be spent in non-development activities. There will be meetings, project status reporting, mentoring, writing documentation, etc.— these all take time. So accuracy will go up when time is actually budgeted for what will inevitably take place.