Five Considerations to Improve Project Estimates
Greg Todd thanks Mohit Sahgal, senior executive and lead solution architect with Accenture's Information Management Services for his contribution to this month's column.
As IT budgets dwindle and quality concernsrise, the need for improved estimation accuracy has never been greater. Business intelligence projects in particular have come under increased scrutiny, as little tolerance exists for projects that promise much but deliver short of expectations.
Businesses today demand more visibility, transparency and accountability in terms of sizing, solution architecting and delivery. Accurate estimates - demonstrated by on-time, on-budget delivery - build confidence that similar projects will also be successful. Inaccurate estimates, conversely, are quickly becoming malpractice in some organizations, eliminating any chance of future funding from increasingly cost-conscious sponsors.
Reasons for inaccurate estimates fall into one or more of the following categories:
- Generalization. Unreliable predictions ignore the real factors driving the work effort, staffing, task duration and costing.
- Misalignment of the estimating approach with the delivery methodology; that is, sizing of the work effort does not adequately reflect the work required.
- Inappropriate estimating factors for sizing the work effort. Hurdles include too many or too few factors, incorrect factors, nonexistent factors or inappropriate tasks.
- Limited use of calibrated estimating models. Unlike function point counts and lines of code estimation, few published standards account for the myriad BI design patterns and technologies. Thus, calibration is problematic and the results cannot easily be corroborated with similar project experiences.
- Limited or no estimation competency. Organizations do not inherently spend on estimation improvements or recognize the need for experienced estimation resources.
How can organizations improve? According to Steve McConnell, a software estimation authority, "Deceptively simple practices produce surprisingly accurate results." I agree, and I offer the following practical considerations.
1. The implications of generalization and guessing are cost overruns and delays. Organizations that rely on arbitrary time periods to "chunk" the work effort knowingly overlook the underlying details necessary to size the actual work. Guessing is an even worse approach because it requires no experience or expertise.
2. A well-formed, proven methodology serves as the bedrock of any successful IT program. Furthermore, the work breakdown structure (WBS) and work effort sizing should be based on a few critical deliverables. All tasks in the methodology must accurately reflect the work required. And, the WBS must be sufficient to produce all of the necessary deliverables. Tasks for which even one required deliverable cannot be defined are wasted effort. A BI framework within which the WBS can be organized is an excellent starting point to define the work and deliverables.
3. Factors are based on deliverables such as number of requirements, use cases, tables, reports, operational procedures, architecture components and executables. Organizations should question all factors not based on deliverables. Derived factors, or factors based on factors, are acceptable provided sufficient empirical data supports them. How sophisticated do the sizing calculations need to be? According to McConnell, "Complex formulas sometimes do more harm than good." Select an approach that works and use it consistently.
4. A proven estimating model is essential, with factors, formulas and methodology forming the basic recipe. Scalability that builds historical context and an expertise knowledge base is achieved through ongoing harvesting, calibration and corroboration. Higher performance can be achieved when the models account for:
- Additional levels of complexity.
- Task-level contingency.
- Productivity or work effort improvements/distribution.
- Subcontracting, third-party or teaming partner interdependencies.
- Phased deployment or release scheduling.
- Technology specialization.
- Staffing pyramids for different types of workforces.
- Site-specific service introduction.
- Production support maintenance and warranty.
Precision does not mean estimating the work effort to the hour, but rather estimating the work effort within generally acceptable tolerances.
5. How competent are the resources performing estimates in terms of their formal training and experience in estimating and implementing projects of similar size and complexity? At one extreme, organizations requiring part-time resources or those that initiate BI projects sporadically should seek professional help. Organizations at the other extreme should consider developing full-time solution architect roles - subject matter specialists with proven problem-solving skills who know both the functional and technical landscape. Organizations falling in between should improve the qualification of resources authorized to create estimates and allow sufficient time for peer review.
Holding the delivery team accountable to the original project economics is critical. Encouraging the use of earned value metrics can warn delivery teams of delays and cost overruns. Earned value is the de facto industry standard for objectively measuring a project's progress. Used in combination, these metrics can help project managers identify schedule delays and cost overruns. The metrics are often automatically calculated by most advanced project planning tools. In addition, graphing the metrics over time is a powerful technique to visualize variance trends. Rigorous application of earned value enables projects to detect variances, make course corrections and improve future estimation.
Estimates that help program and project managers efficiently deliver solutions on time and within budget can improve IT's credibility within the business. Improved estimation practices can help organizations achieve consistent, predictable and repeatable delivery results, reducing cost of poor quality, increasing customer satisfaction and improving bottom-line results.