A software estimate is more than just a projection. It is a commitment to deliver a product within a defined budget and schedule. In order to meet that commitment, it is important for managers to recognize and evaluate the processes, tools and personnel skills that affect software productivity and be prepared to take action when it falls below a predefined threshold.
Product and process measurement is a core process component of project productivity management. The Software Engineering Institute, in discussing the practical software measurement (PSM) process, states: Measurement is a key element of successful management in every well-established engineering discipline.1
Software measurement is the means by which we can determine:
- Progress being made,
- Issues that need to be addressed,
- Risks that are being managed,
- Effectiveness of the process being used, and
- Quality of the products being produced.
This information enables us to make informed decisions that impact project cost, schedule and technical objectives.
The metrics collection process needs to be systematic but flexible and an integral part of the overall project management structure. A useful and effective way to identify metrics and establish a process consistent with the needs and capabilities of the organization is to apply the goal-question-metric (GQM) paradigm, defined as part of the PSM process. Figure 1 shows the relationship of the GQM components.
The top level of the organization establishes certain goals and from them specific questions might be asked as to what is required to meet the goals. From these questions, metrics are developed. Examples of questions and metrics are in Figure 2.
The GQM paradigm is based on the theory that all measurement should be goal-oriented. The metrics or measurements that are collected are used to answer the questions in a quantifiable manner.
While many approaches to measurement have been presented over the years, the one most consistent with the realities of monitoring productivity is based on the PSM approach developed by the Department of Defense. It is built on nine measurement principles:
- Objectives and issues are used to drive the measurement requirements.
- Define and collect measures based on the technical and management processes.
- Collect and analyze data at a level of detail sufficient to identify and isolate problems.
- Implement an independent analysis capability.
- Use a systematic analysis process to trace the measures to the decisions.
- Interpret the measurement results in the context of other project information.
- Integrate measurement into the project management process.
- Use the measurement process as a basis for objective communications.
- Focus initially on project-level analysis.
When the metrics process is functioning, management and development teams will have quantitative information available to them that can answer the following questions:
- Is the program still on track to complete within cost, schedule and performance criteria?
- What is the nature of problems and issues which adversely affect the project?
- What risks could impede the completion of the program within cost, schedule and performance criteria?
- What problems and risks are already being addressed?
- What indicators will reveal the emergence of future problems and issues?
- What indicators will reveal the emergence of negative changes?
- On what issues and in which areas should the program manager (PM) focus to ensure success?
Answers to these questions provide a record that can be used to identify trends related to productivity, project performance and product quality and to make realistic projections of what can potentially occur, providing critical insights into the cost and schedule performance.
Core metrics for software projects generally include:
- Effort: plan versus actual expended,
- Schedule: planned versus actual progress,
- Defects: planned insertion/removal versus actual, and
- Growth: planned scope versus project size.
Productivity: Productivity has one basic definition: size/effort. However, there are multiple definitions for effort and size. Values used for effort can include/exclude any of the software development activities and labor categories. To ensure an apples to apples comparison, computing a productivity value that can actually be used for measurement is useful. Nearly any consistent definition can be useful. Figures 3 through 5 provide some examples:
Earned value is the gold standard for productivity monitoring and is used by both government and commercial projects to monitor project performance. Earned value distinguishes true progress from effort or cost. The goal of earned value is to measure progress against plan. To be effective, earned value requires:
- A current schedule,
- A current Work Breakdown Structure (WBS),
- Realistic cost projections allocated to task areas, and
- A nonambiguous process for reporting milestone completion.
An essential element of an earned value management process is the allocation and specification of work to be performed through a WBS. The WBS is a technique to break a large project into small manageable units of work that can be monitored and whose effectiveness can be tracked. The lowest-level tasks in the WBS are known as work packages.
Using Earned Value Management
Earned value project management relies on the concepts of planned value and earned value. Each work package is assigned start and end dates during project planning and also a budget. The budget is known as planned value and is typically expressed in dollars. The budget at completion (BAC) is the total planned value for all the work packages.
As the project proceeds, work package progress can be tracked by recording the completed work by work package. This is called earned value because it represents the planned value that was earned by completing scheduled work.
The progress can be applied following different rules:
- 50/50 rule Take half credit when the task is started and 50 percent for completion.
- Binary completion Take 100 percent credit at completion, no credit until then.
- Apportioned credit Take a percentage of credit as the task proceeds.
While the rules may be different, the core process remains the same
Tracking progress through earned value is accomplished through the S-curve, a graph which provides a representation of how much value has been earned based on work completed versus how much money has been spent. It shows how the project is performing against plan, based on actual progress.
The three curves in Figure 6 represent:
- Budgeted cost for work scheduled (BCWS) - The cumulative, time-phased budgets for all planned activities. The BCWS curve is derived from the Work Breakdown Structure, the project budget and the Project Master Schedule.
- Actual cost of work performed (ACWP) - The cumulative, time-phased real costs of the work charged against the completed activities. The ACWP curve is found by actual measurement of the work completed.
- Budgeted cost of work performed (BCWP) - The cumulative, time-phased, planned costs of the work. The BCWP is calculated from the measured work completed and the budgeted costs for that work.
Schedule and cost variances can both be calculated in monetary terms from the data needed to produce the S-curves. Schedule variance is the difference between the earned value and the planned budget.
SV = BCWP BCWS
Cost variance is the difference between the earned value and the actual costs of the work.
CV = BCWP - ACWP
The graph shows that cost to date (ACWP) is higher than estimated (BCWP).
CPI = BCWP/ACWP
The schedule performance index (SPI) is defined as:
SPI = BCWP / BCWS
The following definitions are provided as a quick summary of the basic concepts and formulas used in the earned value process.
Estimate at Completion (EAC)min = (BAC BCWP) + ACWP
Estimate at Completion (EAC)mid = ((BAC BCWP) / CPI) + ACWP
EACmid assumes the overrun to date and that the rest of the project will perform at the same level demonstrated to date.
Estimate at Completion (EAC) max = ((BAC BCWP) / (CPI x SPI)) + ACWP
EACmax assumes that the overruns to date are indicative of faulty processes and will continue to degrade throughout the remaining portion of the project.
TCPI = (BACBCWP)/(EACACWP)
Every software project follows a process to fulfill its commitments. A process is the integration of:
- Project rules - product and process standards,
- Tools - the automated and manual facilities which enable the implementation of the process requirements, and
- Procedures - the specific set of practices which are how the project implements the process requirements of the project.
A project that doesnt rigorously develop and maintain a schedule will have difficulty tracking earned value against it.
Problems are often compounded by managers who jump to a right solution based on the promises made by the developers or vendors of a cutting-edge method or tool. Certain questions need to be asked to ensure the solution is really appropriate:
- What specifically is the need?
- Has anyone in the project used it before?
- Are any other solutions available?
- How will this method deal with the cost, resource and schedule constraints?
- Is there any quantitative evidence that the solution will address the projected resource or schedule shortfalls facing the project?
- Will the methods or tools integrate with other methods and tools used by the project?
- What resources are available to support the application of the methods or tools?
- Can we creatively define a way to use this method or tool to meet and satisfy the project requirements?
Understanding the Process Selection Constraints
If possible, process decisions should be nailed down prior to completion of the estimate. The organization has to address a single, specific issue: what processes will support the productivity commitments made by the project through the estimate?
- Is there enough money, time, qualified personnel and other resources to adequately satisfy commitments if we use this process, method or tool?
- Do we understand why the technology is needed?
- What are the essential training, installation and support requirements?
- Is there any evidence that the promised benefits will actually be realized?
One way to help define the specific problems is by looking at the top 10 cost drivers of your program.
Some simple steps that will enable a better selection of appropriate methodologies are:
- Define high-level process considerations which have the potential to negatively impact the project.
- Define the user expectations for the system and rigorously control development to meet them.
- Develop and enforce a management, project monitoring and reporting process that will provide positive control over the expenditures of resources.
- Understand and focus on the functional requirements rather than perceived wants.
- As the cost and schedule constraints become more stringent, the management of the basic project baseline needs to move from emphasizing control of productivity to increased control of product size, requirements and functional capabilities.
You must define a process that matches the reality of the project constraints and then select specific methodologies and tools which provide the highest potential to support the productivity required to meet the cost and schedule commitments.
Performing to the Estimate: Managing and Monitoring Software Development will continue with Part 2 in DM Direct next week.
- Boris Mutafelija and Harvey Stromberg. From Systematic Process Improvement Using ISO 9001:2000 and CMM. Artech House: 2003.
- Tom DeMarco. Peopleware: Productive Projects and Teams. Dorset House Publishing: 1987.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access