Sampling Applied to Pension Calculations

InfoManagement Direct, July 2007

Jessica Pollner, Dubravka Kulisic

Background

Upon retirement or termination, employees are often asked to choose a payout scheme for their vested pension benefits. Alternatives usually include a prorated "lump sum" payout, or an annuity, to be paid over the remaining lifetime of the employee and employee's spouse. In any scenario, it is critical that the pension benefit be calculated properly for both the corporation's and the employee's sake - giving inaccurate information hinders the employee's ability to make the choice of a payout scheme most advantageous to him or her, and also puts companies at risk for liability in providing misinformation.

A large corporation, with numerous divisions and affiliated companies, realized that their pension calculations for employees who retired or were terminated were incorrect. For a number of reasons, including ethical behavior by management coupled with the potential for a class-action lawsuit by the former employees, the company opted to be proactive and determined that pension benefits needed to be recalculated.

Although management recognized the need to recalculate these benefits, the hurdles they encountered were significant. These included but were not limited to:

• The absence of complete and comprehensive electronic data or hard copy files to support many of the earlier pension calculations;
• The changes in the pension plans over the years due to the corporate mergers and acquisitions that occurred and were continuing; and
• The multiple pension plans that provided benefits to more seasoned employees.

General Approach

Although these issues were troublesome, the absence of electronic files, in particular, required a creative approach to quantify the effect of the errors in the pension benefit calculations. Given that there were numerous pension plans in question and several thousand employees were eligible to receive benefits from these plans, it was determined that the efforts to recalculate the correct pension benefit amounts for each employee was, logistically, impossible. As such, an approach was put forth to firm management that required:

• A statistical sample of pension recipients to be selected;
• Correct pension benefits to be calculated for the statistical sample only;
• The results of the calculations for the statistical sample to be extrapolated to the relevant population of all pension recipients;
• The calculation of an average benefit to be paid to each eligible employee, adjusted for age at the time of retirement/termination; years of service with the company; pension plan membership; and payout option (lump sum or annuity); and
• Corporate approval of this methodology.

With corporate buy-in, it was determined that statistical sampling and estimation was an appropriate and credible technique to be used for these calculations. In particular, with the understanding from corporate management and the stakeholders themselves that corrected pension benefits would be based on an average adjusted calculation from a representative sample of the population, we applied statistical methodology in order to select a sample that satisfied the tenets of robust statistical theory. That is, the estimates derived from the sample would be unbiased and precise. Moreover, any one employee would likely receive an estimated benefit that may not be equivalent to what his or her benefit would have been if a new pension calculation were performed for all employees.

There are substantial benefits to statistical sampling. These include:

• Learning something about a population without examining each item in it;
• Reducing cost, labor and time associated with a review or test;
• Producing objective and defensible results;
• Providing rational predictions about the population; and
• Determining a plausible range (at a specified degree of confidence) around the estimates derived from the sample.

Implementation

In order to select a sample that was representative of the population, we requested a list of all retired or terminated employees who were eligible for retirement benefits during the period that the pension calculation was inaccurate. Representations from the company indicated that the pension algorithm calculation was known to be incorrect during only certain years and for only certain plans. (If these representations had not been made and even tested, it would have been necessary to increase the scope of our investigation and include eligible pension recipients over a longer time period and from additional pension plans.)

Employee-specific characteristics provided by the company included:

• Name and Social Security number;
• Age at time of retirement/termination;
• Hire date and years of service;
• Termination and pension end dates;
• Termination reason;
• Affiliated pension plan;
• Affiliated subsidiary or company; and
• Annuity or lump sum amount. (Annuity or lump sum amounts were potentially incorrect for many of the recipients. As was explained to us, the date of retirement or termination within a calendar year was correlated with the correctness or incorrectness of the pension calculation. Additionally, the pension amounts for a large number of employees were missing.)

This data was the starting point for our statistical sample implementation. More specifically, using these variables singly or in combination, we determined that a stratified sample was the most effective design to determine unbiased and precise estimates of the pension liabilities for the entire population. After examining numerous combinations of variables, we ultimately determined that stratifying by the pension amount - with the expectation that a large number of employees had incorrect pension amounts - was sufficient to satisfy the precision and confidence requirements set forth initially.

Under many circumstances, it is quite appropriate to randomly select a sample from a population. However, it is frequently the case that the population from which a sample is to be selected is heterogeneous - that is, there is a great deal of variability in the item that one is interested in measuring or testing. Thus, in order to obtain estimates that are accurate from a sample drawn from a highly variable population, a large number of pension records may need to be sampled and then evaluated. Clearly, this method is still advantageous to testing, measuring or evaluating all items from a population, but may be more onerous or costly than the client anticipated.

An alternative to a simple random sample is a stratified random sample. What is stratification? In the latter specification, "like" items are grouped, so that there is reduced variability within a group (or stratum), whereas there is greater variability across groups (or strata). The advantage of such a design is that for the same level of precision and confidence specified for the simple random sample, fewer items are required to be sampled; that is, for the same sample size as that for a simple random sample, a stratified random design will usually result in enhanced precision (or accuracy) at a specified level of confidence.

With this underlying theory, and considering the level of effort required to calculate corrected pension amounts for the sample of recipients, we elected to use a stratified random sample. Thus, we requested electronic documentation from the company on about 8,500 pension beneficiaries, which consisted of about 1,900 employees who selected an annuity pension benefit, and about 6,600 employees who selected a lump-sum pension benefit. We recognized that much of the information provided electronically was incorrect, and a significant portion was missing- however, it was deemed a reasonable starting point from which a sample could be developed.

The annuity population was stratified into four groups, while the lump-sum population was stratified into six groups, defined by the initial (and often incorrect) pension amount. The design specifications for the strata were:

Figure 1: Design Specifications for the Strata

Once we specified the strata boundaries, we selected a random sample (by stratum). Note that the sample size for each stratum is calculated according to a specific mathematical formulation, and is therefore, not arbitrary. For a given stratified design, this mathematical formulation and sample size calculation (known as Neyman allocation) results in estimates that are more efficient (that is, a smaller standard error) and more precise than any alternative design.

In addition, it is critical that for any sample design - for example, a simple random sample or a stratified design - the sample is random. A judgmental or a targeted sample does not allow the analyst to draw meaningful conclusions about the entire population, as it is likely that the sample itself is biased, and therefore not representative of the population.

Because we were aware of the fact that management was missing both electronic and paper files for many of its former employees, we randomly selected additional records for all strata. These are used as replacement records for those employees where pension information was not available. Size specifics of the sample design are provided in Figure 2.

Figure 2: Size Specifics of the Sample Design

As noted in this table, the number of records used for the calculations were less than the original sample sizes (272 and 162 for the lump sum and annuity populations, respectively) recommended by our preliminary calculations. This is by no means a fatal flaw in statistical sampling theory and methodology - rather, if the number of items sampled is less than required, it is likely that the precision of the final estimate will be less than desired. In the presence of conservative assumptions, because information from the onset is limited and the data are less than pristine, the reduction in sample size often occurs.

Once the sample was drawn, we requested management to retrieve paper documentation for the selected employees in order to develop a database of those variables required to calculate the correct pension amount for the sampled pension recipients. While the database was being developed, the actuarial team reviewed the pension documentation for each of the plans that were in question, and met with management to summarize and discuss their understanding of the calculations necessary to correct the pension amounts. These discussions included but were not limited to:

• The crosswalk between annuities and lump-sum payments;
• The differences between retirement/termination dates and eligible pension dates;
• The inclusion or exclusion rules for bonuses or overtime, for example, in the pension calculations;
• The eligibility of a pension recipient in multiple pension plans;
• The required years of service for eligibility in a plan;
• The accuracy of the employee-specific characteristics provided to us (including, for example, age at time of retirement/termination, date of hire and salary); and
• The problems with the corporate calculations of pensions.

To determine the amount by which the correct pension benefit amount differed from the original pension benefit, we performed the following steps:

• For all sample records, we calculated the correct pension amount as an annuity, using the specific plan information, plan formulas and all available electronic data and hard copy documents. Note that pension benefits are usually calculated as a single life annuity and then converted to the actuarially equivalent form of the benefit selected by the participant.
• For the same sample records, we matched the correct amount to the original annuity amount shown in their hard copy file, and calculated the difference between the two.
• For each stratum, we calculated the average difference between the newly calculated pension amount and the originally calculated pension amount. The average difference (or error) is extrapolated to the total population by multiplying the average stratum error by the total number of beneficiaries in the stratum population. This is the mean per unit estimate.'
• Lastly, we applied interest to each person's estimated error (using the average PBGC interest rate), in order to provide an estimated present value of the calculated pension benefit errors.

In addition to the mean per unit extrapolation methodology, we implemented a regression-based approach, which was used to confirm the estimates obtained from the mean per unit methodology. In addition, this latter methodology was used to determine whether adjusting the recorded pension amounts for age, type of plan, and years of service would yield different estimates of the errors than that of the mean per unit methodology.

More specifically, we used the multiple regression model for the sample data to quantify the relationship between the newly calculated pension amounts and age as of benefit end date, pension plan and years of service (calculated as the time elapsed between benefit start date and benefit end date). For each separate stratum, we applied the estimated coefficients from the regression analyses to all employees in the stratum and population (lump sum and annuitants). This approach does not assume that all employees within a stratum (and population) will have the same "error" prior to interest - rather, the employee-specific characteristics of age, plan and years of service are accounted for explicitly. Note that we constrained the extrapolated stratum-specific results from the regression approach to equal the totals from the mean per unit estimation methodology. Thus, the estimated dollars in error, from both a pproaches, were consistent.

Using the mean per unit extrapolation methodology for the sample, we calculated the total dollars in error for the populations:

Figure 3: Total Dollars in Error for the Populations

Using the stratum-specific average error estimates for the population and applying interest to these errors, we estimated that the present value of the total error in the pension benefit calculations for the relevant lump-sum population ranged between approximately $22.5 million and $25.6 million. For the relevant annuity population, the total error in pension benefits was approximately $4.5 million. Thus, the present value of the total estimated error in the pension benefit calculations ranged between approximately $27 million and $30 million. Using the high end of this range, this represented less than $4,700 per relevant employee. The regression approach for the allocation resulted in consistent results.

In conclusion, this company made a bold decision to proactively adjust pension calculations for their retired and terminated employees. In addition, they used a creative, yet defensible analytic approach to determine this liability in the absence of electronic documentation. As such, they successfully integrated business and mathematics/statistics in order to solve a logistically difficult problem.

Jessica Pollner, Ph.D. is a principal in the Economics and Statistics group within the advisory practice of PricewaterhouseCoopers LLP in Washington D.C. Pollner's areas of expertise include the analysis of complete datasets using statistical modelling and computational techniques, sample design and evaluation, risk analysis and tie series analysis. She may be reached at (202) 414-1380 or jessica.pollner@us.pwc.com.

Dr. Kulisic is a director in the Advisory practice of PricewaterhouseCoopers LLP, with over 14 years of experience in conducting and managing projects involving complex data gathering and management, economic and statistical analyses, and damage quantification. She has done this for a variety of clients, across multiple industries and geographic regions. Dr Kulisic has a Ph.D. in labor economics, has authored numerous expert reports and has testified in Federal District Court. Dr. Kulisic can be reached at (646) 471-3728, or Dubravka.kulisic@us.pwc.com.

For more information on related topics, visit the following channels:

• Analytics
• Business Intelligence (BI)