No. 86 Should EPA Cost-Benefit Analyses Be Conducted by Independent Experts? The GLI as a Case Study

Quick Links

• Policy Study (pdf)

In 1995, the Environmental Protection Agency issued its Great Lakes Initiative, increasing the stringency of pollution controls across the Great Lakes basin. Touted by proponents as a cost-effective regulation, the GLI in fact will produce few, if any, human health or economic benefits. A careful review of EPA’s own GLI Benefits Analysis suggests the initiative might avert one cancer over the next 6,000 years or so. The initiative is likely to produce less than $5.00 of benefits for each $1 million spent.

1. EPA’s Great Lakes Initiative will produce a likely benefit of less than $5.00 for every $1 million spent by industry and taxpayers.

In 1995, the Environmental Protection Agency (EPA) issued the Great Lakes Initiative (GLI), a controversial rule requiring manufacturers and municipalities in the Great Lakes Basin (Indiana, Illinois, Michigan, Minnesota, New York, Ohio, Pennsylvania, and Wisconsin) to spend between $80 million and $380 million every year to reduce point-source emissions of certain chemicals into the Lakes.

GLI proponents boasted of EPA’s use of cost-benefit analysis throughout the GLI’s development. A highly favorable cost-benefit analysis was presented with the rule. Closer examination reveals, however, that the GLI Benefits Analysis was based on a number of incorrect, unlikely, and even impossible assumptions that dramatically inflated predicted benefits. The GLI’s positive impact on human health and the economy is exaggerated by EPA’s Benefits Analysis by 3000-fold or more.

While the Benefits Analysis estimated between 25 and 47 cancers would be averted over 70 years for Great Lakes fish consumers, more likely between 0.008 and 0.015 cancer deaths would be averted over that period. At most, one cancer death is likely to be averted over the next 6,000 years or so. Correcting for errors in the Benefits Analysis suggests that each $1 million invested in the GLI will produce a likely benefit of less than $5.00: a cost-benefit ratio of 185,000 to 1.

EPA predicted the GLI would reduce cancer mortality by reducing chemical concentrations in Great Lakes fish. The agency made at least seven errors when calculating this estimate. Taken together, the errors result in a minimum error of over 312,000 percent. (See summary table.)

• Fish consumption rate. EPA assumed Lake Michigan anglers would consume roughly 6,785,000 kg of fish per year. According to data compiled by the Department of Interior, however, Lake Michigan recreational anglers actually catch only enough to supply about 2,200,000 kg of cooked fish per year.

• Current fish concentrations of chemicals. To estimate chemical concentrations in the average fish, EPA relied almost entirely upon concentrations in lake trout–an unusually contaminated fish at the top of the food chain. In fact, lake trout can have 20 times or more the concentrations of chemicals found in other species of recreationally caught fish.

• Percentage point-source contribution to total loading. To estimate point-source loading into the Great Lakes of a group of organochlorine chemicals, including DDT, EPA looked at point-source loading for a group of dissimilar metals. A far better indicator of chemical point-source loading is DDT. Point sources contribute roughly 0.8 percent of DDT loading to the Great Lakes–substantially less than EPA’s estimate of 5 to 10 percent.

Expected decrease in loading with GLI. EPA assumed chemical concentrations in fish would respond linearly to decreases in pollution loading, ignoring the fact that river and lake sediments and the water column act as long-lived reservoirs for chemicals. Chemical concentrations currently measured in fish have most likely not yet caught up with past reductions in loading and are falling even without the GLI.

• EPA cancer slope factor. EPA’s use of cancer slope factors (CSFs) as indicators of most likely events is invalid. CSFs include a number of conservative assumptions, and EPA acknowledges that they are “not predictive of cancer incidence.” CSFs rely on a scaling factor (which translates laboratory animal doses to human doses) that EPA itself no longer recommends as valid. CSFs are based on the upper 95 percent confidence interval of the slope, rather than the most likely slope; the latter is a more accurate predictor of most likely events. CSFs also ignore data suggesting that chemicals are not carcinogenic at low doses.

3. Similar errors were made in EPA’s analysis of the Green Bay subwatershed.

EPA has responded to criticism of its whole watershed Benefits Analysis by contending that its subwatershed analyses were accurate and more relevant to the question of the GLI’s cost-effectiveness. In fact, however, the subwatershed analyses–typified by the analysis conducted for the Green Bay subwatershed–are also plagued by obvious and repetitive errors. Most problematic, EPA’s subwatershed benefits analysis relied heavily on an estimated loading value for PCBs that 1) greatly exaggerated benefits and 2) was widely known to be untrue, based on EPA’s own research.

When all of the errors are corrected, the Green Bay subwatershed results are little different from those for the whole watershed: point sources are found to contribute 1 percent or less of total PCB loading, and implementation of the GLI will cause no reduction in the loading of problematic chemicals.

Based on its projected reductions in point-source loading of chemicals into the Green Bay subwatershed, EPA estimated five different types of benefits that GLI implementation would bring to the Green Bay subwatershed. Those benefits are listed below, along with the agency’s low and high values for “benefits likely to occur” following adoption of the GLI. Also identified in the table are the more likely estimates of benefits, after EPA’s analyses are corrected for errors.

• Reduction in cancers. As with the whole watershed analysis, EPA’s calculation of cancer reduction for the Green Bay subwatershed suffers from multiple errors: a 3-fold overestimation of fish consumption; a 10-fold overestimation of chemical concentrations in fish; a 32-fold error for using the worst-case cancer slope factor to predict most likely events; and a 19- to 38.5-fold error associated with incorrect predictions of loading reductions. EPA’s estimate that 8.7 to 17.4 cancers will likely be averted over the next 70 years by implementation of the GLI is 18,000 to 37,000 times too high.

Correcting the errors produces a more likely, but still optimistic, estimate that 0.00047 cancers would be avoided over the next 70 years in the Green Bay subwatershed. At that rate, one cancer can be expected to be avoided sometime by the year 150,000, give or take a few millennia.

• Estimated value of the Green Bay fishery. EPA estimated the improvable fishery in the Green Bay subwatershed was worth, at most, $11 million per year: $1.9 million for the yellow perch fishery and $9.1 million for the salmonid fishery. After removing the unjustified yellow perch value and correcting errors in over-estimating the salmonid fishery, a best estimate of the worth of the improvable portion of the Green Bay recreational fishery is about $1.1 million per year. Consequently, the analysis for this benefit began with a 10-fold (1,000 percent) error.

Moreover, anglers can value the fishery more highly only if they can perceive a reduction in chemical concentrations. Even a 1 percent decrease in PCB concentrations–an optimistic projection about one-tenth of the value assumed by the EPA–would never be noticed under current sampling methods.

4. Rigorous, truly independent cost-benefit analysis is needed to avoid the errors that plague the GLI Benefits Analysis.

The benefits analyses for both the whole watershed and the Green Bay subwatershed are beset by repetitive, serious errors. In addition to their magnitude, which is daunting, the nature of the errors is disturbing. They show evidence of a systematic bias that consistently over-estimates benefits.

As regulations become more costly and controversial, we need an EPA with a reputation for competence, veracity, and common sense. Error-ridden and biased analyses such as those developed to support the GLI only lend substance to accusations that the agency twists the science and data to fit politically driven policies and squanders society’s resources on minor ecosystem threats.

The best way to promote cooperation and creative problem-solving is to have policies that are scientifically defensible and demonstrably sensible. The problem with current cost-benefits analyses is that they are conducted by EPA and the agency has complete control over the review process. Rigorous, truly independent cost-benefit analysis offers the most promising solution.

Based on Heartland Policy Study #86, “Is External Review of EPA Policies Warranted? The GLI as a Case Study,” by Daniel W. Smith Ph.D. Printed copies are available from The Heartland Institute for $10 each. You can also download the full text, free of charge, in Adobe’s PDF format; click here.

Copyright 1998 The Heartland Institute. Nothing in this Executive Summary should be construed as reflecting the views of The Heartland Institute, nor as an attempt to aid or hinder the passage of any legislation. Permission is hereby given to reprint or quote from this Executive Summary; please send tearsheets to The Heartland Institute, 19 South LaSalle Street #903, Chicago, Illinois 60603.

Questions? Call us at 312/377-4000; fax 312/377-5000; email [email protected]; Web www.heartland.org.