A publication of Work On Waste USA, Inc., 82 Judson, Canton, NY 13617 315-379-9200 April 18, 1991

Mathematicians find serious error in risk assessment
performed by Kay H. Jones for a
WESTINGHOUSE 1,700 tpd incinerator
proposed for Monmouth County, NJ.

Kay H. Jones worked for the consulting firm of Roy F. Weston until early 1990 and now is the president of Zephyr Consulting in Seattle, Washington. Jones was Weston’s consultant for several incinerator projects including Detroit; South Philadelphia; Preston, CT; Spokane, WA; Conshohocken, PA; and many others. In a 4-8-91 letter to Scott Weiner, Commissioner of the Department of Environmental Protection (DEP) in New Jersey, Dr. John Healy, (PhD in statistics from Purdue U.) and Dr. Jerome Stanshine (PhD in mathematics from MIT) identify a serious mathematical error in a risk assessment performed by Kay Jones for the Westinghouse incinerator proposed for Tinton Falls in Monmouth County. They write:

“The mathematical error that has been identified results in serious underestimates of the risks from incinerators. Dr. Kay H. Jones, the expert hired by Monmouth County to lead the risk assessment effort, has indicated that this error is part of a standard way to do risk assessments. Therefore, it is likely that some, and perhaps all, of the incinerator risk assessments previously filed with the state of New Jersey have been affected by the technical error that we uncovered. Therefore, we recommend that the DEP re-open all previously filed incinerator risk assessments to determine if they have been contaminated by the technical error, and correct for the error wherever it has been made.”

Description of the technical issue:

There are serious flaws in the risk assessment process. These flaws have a large impact on a risk analysis and the conclusions that might be drawn from a risk analysis. A risk analysis should provide the community with a realistic assessment of the risks of an incinerator. A major result of a risk analysis are estimates of the chances of a dangerous amount of toxic emissions. In summary:

* The current process could underestimate the risks by a factor of 5, 10 or more.

* For risk assessment, the incinerator emissions should be assumed to be at a high enough level so that very few incinerators (perhaps 5 percent) emit higher levels than the ones used in the calculations.

“Current Process. No one knows in advance how well an incinerator will perform, and exactly how much of various toxins and carcinogens will be emitted into the air by an incinerator. This can only be determined by taking measurements after the incinerator is operational. Nonetheless, in order to conduct a risk assessment, some estimates of performance of the unbuilt incinerator must be made. These estimates are based on historical data. Our understanding of the current process for generating the estimates is based on a paper entitled Review of Analysis of Worldwide Resource Recovery Emissions Data and the Implications for Risk Assessment by Paul C. Siebert, Denise Alston-Guiden, and Kay H. Jones and on comments by Dr. Jones to the Monmouth County Risk Assessment Committee.

“In the above-mentioned paper, Dr. Jones calculates geometric means and 95 percent confidence intervals for emissions of several kinds of pollutants from incinerators. He assumes that geometric mean is the best estimate of emissions, and that the 95 percent confidence interval for the median is a conservative worst-case estimate. This upper 95% confidence interval is used to bound the risks of toxic emissions.** However, what he refers to as the ‘upper 95 percent confidence interval’ is not the 95th percentile, and this is a serious error. The following example may serve to illustrate the nature and seriousness of the error.

“Example. Dr. Jones starts with the EPA Toxic Equivalency Factors (TEFs) for twelve operational incinerators with acid gas controls (see Table 2 in his paper):

0.034 0.071 0.374

0.050 0.178 1.716*

0.058 0.330 4.705*

0.067 0.354 7.524*

“He then calculates the ‘Upper 95% Confidence Interval’ for the EPA TEF (see his table 4) for these incinerators:


“Three of the twelve incinerators (twenty-five percent) have Toxic Equivalency Factors that are two, five and eight times Dr. Jones’ calculated ‘Upper 95% Confidence Interval.’ (Incinerators marked with asterisk). The upper limit of a 95% confidence interval does not represent a worst-case.

“Dr. Jones’ ‘Upper 95% Confidence Interval’ does not provide a conservative estimate of how badly an incinerator can perform. A quarter of the incinerators performed worse, one of them eight times worse. They cannot be ignored. Dr. Jones’ analysis gives the wrong answer, not by a little, but by a lot.

“Better Analysis. A better ‘worst-case’ estimate is easy to calculate. Dr. Jones uses a set of lognormal distributions to model the collected incinerator emissions data. We recommend using the same distributions. However, instead of using his ‘upper 95 percent confidence interval’ for those distributions, we recommend using the 95th percentile. For the twelve Toxic Equivalency Factors in the example cited above, the estimate of the 95th percentiles comes out to be 5.58, rather than 0.88 that Dr. Jones uses. With the 95th percentile, we expect about 5% of the incinerators to have Toxic Equivalency Factors worse than (greater than) 5.58. It is unlikely that a new incinerator will have a Toxic Equivalency Factor substantially greater than 5.58, and this number realistically bounds the risks. Note that it is over 6 times the value used in the current process.

“For risk assessment, the incinerator emissions (including Toxic Equivalency Factors) should be assumed to be at a high enough level so that very few incinerators (perhaps 5 percent) emit higher levels than the ones uses in the calculation.”

** In Dr. Jones’ own words, “The emissions factor used in the risk analysis most likely case was the geometric mean value from all observed world data.... The risk analysis was also performed using the upper 95% value. All Weston risk analyses (which I directed) employed both values.” See Kay Jones, “Connett et al. R/A Paper,” memorandum to the Ad Hoc Risk Assessment Committee, January 2, 1991, Comment # 23.

“Community Composting in Zurich”

Video now available from Video-Active Productions
Route 2, Box 322, Canton, NY 13617 - 50 minutes, Cost: $25

Community composting is elegant, simple, successful, and replicable. Filmed by Roger Bailey in Zurich, Switzerland, in February 1991, Paul Connett interviews Thomas Waldmeir who heads up Zurich’s community compost program initiated by the City of Zurich five years ago. Currently there are 482 community composts in Zurich which serve between 3 to 200 householders. The second part of this video is a tour through a centralized 200 ton per day yard waste compost facility run by the City of Zurich. Marcus Neiderman, the director of the yard waste composting, stresses the importance of source separation of materials by the customer bringing material to the facility. He emphasizes that a compost that you can use with a “good conscience” for garden plants, etc., has to have up-front source separation of plastic, metals, “and anything that cannot be used for a good soil.”

WASTE NOT # 147 A publication of Work on Waste USA, published 48 times a year. Annual rates are: Groups & Non-Profits $50; Students & Seniors $35; Individual $40; Consultants & For-Profits $125; Canadian $US45; Overseas $65. Editors: Ellen & Paul Connett, 82 Judson Street, Canton, NY 13617. Tel: 315-379-9200. Fax: 315-379-0448.