Independent Consultants in Environmental and Forensic Chemistry

Volume 5, Issue 2, Winter 2002

Does Peer Review Mean That the Paper Is Scientifically Defensible?

Recently, a state regulatory agency asked a company and their site environmental contractor to look for a dense non-aqueous phase liquid (DNAPL) of trichloroethene (TCE) using ultraviolet (UV) fluorescence. This request was based on the work in a peer reviewed paper entitled "Evaluation of Visual Methods to Detect NAPL in Soil and Water" by R.M. Cohen, et. al., published in Ground Water Monitoring and Remediation, pages 132-141, Fall 1992. The contractor placed 19 borings on the site and studied the UV-fluorescence on 11 two-feet sections of each boring. The results showed four positives.

Cohen's paper stated that "Many unsaturated aliphatic hydrocarbons (such as trichloroethene and tetrachloroethene)" fluoresce. As support for this statement, a reference was made to a book (Konstantinova-Shlezinger, M.A, ed. 1961, Fluorimetric Analysis, Translated from Russian by the Israel Program for Scientific Translations Ltd. in 1965, Jerusalem). The paper went on to test tetrachloroethene (PCE), chlorobenzene, and kerosene in various soils at different concentrations using UV-fluorescence. Cohen, et. al., concluded that UV-fluorescence was a good method to visually determine the presence of a free phase liquid in soil and water. Since the technique appeared in a peer reviewed journal, the regulatory agency saw it as a viable screening tool in conducting environmental surveys.

However, the control experiment was never done. The free phase by itself was never tested. Did any of these pure substances fluoresce with the black light in a dark room? With this single, simple experiment left undone, the conclusions of this paper became unfounded, perhaps wishful, conjecture. If the Konstantinova-Shlezinger book had been reviewed, it would have been difficult to find the reference that stated that unsaturated aliphatic hydrocarbons (such as trichloroethene and tetrachloroethene) fluoresce. However, you would find the statement under "unsaturated hydrocarbons" that "As far as is known, there are no published data on the fluorescence of this class of substances. A priori, it might be expected that hydrocarbons with a system of conjugated double bonds would fluoresce." Neither trichloroethene nor tetrachloroethene have conjugated double bonds, and they are not hydrocarbons. They are chlorinated compounds. The two chlorinated compounds that are mentioned by Konstantinova-Shlezinger are chloroform and carbon tetrachloride with the notation "no fluorescence." What evidence, then, exists that trichloroethylene and tetrachloroethene fluoresce?

Interestingly, a paper entitled "A Comparison of Field Techniques for Confirming Dense Nonaqueous Phase Liquids" by T.W. Griffin and K.W. Watson also published in Ground Water Monitoring and Remediation, pages 48-59, Spring 2002, reported that "DNAPL product was also not discernible where TCE and Freon 113 were directly added to the sample." Apparently, these authors found that trichloroethene did not fluoresce.

While the four positives found in the current survey could be easily discounted by subsequent verification analyses, it is doubtful that verification analyses would have been conducted on the samples yielding negative results. Using the UV-fluorescence method to screen samples in the field, the presence of TCE could have gone undetected. Tens of thousands of dollars were spent attempting to find a DNAPL by a technique that cannot work. The four positive results in this investigation are false positives because tetrachloroethene does not fluorescence visible light when energized with a black light.

The error concerning whether TCE or PCE fluoresce should have been caught prior to publication. Peer review needs to be accomplished by people knowledgeable in the subject matter. In this case, even without the previous knowledge, the reviewer should have been able to check the referenced book or to have questioned the lack of analyses of the pure compounds in question. Peer review needs to be taken more seriously if environmental journals are to be defensible scientific publications instead of nonsense.

Is Your Laboratory Honest?

Laboratory reports concerning the results of analyses of environmental samples appear in various stages of detail. Sometimes, a letter listing only the concentrations of contaminants of concern constitutes the report. Other times, a "full" data package is requested which includes calibration data as well as the results of quality control samples analyzed with the samples. However, there are times when a data package is delivered that appears to be complete when, in fact, it is not.

For example, in order to determine the concentrations of contaminants in an environmental sample, an instrument calibration curve must first be constructed to define the relationship between the instrument response and the concentration. The data to construct this curve consist of a set of concentrations and respective responses to demonstrate that the data conform to a defined curve. For some published methods, five calibration standards are required to define the calibration curve. However, there are times when a laboratory will analyze more than five concentration standards to calibrate the instrument, or one or more of the five standards are analyzed more than once. The five sets of concentration/response data that provide the best calibration curve are selected from all of the pairs of data. The remaining sets of data are not utilized or reported.

Similarly, method blanks are often analyzed to determine if stray laboratory contamination is being introduced into samples. A clean blank suggests that the laboratory is not a source of contamination while a contaminated blank suggests that the laboratory may be contributing to contamination in the samples. Some laboratories analyze more than one method blank per analytical batch. To demonstrate that the laboratory is not a significant source of contamination, the blank with the least amount of contamination is reported. The other method blanks are not reported.

Calibration and quality control data provide information regarding the analytical system. In the case of the calibration data, if some concentration/response sets do not provide the linearity desired for the calibration curve, this may suggest an imprecision in the instrumentation or in the preparation of the standards. If contamination is found in some method blanks, this may suggest that associated field samples may also have been contaminated in the laboratory. These situations need to be investigated further, not ignored. To report a clean method blank and ignore a contaminated method blank could lead to the interpretation that field contamination exists when, if fact, it does not.

Analyzing extra calibration standards or extra method blanks is not forbidden. Review of the instrument run logs will indicate if more calibration standards and quality control samples have been analyzed than reported. However, unless justifiable reasons exist to disregard specific data, all data should be used. Choosing to use particular data because it conforms to desired results introduces a bias into the system. There is no reason to believe that the selected data are any better than the discarded data. The deliberate selection of some data from a set of data constitutes inappropriate and dishonest laboratory practices.

Could There Still Be Volatiles out There?

The fact that volatile organic compounds (VOCs) are rapidly lost from moist soil and sediment during sample handling procedures (collection, transport, preparation, and analysis) has been well established. Early suggestions that all soil and sediment samples be preserved in methanol at the time of collection were not readily accepted due to concerns about the use and transport of a hazardous solvent in and from the field as well as a perceived loss of sensitivity because higher detection limits (DLs) are typically reported from this approach. Current EPA-approved methods to minimize these losses and still allow low DLs to be reported include the use of sealed sample vials (e.g., Encores™) for "low-level" samples, with the use of methanol limited to preservation at the laboratory.

Typically, three Encores are collected from a sample location. Two are prepared for low-level analyses, and the third is preserved in methanol. If high target analyte concentrations are found in the low-level analyses, the methanol extract is run as a "high-level" analysis. In effect, the methanol-preserved sample is viewed as a dilution of the low-level analysis. However, are the results from the two analytical approaches comparable?

In a recent site investigation, soil samples containing high concentrations of numerous volatile organic compounds were analyzed by both the low-level and high-level procedures. For the seven soil samples in one sample delivery group, results for all analytes that were within the established calibration range (i.e., that were quantitatively valid) in both the low-level and the high-level analyses were compared. Twenty-three paired results, including both chlorinated and nonchlorinated analytes, were available for this evaluation. In every case, the concentration detected in the methanol extract was higher than the concentration found in the low-level purge analysis. Ratios of the high-level analysis results to the low-level analysis results ranged from 3.6 to 40.5; on average, the high-level results were higher than the low-level results by a factor of 16.2.

Perhaps, in addition to the presumed handling losses from the low-level purge method, methanol extraction is simply more efficient than purging for these particular compounds. Regardless, it is obvious that current low-level volatiles collection and analysis methods still do not accurately represent environmental field conditions.

The implications are crucial. Even if you comply with all agency-specified criteria for investigation and remediation of volatile organics in soil, you may be underestimating the contamination by more than an order of magnitude. If or when someone eventually figures that out, who do you think will get to do it all over again? Be careful!

Carol Erikson

Are Simple Linear Regressions Really Simple?

When comparing two sets of data, the most commonly used technique to demonstrate a correlation between the two sets is linear regression. Linear regression analysis is a mathematical procedure that yields, in addition to other significant information, a value known as the correlation coefficient. The correlation coefficient is often quoted as a measure of the linearity of the regression, that is, how well the two sets of data correspond to each other. A correlation coefficient of one (1.00) indicates a perfect correlation while a value less than unity indicates a correlation somewhat less than perfect. When using real world data, a perfect correlation is seldom achieved.

One of the areas in the environmental field in which linear regression is commonly used is in the calibration of analytical instruments. The process of calibrating an instrument involves analyzing known concentrations of a chemical species and noting the instrument responses to those concentrations. Responses and concentrations are then subjected to regression analysis. Correlation coefficients of 0.99 or higher are often required to demonstrate adequate correspondence between responses and concentrations to reliably estimate the unknown concentrations in samples whose responses fall within the range of the known concentrations.

However, correlation coefficients can often be misleading. For example, if the data in Table 1 are subjected to linear regression analysis, a correlation coefficient of 0.999 is obtained. The high correlation coefficient implies that an extremely good correlation exists between the concentrations and responses of the instrument. The plot of the data, as shown in Figure 1, tends to reinforce this concept. The data points for the higher concentrations appear to fall very close to the regression line. At this point, many laboratories would simply use the resultant equation for the regression line to calculate the unknown concentrations from responses from samples.

However, closer inspection of the lower end of the curve, as shown in Figure 2, indicates that deviations from the regression line exist.

Frequently, the lower end of the calibration range is either a method detection limit or a regulatory compliance standard. In this example, if a concentration of 2 were a compliance standard, a sample with a concentration of 2 would be reported to have a concentration of 1.1 and would, therefore, be below that standard. If the compliance standard were 1, a concentration very close to 2 could be present and still be reported as less than 1.

If a regression analysis were applied to the lower four concentrations in the above table, a correlation coefficient of 0.993 would be obtained that better represents the experimental data at the lower concentrations. Figure 3 shows this correspondence. If the response of 2.4 from the calibration data for the target concentration of 2 were used in this regression equation, a value of 2.1, not 1.1 as from the previous equation, for the found concentration would be obtained. If the response of 1.3 from the calibration data for the target concentration of 1 were used in the regression equation, a value of 1.3, not 0.1, for the found concentration would be obtained.

From the laboratory standpoint, extended calibration ranges are advantageous. Observing a response within a calibrated range means that additional dilutions of the sample are not necessary and sample throughput is increased. However, the primary responsibility of the laboratory is to provide the most accurate results to the client. To accomplish this, the laboratory must evaluate calibration curves to ensure accuracy is achieved over the calibrated range and particularly at concentrations of environmental concern.