The CPCAT as a novel tool to overcome the shortcomings of NOEC/LOEC statistics in ecotoxicology: a simulation study to evaluate the statistical power

Species reproduction is an important determinant of population dynamics. As such, this is an important parameter in environmental risk assessment. The closure principle computational approach test (CPCAT) was recently proposed as a method to derive a NOEC/LOEC for reproduction count data such as the number of juvenile Daphnia. The Poisson distribution used by CPCAT can be too restrictive as a model of the data-generating process. In practice, the generalized Poisson distribution could be more appropriate, as it allows for inequality of the population mean \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mu$$\end{document}μ and the population variance \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma ^2$$\end{document}σ2. It is of fundamental interest to explore the statistical power of CPCAT and the probability of determining a regulatory relevant effect correctly. Using a simulation, we varied between Poisson distribution (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mu =\sigma ^2$$\end{document}μ=σ2) and generalized Poisson distribution allowing for over-dispersion (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mu <\sigma ^2$$\end{document}μ\sigma ^2$$\end{document}μ>σ2). The results indicated that the probability of detecting the LOEC/NOEC correctly was \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\ge 0.8$$\end{document}≥0.8 provided the effect was at least 20% above or below the mean level of the control group and mean reproduction of the control was at least 50 individuals while over-dispersion was missing. Specifically, under-dispersion increased, whereas over-dispersion reduced the statistical power of the CPCAT. Using the well-known Hampel identifier, we propose a simple and straight forward method to assess whether the data-generating process of real data could be over- or under-dispersed.


Introduction
In environmental risk assessment scientists often focus on assessing the effects of chemicals on an ecological system or specific environmental compartments [1]. Species reproduction considerably affects population dynamics and ecology. As such, the new closure principle computational approach test (CPCAT) was proposed for the evaluation of discrete reproduction data [2]. Numbers of new Lemna fronds (Lemna minor L.), numbers of juvenile Daphnids (Daphnia magna), and numbers of fish eggs laid are popular examples of reproduction count data. It is well known that count and proportion data in ecotoxicology are not normally distributed [3]. The reproduction data mentioned above are generally assumed to be Poisson distributed [4,5].
Recently, a Poisson distribution together with CPCAT was used to test for differences in mean reproduction of different species [2]. Poisson distribution is known as the law of rare events [6]. Let µ represent mean reproduction and σ 2 the variance. The Poisson model implies equality of mean reproduction and variance, see Eq. (1): If a chemical substance affects mean reproduction, it affects variance, too. Consequently, effects on mean (1) µ = σ 2 .

Open Access
*Correspondence: rene.lehmann82@t-online.de 1 FOM Hochschule für Oekonomie & Management, Herkulesstraße, Essen, Germany Full list of author information is available at the end of the article reproduction cause inhomogeneous variances among treatments. Furthermore, normal approximation of Poisson distributed data is only valid if mean reproduction µ ≥ 5 holds (e.g., mean numbers of laid eggs ≥ 5 ). If a chemical substance reduces mean reproduction to near zero this normal approximation fails. Currently, the statistical power of CPCAT is unknown. As such, we conducted a simulation to assess how reliable the results obtained using CPCAT are. A key point of CPCAT is the assumption of a Poisson distribution. A Poisson distribution can be too conservative, as it implies expectation and variance to be equal [see Eq. (1)]. A generalized Poisson distribution allowing for over-and under-dispersion could be more appropriate. Using the Hampel identifier [7], we propose a simple and straight forward approach to assess whether observed data is over-or under-dispersed. The simulation included several scenarios of generalized Poisson distributed data and the statistical power of CPCAT was thus demonstrated. The probability of detecting the correct lowest observed effect concentration (LOEC) was also computed.

Materials and methods
The LOEC is defined as the lowest treatment concentration at which an effect was seen, for example, reproduction differed statistically significantly from the control group. The NOEC is defined as the highest concentration at which no effect was seen. If the lowest tested concentration significantly affects reproduction, it is concordant with the LOEC and no NOEC can be derived as a consequence. We, therefore, used LOEC values instead of NOEC values.

CPCAT: a short overview
The CPCAT is a combination of the closure principle (CP) [8] and the computational approach test (CAT) [9]. The mean reproduction of the control group is represented by µ 0 and the mean reproduction of the ith treated group is represented by µ i .
The CAT is used to test any (intersecting) hypothesis H 0,i 1 ,i 2 ,...,i r : µ 0 = µ i 1 = µ i 2 = · · · = µ i r , where i 1 , i 2 , . . . , i r represent specific treatment groups. Instead of normal approximation, the CAT applies an estimated Poisson distribution of the reproduction count data. It is based on a parametric bootstrap, i.e., control and treatment data are re-sampled according to their estimated underlying Poisson distributions. Maximum Likelihood estimators (MLE) are used to compute the test statistic.

Using the Hampel identifier as a pre-test for Poisson distribution
One feature of assuming a Poisson distribution is that the mean and variance are equal, see Eq. (1). If Eq. (1) is correct, then for the sample mean μ and the sample variance σ 2 follows also. That is, ( μ,σ 2 )-pairs of Poisson distributed data should scatter around a line of slope = 1 with an intercept = 0 (identity line). To identify data sets not fulfilling Eq. (1) distances of ( μ,σ 2 )-pairs from the identity line (i.e., residuals) should be considered. Using the Hampel identifier and a cut-off value of 4.3 ( α = 0.01 ) deviations from Eq. (1) can be determined [10]. Sachs [11] proposed a cut-off value of 5, independent of the sample size. Residuals exceeding the cut-off value correspond to ( μ,σ 2 )-pairs located far from the identity line (so-called outliers). Such ( μ,σ 2 )-pairs represent treatments that do not seem to fulfill Eq. (1). The underlying data-generating process is rather generalized Poisson. If

The generalized Poisson distribution
Poisson distribution can be too conservative, because it requires expectation µ and variance σ 2 to be equal. The generalized Poisson distribution allows for µ � = σ 2 . Let X be a generalized Poisson distributed random variable. The probability density function of X is presented in Eq.

Applicability of CPCAT in a generalized Poisson setting
Regarding the assumption of an underlying Poisson distribution the question arises whether CPCAT can be applied to generalized Poisson distributed data, too. A property of the Poisson distribution is that the population mean µ and the distribution parameter are equal. The MLE of µ under the null hypothesis is given by the sample mean. Moreover, the sample mean is an estimator of the first moment. Moment estimators are consistent and may be used as an approximation to MLE. They converge to the real underlying moments (e.g., population mean) as sample size increases and can be applied to every probability distribution [18]. Thus, CPCAT estimates the population mean µ approximately correct if the data-generating process is generalized Poisson distributed.

The simulation
The R-package ZIGP (version 1.3) was used for the generation of (generalized) Poisson distributed data. ZIGP uses a re-parametrization of the generalized Poisson model allowing for larger over-dispersion factors than possible in the standard parametrization. For details, refer to [19].
All simulated trials consisted of one control group ( i = 0 ) and k = 4 treatments, where each group contained 5 replicates. The number of simulations per scenario was N = 1000.
We examined different situations of increasing and decreasing trends in µ i -values. Setting µ 0 = µ 1 � = µ 2 expectation of the control and treatment 1 are identical. As a consequence, an effect was given by treatment 2.

Results and discussion of the simulation
The simulation was designed such that µ 0 = µ 1 � = µ 2 . That is, the lowest effect concentration (LEC) refers to treatment 2. Thus, the LOEC derived using CPCAT should equal the LEC. PROB is defined to be the probability of the event "LOEC=LEC", that is, the probability of deriving the correct LOEC. PROB values are presented in Tables 1, 2. The significance level was set to α = 0.05.
It can be seen that PROB was larger for rapidly increasing/decreasing trends between the control group and treatment 3 than for slowly. This result is not surprising, because the larger the difference between µ 0 and µ 2 the more easily the true LOEC can be derived.

Table 1 PROB of the CPCAT (decreasing trend)
Parameter c > (<)1 indicates generalized Poisson distribution with over-(under-)-dispersion ( σ 2 = c · µ ) or Poisson distribution ( c = 1 ). Parameter a indicates the value of the true LOEC via µ 2 = a · µ 0 . Parameter µ 0 denotes the mean reproduction of the control group     On the other hand, over-dispersion reduces statistical power. PROB depends on various parameters, namely, direction of the observed trend, magnitude of the effect, and steepness of the observed effects. PROB is larger for decreasing than for increasing trends. This is due the characteristics of the Poisson distribution. For µ 0 ≤ 75 , a difference |µ 0 − µ 2 | ≥ 15 provides PROB of approximately 0.8. For µ 0 ≥ 100 , a difference |µ 0 − µ 2 | ≥ 18 is required to obtain PROB ≈ 0.8 . Thus, PROB ≥ 0.8 if |µ 0 − µ 2 | ≥ 0.2µ 0 and µ 0 > 50 while over-dispersion is missing.
Mean reproduction varies between species. It can be low (e.g., number of laid bird eggs) or large (e.g., reproduction of Collembola Folsomia candida). We tried to cover a large range of mean reproduction values using a set of many different µ values. However, a simulation using a mean reproduction level greater than 150 × 1.7 = 255 could not be applied, because the number space of a computer is limited and infinite values were generated for µ > 255 (see "The simulation" section). For some species, it could be appropriate to assume over-or under-dispersion of reproduction data. We tried to choose a realistic range of dispersion factors c ∈ {0.1; 0.5; 1; 5; 10} . For example, in some real data, we found reproduction of Collembola (Folsomia candida) being over-dispersed by factor 10. On the other hand, if a substance reduces reproduction to nearly 0 dispersion will be reduced to nearly 0, too. Thus, a factor of 0.1 can be reasonable, too.

Conclusion
Statistical theory and results of the simulation indicated that the CPCAT is applicable and powerful provided µ = σ 2 or µ > σ 2 holds. In the case of µ < σ 2 , the statistical power is reduced.
To determine whether or not the data are over-or under-dispersed, the cut-off value of the Hampel identifier, as explained in "Using the Hampel identifier as a pretest for Poisson distribution" section should be used. The difference between any pair (μ,σ 2 ) and the identity line is given by σ 2 −μ . The Hampel identifier is relevant in real Over-dispersion reduces the probability of detecting regulatory relevant effects. From the simulation, we can estimate the loss of statistical power.
Overall, the CPCAT is applicable to generalized Poisson distributed data. A future version of CPCAT must explicitly take into account over-and under-dispersion, e.g., using MLE of the distribution parameters θ and .