roots (Программы с лекций (2013))

Fitting Zero-Inflated Count Data Models byUsing PROC GENMODOverviewCount data sometimes exhibit a greater proportion of zero counts than is consistent with the data havingbeen generated by a simple Poisson or negative binomial process. For example, a preponderance of zerocounts have been observed in data that record the number of automobile accidents per driver, the numberof criminal acts per person, the number of derogatory credit reports per person, the number of incidencesof a rare disease in a population, and the number of defects in a manufacturing process, just to name a few.Failure to properly account for the excess zeros constitutes a model misspecification that can result in biasedor inconsistent estimators.Zero-inflated count models provide one method to explain the excess zeros by modeling the data as amixture of two separate distributions: one distribution is typically a Poisson or negative binomial distributionthat can generate both zero and nonzero counts, and the second distribution is a constant distribution thatgenerates only zero counts.

When a zero count is observed, there is some probability, called the zero-inflationprobability, that the observation came from the always-zero distribution; the probability that the zero camefrom the Poisson/negative binomial distribution is 1 minus the zero-inflation probabilty. When the underlyingcount distribution is a Poisson distribution, the mixture is called a zero-inflated Poisson (ZIP) distribution;when the underlying count distribution is a negative binomial distribution, the mixture is called a zero-inflatednegative binomial (ZINB) distribution.This example demonstrates how to fit both ZIP and ZINB models by using the GENMOD procedure.The SAS source code for this example is available as an attachment in a text file. In Adobe Acrobat,right-click the icon in the margin and select Save Embedded File to Disk.

You can also double-click toopen the file immediately.source codeAnalysisCount data that have an incidence of zeros greater than expected for the underlying probability distribution canbe modeled with a zero-inflated distribution. The population is considered to consist of two subpopulations.Observations drawn from the first subpopulation are realizations of a random variable that typically has eithera Poisson or negative binomial distribution, which might contain zeros. Observations drawn from the secondsubpopulation always provide a zero count.Suppose the mean of the underlying Poisson or negative binomial distribution is and the probability of anobservation being drawn from the constant distribution that always generates zeros is !.

The parameter ! isoften called the zero-inflation probability.2 FThe probability distribution of a zero-inflated Poisson random variable Y is given by(! C .1 !/e for y D 0Pr.Y D y/ Dye .1 !/ yŠfor y D 1; 2; : : :The mean and variance of Y for the zero-inflated Poisson are given byE.Y / D D .1 !/!Var.Y / D C21 !The parameters ! and can be modeled as functions of linear predictors,h.!i / D z0i g.i / D x0i ˇwhere h is one of the binary link functions: logit, probit, or complementary log-log. The log link function istypically used for g.The excess zeros are a form of overdispersion.

Fitting a zero-inflated Poisson model can account for theexcess zeros, but there are also other sources of overdispersion that must be considered. If there are sourcesof overdispersion that cannot be attributed to the excess zeros, failure to account for them constitutes a modelmisspecification, which results in biased standard errors. In a ZIP model, the underlying Poisson distributionfor the first subpopulation is assumed to have a variance that is equal to the distribution’s mean.

If this is aninvalid assumption, the data exhibit overdispersion (or underdispersion).A useful diagnostic tool that can aid you in detecting overdispersion is the Pearson chi-square statistic.Pearson’s chi-square statistic is defined as2 Di /2V .i /X .yiiThis statistic, under certain regularity conditions, has a limiting chi-square distribution, with degrees offreedom equal to the number of observations minus the number of parameters estimated. Comparing thecomputed Pearson chi-square statistic to an appropriate quantile of a chi-square distribution with n pdegrees of freedom constitutes a test for overdispersion.If overdispersion is detected, the ZINB model often provides an adequate alternative.

The probabilitydistribution of a zero-inflated negative binomial random variable Y is given byExample: Trajan Data Set F 3(Pr.Y D y/ D1! C .1 !/.1 C k/ k€.yC1=k/.k/y.1 !/ €.yC1/€.1=k/.1Ck/yC1=kfor y D 0for y D 1; 2; : : :where k is the negative binomial dispersion parameter.The mean and variance of Y for the zero-inflated negative binomial are given byE.Y / D D .1 !/!kVar.Y / D CC21 !1 !Because the ZINB model assumes a negative binomial distribution for the first component of the mixture, ithas a more flexible variance function.

Thus it provides a means to account for overdispersion that is not due tothe excess zeros. However, the negative binomial, and thus the ZINB model, achieves this additional flexibilityat the cost of an additional parameter. Thus, if you fit a ZINB model when there is no overdispersion, theparameter estimates are less efficient compared to the more parsimonious ZIP model.

If the ZINB modeldoes not fully account for the overdispersion, more flexible mixture models can be considered.Example: Trajan Data SetConsider a horticultural experiment to study the number of roots produced by a certain species of appletree. During the rooting period, all shoots were maintained under identical conditions, but the shootsthemselves were cultured on media that contained four different concentration levels of the cytokinin 6benzylaminopurine (BAP), in growth cabinets with an 8 or 16 hour photoperiod (Ridout, Hinde, and Demétrio1998). The objective is to assess the effect of both the photoperiod and the concentration levels of BAP onthe number of roots produced.The analysis begins with a graphical inspection of the data. The following DATA step reads the data andTable 1 summarizes the variables in the data set Trajan.data Trajan;input roots shoot photoperiod bap;lshoot=log(shoot);datalines;0 40 8 17.60 40 8 17.60 30 16 2.20 30 16 2.2...

more lines ...4 F1314141417;3040404030888884.48.88.817.62.2Table 1 Trajan Data SetVariable NameDescriptionRootsShootLshootPhotoperiodBAPNumber of rootsNumber of micropropogated shootsNatural logarithm of the number of shootsEight- or 16-hour photoperiodConcentrations of the cytokinin 6-benzylaminopurine (BAP)The FREQ procedure is then used to produce plots of the marginal and conditional distributions of theresponse variable Roots.ods graphics on;proc freq data=Trajan;table roots / plots(only)=freqplot(scale=percent);run;Inspection of Figure 1 reveals a percentage of zero counts that is much larger than what you would expect toobserve if the data were generated by simple Poisson or negative binomial processes.Figure 1 Marginal Distribution of Response Variable RootsThe following SAS statements produce plots of the distribution of Roots conditional on Photoperiod:Example: Trajan Data Set F 5proc sort data=Trajan out=Trajan;by photoperiod;run;proc freq data=Trajan;table roots / plots(only)=freqplot(scale=percent);by photoperiod;run;Figure 2 Distribution of Roots Conditional on PhotoperiodPhotoperiod = 8Photoperiod = 16Figure 2 reveals that under the 8-hour photoperiod, almost all of the shoots produced roots.

In fact, conditionalon Photoperiod=8, the distribution appears consistent with the data having been generated by a simple Poissonor negative binomial process. However, under the 16-hour photoperiod, almost half of the shoots produced noroots. This provides compelling evidence that the data generating process is a mixture and that the probabilityof observing a zero count is conditional on the photoperiod.The following SAS statements produce plots of the distribution of Roots conditional on BAP:proc sort data=Trajan out=Trajan;by bap;run;proc freq data=Trajan;table roots / plots(only)=freqplot(scale=percent);by bap;run;6 FFigure 3 Distribution of Roots Conditional on BAPBAP = 2.2BAP = 4.4BAP = 8.8BAP = 17.6Figure 3 reveals differences in the modes and the skew of the conditional distributions.

It is reasonable toconclude that the expected value of Roots is a function of the level of BAP. However, there is little variationin the percentage of zero counts in these conditional distributions, suggesting that BAP is probably not apredictor of the probability of a zero count.The following SAS statements produce plots of the distribution of Roots conditional on Photoperiod andBAP:proc sort data=Trajan out=Trajan;by photoperiod bap;run;proc freq data=Trajan;table roots / plots(only)=freqplot(scale=percent);by photoperiod bap;run;Example: Trajan Data Set F 7Figure 4 Distribution of Roots Conditional on Photoperiod and BAPPhotoperiod = 8 and BAP = 2.2Photoperiod = 8 and BAP = 4.4Photoperiod = 8 and BAP = 8.8Photoperiod = 8 and BAP = 17.6Photoperiod = 16 and BAP = 2.2Photoperiod = 16 and BAP = 4.4Photoperiod = 16 and BAP = 8.8Photoperiod = 16 and BAP = 17.6The conditional distributions in which Photoperiod = 8 reveal some differences in the modes and skew.

Theconditional distributions in which Photoperiod = 16 are dominated by the large percentages of zero counts.There is some indication of interaction effects, but it is difficult to predict whether they are significant.To summarize, the graphical evidence indicates that a simple Poisson or negative binomial model will notlikely account for the prevalence of zero counts and that a mixture model such as a zero-inflated Poisson (ZIP)8 Fmodel or zero-inflated negative binomial (ZINB) is needed. There is also clear evidence that the probabilityof a zero count depends on the level of Photoperiod.The following SAS statements use the GENMOD procedure to fit a zero-inflated Poisson model to theresponse variable Roots.proc genmod data=Trajan;class bap photoperiod;model roots = bap|photoperiod / dist=zip offset=lshoot;zeromodel photoperiod;output out=zip predicted=pred pzero=pzero;ods output Modelfit=fit;run;The CLASS statement specifies that the variables Photoperiod and BAP are categorical variables.