A. Wood - Softare Reliability. Growth Models (798489), страница 8
Текст из файла (страница 8)
Second set, based on Poisson distribution, from Equation (A6) in Appendix 1.Table 3-13. Confidence Interval Comparison for Release 43.7Grouped Data StabilityAs mentioned in Section 2.1.3, we have access only to weekly data rather than exact defectdetection data. To simulate exact data, we took the data for Release 4 and distributed thedefects throughout the week in which they arrived. We did this randomly (using therandom number generator in Excel) and using a fixed pattern, e.g., 2 failures in a weekwere assumed to be evenly spaced - at 2.3 and 4.7 days.
Table 3-14 shows the results. Wethought that having this extra data might cause the prediction to stabilize sooner, but thatwas not the case. Note that the predictions from the simulated exact data and the weeklygrouped data are essentially identical. Our conclusion is that the weekly grouped dataworks fine for our development process. This is useful because it means that we do nothave to try to change the data input and collection processes.25TestWeek10111213141516171819ExecutionHoursNo. ofDefects6,0037,6218,7839,60410,06410,56011,00811,23711,24311,305Predicted TotalNo.
of Defects Grouped Data2932323638393941424284534445464848505152Predicted TotalNo. of Defects Ungrouped Data(Random)149605746474748494950Predicted TotalNo. of Defects Ungrouped Data(Fixed)116615446474848494950Table 3-14. Release 4 Results for Ungrouped Data3.8Rerun Test HoursThe test hours that have been used in all the models include test hours for tests that wererun a fIrst time and test hours for tests that were rerun. Tests are rerun to either ensure thata defect has been correctly repaired or that fIxing a defect did not cause another failure.
Onehypothesis is that the rerun test hours are less likely to fInd defects than tests run for thefIrst time because the tests have once been successfully (usually) executed. We tested thishypothesis by counting rerun test hours at only 50% of the value of the initial test executiontime. We fIt the G-O model to the adjusted hours for Release I with the results shown inTable 3-15. The results do not indicate that this technique is any better than our usualtechnique for fItting the data.
We also tried different factors such as 25% for adjusting thedata but did not get any better results than we did with the original data. This, togetherwith our excellent results from treating all execution hours as equivalent, leads us toconclude that there is no need to adjust the test hours for modeling our data.TestWeek1012141720No. ofDefects75869399100Predicted Total No. of Defects - Predicted Total No. of Defects Rerun hours discounted by 50% Rerun Test Hours Not Discounted9897116116129134139157133159Table 3-15. Release 1 Results for Discounting Rerun Test Hours26AcknowledgmentI would like to thank Jocelyn Miyoshi and Helen Cheung for their help in acquiring andanalyzing the data. I would also like to thank Jeff Terry for many enlighteningconversations about the Tandem software development and QA processes.References1.
[Ehrlich,90] Ehrlich, Willa, S. Keith Lee, and Rex Molisani, "Applying ReliabilityMeasurement: A Case Study". IEEE Software, March 1990, pp 56-64.2. [Goel,79] Goel, A. L. and Kazuhira Okumoto, "A Time Dependent Error DetectionModel for Software Reliability and Other Performance Measures", IEEE Trans. Reliability,vol R-28, Aug 1979, pp 206-211.3. [Hossain,93] Hossain, Syed and Ram Dahiya, "Estimating the Parameters of a NonHomogeneous Poisson-Process Model of Software Reliability", IEEE Trans. Reliability,vol 42, No.4, Dec 1993, pp 604-612.4. [Kececioglu,91] Kececioglu, Dimitri, Reliability Engineering Handbook. Volume 2,Prentice-Hall, 1991.5.
[Lee,93] Lee, Inhwan and Ravishankar K. Iyer, "Faults, Symptoms, and SoftwareFault Tolerance in the Tandem GUARDIAN90 Operating System", Proceedings of the23rd International Symposium on Fault-Tolerant Computing (FfCS-23), Toulouse,France, June 22-24, 1993.6. [Littlewood,81] Littlewood, B., "Stochastic Reliability Growth: A Model for FaultRemoval in Computer Programs and Hardware Design", IEEE Trans.
Reliability, vol R30, Dec 1981, pp 313-320.7. [Mood,74] Mood, Alexander, Franklin Graybill, and Duane Boes, Introduction to theTheory of Statistics, McGraw-Hill, 1974.8. [Musa,87] Musa, John, Anthony Iannino, and Kazuhira Okumoto, Software Reliability,McGraw-Hill, 1987.9. [Yamada,83] Yamada, Shigeru, Mitsuru Ohba, and Shunji Osaki, "S-Shaped ReliabilityGrowth Modeling for Software Error Detection", IEEE Trans. Reliability, vol R-32, Dec1983, pp 475-484.10. [Yamada,86] Yamada, Shigeru, Hiroshi Ohtera, and Hiroyuki Narihisa, "SoftwareReliability Growth Models with Testing Effort", IEEE Trans.
Reliability, vol R-35, No.1,April 1986, pp 19-23.27Appendix 1Least Squares CalculationsAn alternative to classical least squares is to minimize the likelihood function directly ratherthan the log-likelihood function. In this case, we assume that the difference between thecumulative number of defects in week i and the model prediction for week i is a normallydistributed random variable with mean O. In other words,fi = Jl(ti) + Ei, whereEj are independent, identically distributed, normal random errors with mean 0 and.2common vanance 0 .The least squares technique is to minimize the sum of the squared errors, Le., minimize:WWL j =IEj 2 = L j =1 (fj - Jl(1j))2, which is Equation (6) in Section 2.3.3, withw =current number of weeks of QA testli = cumulative test time at the end of the ith weekfj cumulative number of failures at the end of the ith week.=For the G-O model, Equation (6) becomes:(AI)wbL j =l(fj - a(1_e- t;))2, which is Equation (7) in Section 2.3.3.The confidence interval for a assuming that a is nonnally distributed is(A2)a ± t w - 2 ,al2 (Var(a))O.5 , which is Equation (5) in Section 2.3.2.The calculation of the variance of a is described in [Musa,87,Section 12.3] for the classicalleast squares technique described in Section 2.3.2.
In this Appendix, the calculation of thevariance of a for the alternative least squares technique is derived.2The assumptions that the errors Ei are normally distributed with a common variance 0means that the following equation can be used for the variance of a and b (see e.g.[Musa,87,Equation 12.150]):(A3)Var(a) = (02 /SS ) L jw =I[(O/Ob) (a(l_e- bti ))]2, where0 2 L jW =1 (fj - a(l-e-bt; ))2 l(w-2)SS = a 2 LjW=I(l_e-bti)2 LjW=lt/(l-e-bti)2 - a 2 {LjW=ltj(1_e-bti)2}2=(o/ob) is a partial derivative with respect to b, so (o/ob) (a(l_e-bti ) = alie-btj.(A4)Var(b)=(02/SS) L w =I[(%a) (a(l-e-bt;))f, wherejbtj(o/oa) is a partial derivative with respect to a, so (o/Ba) (a(l_e-bti ) = 1_e- .To find a confidence interval for a, Equation (A2) can be used directly by assuming that a isnormally distributed and substituting from Equation (A3).
This method was used to derivethe first set of confidence intervals reported in Section 3.6. Alternately, Equation (A4) can28be used to derive a confidence interval for a assuming that b is nonnally distributed, and wecan fmd a confidence interval for a by substituting the confidence limits for b into:A different method for calculating confidence intervals is to assume that fj is a Poissonrandom variable with expected value J.1(!j). Then, from P.270 of [Musa,87], the a per centconfidence interval (e.g., 95%) for fi is given by:J.1(!j) ± Zl-lXI2 J.1(tj )O.5, which, since J.1(oo) =a, becomes:(A6)a± Zl-lXI2 aO.
5 .Equation (A6) was used to derive the second set of confidence intervals reported in Section3.6. This is not a very satisfying fonn for a confidence interval because it depends only onthe parameter and not on the data used to derive the parameter.Appendix 2Parameter ScalingIn this appendix, we show that the parameter estimates scale linearly for all statisticaltechniques. Also, we show that the confidence intervals for the least squares method scalelinearly while the confidence intervals for the maximum likelihood method do not.
Linearparameter scaling means that if we multiply the number of defects observed by a factor k ,1then the estimate of the total number of defects will also be scaled by a parameter k . It is1very important that the parameter estimates scale linearly since the data has been scaled. Ifthey did not, then the data translation technique used throughout this report would not beuseful because other researchers could not duplicate the results. It is also important that testtime scaling does not affect the predicted total number of defects, Le., if we scale the testtime by a factor k , then the estimate of the total number of defects will be unchanged.2Confidence interval scaling means that the confidence intervals should scale by the samefactor as the total number of defects.
For example if the estimated total defects parameterwas 100 with upper and lower confidence intervals of 80 and 120 and we multiplied thenumber of defects observed in each test interval by 2, then the scaled total defects parametershould be 200 and the scaled confidence intervals should be 160 to 240. It is lessimportant, but unfortunate, that the confidence intervals do not scale linearly for themaximum likelihood technique.For the maximum likelihood technique, the following equation must be solved (see Section2.3.1):~ W (fi- f i-I )( tie -bt-'-!j.I e -bt; -1)/(e -bt-'-e-bt-,- 1) = f wtw/( 1 -ew,werebt) h(A7) ~i=Iw =current number of weeks of QA test!j = cumulative test time at the end of the ith weekfi =cumulative number of failures at the end of the ith week.If we replace the number of failures fi by k/i and replace the test time ti by k 2!j, the samesolution is obtained for Equation (A7) if the parameter b is replaced by b1k2 .
FromEquation (3) in Section 2.3.1, the total defects parameter is thenk f W /(1 - e-(bIk2)(k2tw») = k f W /(1 - e- btw ) = k aIIl'where a is the total defects parameter obtained prior to scaling the data. Thus, the totaldefects parameter scales linearly.29From Equation (2) in Section 2.3.1, the a per cent confidence interval (e.g., 95%) for b isgiven by:(AS) b ± ZI_an/(Io(b))O.5, whereI (b) = ~ W (f. _f. )(t. _t. )2 -b(tj +tj.1)/(e,bti.1_e- bti )2 _ f t 2e btw/(ebtw _ 1)2j =1 1 1-1 1 1-1 eW W°If we replace the number of failures f j by k f j and replace the test time ~ by k z~ in the12equation for lo(b), the result is k k 10(b). If we then substitute into Equation (AS), the1 zresult isb/k2 ± Zl-al2l(k\k22loeb)) 0.5 = (l/k2)[b ± Zl.al2l (k 1lo(b))0.5] ·.Note that the extra factor of k in the denominator prevents the confidence mterval for bfrom scaling linearly.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
