HELP (Программа GPSS), страница 7

13.2 Experimentation and The Analysis of Variance

Next we explore some of the technical considerations that are relevant to experimentation in GPSS World. This section presents a discussion of the main issues that need to be considered when you invoke the ANOVA Procedure or when you generate an Experiment.

13.2.1 Motivation

Simulations are not completely faithful to the real world system they are intended to represent. Instead, they are merely representations intended to capture the most important behavioral characteristics of the target system.

One major difference with the real world is that simulations are perfectly repeatable. That is, if we repeat a simulation many times, we get precisely the same results every time. Such a thing is extremely unlikely in the natural sciences. In fact, it is so unnatural that we intentionally introduce variability into our simulations in order to make them look more realistic. In practice, we often find simulations in which the random variation in modeled processes, artificially introduced by us, is essential for capturing the behavior of the target system.

Another source of variability that does not occur in simulations is that due to the measurement itself. Whereas in the real world some measurement tools are very noisy and troublesome, in the simulated environment we enjoy a god-like perspective where all things are ultimately knowable without disturbing the measured environment. The point is that the observations from simulations are unnaturally crisp, and may occasionally reveal effects that exist in the real world but are extremely difficult to observe there.

Unlike experiments conducted in the real world, in simulation we have much better control over the variability introduced to mimic the apparent randomness of repeated real world measurements. Generally, we introduce one or more streams of “pseudo” random numbers, which are able to pass certain statistical tests of randomness. We go even further when we select probability distributions thought to accurately reflect the target system’s behavior. GPSS World provides over twenty of these, to be selected and used by the simulation analyst. In our quantitative analysis of simulation results, we will assume we have successfully modeled the random variance of measured values in the real world in this way.

The Analysis of Variance, or ANOVA, is a highly developed methodology for extracting information from the results of an experiment. In essence, it breaks up the variance of observations, and associates the pieces with the experimental factors and their interactions. One part, called the Error term, is associated with the unavoidable intrinsic randomness of the observations. In one sense, ANOVA attempts to account for all the variation of the observations from their average. That which is left over, the “Error”, is a variation that has not been accounted for by association with the experimental conditions. It is an estimate of the intrinsic variability of observations called the Standard Error. Usually, only those effects that exceed the magnitude of the Standard Error are presumed to be real and not due to random fluctuations. A test using the “F Statistic” is normally used to ascertain this.

We must be careful to distinguish the variance introduced for realism from the variance encountered in our statistical models, the Standard Error. Generally, we do not want to distort the former because it will often cause our simulation to miss important real world behavior. On the other hand, the purpose of Design of Experiments is to reduce the unaccounted variance in our statistical analysis. This is quite a different matter. Variance Reduction Techniques, which distort variance introduced for realism, are to be avoided in the methodology we are presenting here. Our goal is to model real-world randomness accurately, not to reduce it.

We are, however, motivated to find ways to reduce the variability of the observations in the Analysis of Variance, as long as we do not distort the way we represent natural randomness. Reducing the estimate of the Standard Error in an Analysis of Variance allows significant effects to emerge above the statistical noise level. Two methods of doing so, which we will discuss below, improve the residual data used in the estimate of the Standard Error. The first by increasing the number of observations, the second by changing the underlying statistical model. These are discussed in Section 13.2.3.

13.2.2 Nomenclature

When we conduct an experiment we begin by selecting one or more metrics that quantify the state of the system or some other outcome of interest. Measurements of these quantities are called “observations” or "yields", and the set of all observations comprises the “results” of the experiment.

We will examine one or more forces called “factors” that are believed to influence the value of some of the observations. We will assign values called “treatment levels” to the factors when we specify the conditions for each execution of the simulation. When there are multiple factors, we say that the conditions are specified by “

treatment combinations”, since multiple treatment levels must be specified. If the influence of some factor differs when the treatment level of some other factor is varied, we say that there is an “interaction” between the two factors. Since we will be using an additive model, when the effects of two factors together is not the sum of their separate effects, we say that there is an interaction between them. There can be distinct interactions involving any number of factors.

We will simulate the target environment multiple times calling each instance a “run”. The treatment combination specifies the conditions of the run, and one or more observations form the results of the run. When the conditions of a set of runs are the same except for randomization, we say that the runs in the set are “

replicates”, and form a "cell" of the experiment.

In the course of the Analysis of Variance, done for you by GPSS World, the observations are partitioned into components, called “effects” which are presumed to be due to the influence of the factors and their interactions. How this partition is done depends on an underlying additive model called the “statistical model”. An intrinsic random deviation causes the observations to differ slightly from the sum of effects in the statistical model. The “

error term” in an observation is derived by subtracting estimates of the effects corresponding to terms in the statistical model. Since the error is the quantity left over, it is often called the “residual”.

For multiway experiments, that is those with more than one factor, the Analysis of Variance in GPSS World requires that the experiment be orthogonal in order to complete the analysis. This means that the estimators within the analysis must be uncorrelated. In practice, providing the same number of runs within each treatment combination of a balanced design guarantees orthogonality.

Effects, as denoted by letter groups, have important uses in the design of fractional factorial experiments, discussed below. To fractionate a factorial design, a small number of effects are chosen by the designer as “

Generators” in order to specify the set of runs in the experiment. Unfortunately, this also causes some effects to become indistinguishable, or “aliased”, with others. In GPSS World, the set of generators separated by equal (=) signs is called the “defining relation” of the experiment.

GPSS World supports experimentation through internal library procedures and through PLUS, the embedded Programming Language Under Simulation. PLUS is both a low-level procedural language accessible from within simulations, and a high-level control language that can direct the conditions and sequence of runs in an experiment.

13.2.3 ANOVA

The Analysis of Variance is a tool, pioneered by Sir Ronald Fisher, which is able to extract much of the information available in a set of measurements. In it, we quantify the variation of observations from the overall average, and then break it into pieces, each of which has a separate cause. If any experimental factor cannot be found to induce variability in a measurement, we say that it does not have a significant effect on it. On the other hand, if a factor does appear to induce variability, we compare the amount of it to an estimate of the intrinsic variability of the observation, the Standard Error. We do this to rule out apparent effects that are nothing more than random fluctuation. Our standard of comparison is that the variation from any source must be much larger than the Standard Error in order to be deemed a significant effect. The F test, named for Fisher, is used for this purpose. We use the F test as the criterion by which we declare the effects of experimental factors and their interactions to be statistically significant.

Implicit in the use of ANOVA is the existence of an additive mathematical model used to explain the components of variation in the observations. We will call this the “statistical model”. The simplest statistical model is given in Figure 13-1.

yi = m + ei

Figure 13-1. A Simple Statistical Model

In this example, each observation is broken down into only two components, the grand mean of all observations, m, and the random component, e. Each observation has a starting point, the grand population mean, and then incurs a random deviation leading to its final value. The grand mean does not vary from observation to observation, whereas the random component does, and is subscripted accordingly. Although this model is suggestive, it allots all variation of the observations to random sources and none to factors, and so is not very useful in analyzing the results of an experiment.

Next, we turn to a statistical model used to analyze the data from an experiment with a single factor, namely, factor A.

yi,j = m + ai + ei,j

Figure 13-2. One Factor Statistical Model

In Figure 13-2, notice the introduction of the subscripted alpha term, which denotes the effect of the ith treatment level of the one and only factor considered in the model. The experiment would include one or more runs at each of the treatment levels of factor A. All observations at a given treatment level are analyzed using the same value for a. Since there is only one factor in this experiment, the number of treatment combinations is just the number of treatment levels of that factor. An Analysis of Variance based on this statistical model will result in an ANOVA table that partitions the variation of the observations into that due to treatment A, and that due to the random variation.

yi,j,k = m + ai + bj + i(ab)i,j + ei,j,k

Figure 13-3. Two Factor Statistical Model

In Figure 13-3, we move to a model where two factors are considered. Notice that we now include a term for the interaction between factor A and factor B, denoted i(ab). This term may differ at each treatment combination, and therefore is subscripted twice since there are two factors. Recall from above that when the effects of two factors together is not the sum of their separate effects, there is an interaction between them. The strength of the interaction between factors A and B is denoted “AB” and would be reported in a line of the ANOVA table.

In a complete factorial experiment all interaction terms would be included. Just as the complete 2-factor model in Figure 13-3 has 5 (RHS) terms, a complete 3-factor model has 8 terms, and a complete 4-factor model has 16 terms. All this information is normally presented in the resulting ANOVA table. GPSS World can handle up to 6-factor models with up to and including 3-way interactions.

Figure 13-4, below, presents an ANOVA table reported by GPSS World. As explained above, the single capital letters denote factors and capital letter combinations denote interactions.

First, look at the bottom of the table. The Total Sum of Squares is to be partitioned, and the components are to be associated with the effects of factors and their interactions. Anything that is left over, that is, the residual sum of squares, is shown in the previous line, labeled “Error”. The Mean Sum of Squares of the error term is used to estimate the Standard Error of the experiment.

Each Sum of Squares has a divisor associated with it called the Degrees of Freedom. From statistical considerations, the Degrees of Freedom is that divisor that must be used to create an unbiased estimator of the Standard Error, in the absence of other effects. For our purposes, it is sufficient to think of Degrees of Freedom as the proper divisor associated with a Sum of Squares in the ANOVA table. GPSS World will always calculate the Degrees of Freedom for you.

Figure 13-4. An ANOVA Table

Each factor and interaction in the statistical model is represented by a distinct line in the ANOVA table. In each line we have first the Sum of Squares and the Degrees of Freedom associated with that estimate. These are the basics from which the other numbers are derived. A simple division results in the Mean Square, and by dividing that quotient by the Mean Square of the Error, from the bottom row of the table, we get the F statistic for that effect.