HELP (Программа GPSS), страница 8

Now we are ready to draw some conclusions. We must decide if the F value is large enough to declare that the effect is significant. The threshold value we will use to make the comparison is called the “Critical Value of F” and is placed just to the right of our F statistic on the same line. If our F value exceeds the Critical Value, we conclude that we are dealing with a significant effect. If not, we conclude that the effect is not significant and we disregard any associated variation in observations as only due to random noise. The larger the F value the stronger the effect. The ANOVA table in Figure 13-4 shows the effect of factor A to be significant and the effects from factor B and the AB interaction to be not significant.

Sometimes an experiment will fail to detect an effect even though one actually exists. One of our goals is to make this as unlikely as possible. According to the ANOVA table, there are two ways to make an experiment better able to detect real effects. To get more positive results we would need either a larger F statistic or a smaller Critical Value of F.

It is desirable to remove any part of the Sum of Squares of the Error that is due to any important effect we have not included in the analysis. If we can do this, our F statistic will generally be larger. In experimentation in the natural sciences, this is done by making comparisons in as homogenous an environment as possible, a technique known as “blocking”. However, in simulation studies this goal is perhaps best approached by identifying additional factors that must be included in the experiment.

Two other approaches are directed at increasing the Degrees of Freedom of the Error term. The first is simply to increase the number of replicates in the experiment. This is usually the most expensive approach but it can be quite effective. The second possibility relates to the design of the experiment and the statistical model behind the Analysis of Variance. The Mean Square of the Error is actually a residual term left after all other squares have been removed. If we can find an acceptable way to allow more of the data to remain after the effects are removed, we will have an estimate of the Standard Error that has more degrees of freedom. The resulting Critical Value of F will be smaller, thereby increasing the power of the analysis. This is what we are attempting to do when we choose to ignore some of the interactions.

None of the techniques we are considering will distort the randomness introduced to simulate real-world randomness. Variance Reduction Techniques that do should not be used here. Intentionally reducing the intrinsic variability of the observations would cause the F statistics to be overstated.

In many multifactor experiments we will chose to remove the highest order interactions in order to improve the information available on the main factors and the low order interactions. You can do this using the third argument of the ANOVA Procedure. If it is true that these interaction effects do not exist, this will allow GPSS World to use the extra degrees of freedom to achieve a better estimate of the F statistic. In addition, more Degrees of Freedom means a smaller critical value of F, as well. However, we must realize that removing terms from the statistical model assumes that there is no high-order effect. If there is a reasonable chance of one, we would be better off adding replicates to improve the statistics rather than limiting the analysis to low order interactions. Replicates are made known to the ANOVA Procedure in its second argument.

yi,j,k = m + ai + bj + ei,j,k

Figure 13-5. Two Factor Statistical Model without Interactions

Figure 13-5 shows a statistical model in which the 2-way interaction term was removed. If it is true that the interaction effect does not exist, we have improved the crispness of the Analysis of Variance by providing more Degrees of Freedom for the Mean Square of the Error. When you choose to apply this technique, you simply change the 3rd argument of the call to the GPSS World ANOVA library procedure to remove higher order interactions from the analysis. This built-in procedure can handle up to 3-way interactions, and allows you to give up 2-way or 3-way interactions, and above, to improve the significance testing of the other effects.

13.2.4 Selecting Factors

When you use the ANOVA library procedure you must define a GPSS Matrix Entity, called the Result Matrix, to store the individual results of each run. If you were analyzing more than one metric, you would have more than one Result Matrix. Since GPSS World Matrix Entities can have up to 6 dimensions, each of any size, the Result Matrix is also limited to 6 dimensions.

When you write your own GPSS World Experiment, the most important decision you will make is to select the factors of the experiment. These are the quantities that you can control in order to optimize your system. Each factor is to be represented by a User Variable that takes on treatment levels as values. After you run each simulation in the experiment, you must place the resulting value into a Results Matrix so that the data can be analyzed by the ANOVA Procedure.

The position in the Result Matrix for each result is determined by the treatment combination. For example, if the experiment considers 4 machine types and two speeds of each, the result of simulating the third machine at high speed would go into the Result Matrix at position [3,2]. Actually, since replicates are usually tagged onto the end, the first run at this treatment combination would go into element [3,2,1] of the Result Matrix. Each dimension of the Result Matrix is analyzed as a factor by the ANOVA library procedure. The size of the dimension in the Result Matrix must be at least as large as the number the treatment levels of that factor, or of the maximum replicate count.. There is no arbitrary limit to the number of treatment levels within a factor, only that imposed by the virtual memory of your computer.

We can examine any effect by giving it it’s own dimension in the Result Matrix. If we so desire, we could even separate random number streams and treat them individually as distinct factors. Each random number seed would represent a distinct treatment level. This would provide information on the relative importance of each random input process. Usually, though, we use a single dimension for replicates, and vary all seeds from one replicate to the next within a single treatment combination. We then make known to the ANOVA Procedure which dimension is used for replicates. This is done in the second argument.

Not all experiments need a Replicate Dimension. The ANOVA is often able to extract an estimate of the Standard Error without one. The Latin Square sample model, “Latin Square.gps” is an example of this. However, depending on the design of the experiment, sometimes there is not enough residual data to estimate the Standard Error. Then the Analysis of Variance will not be able to calculate F statistics for the ANOVA Table. In such cases, you will have to either add replicate runs to the experiment or remove high level interactions from the statistical model in order to provide enough residual data for the estimate of the Standard Error. Both of these methods require a change in the call to the ANOVA library procedure.

13.2.5 The Random Number Streams

A guideline often used in the design of experiments is this: control all factors that you can; what you can’t control, block; what you can’t block, randomize. This applies to the use of the random streams in a discrete event simulation, as well.

To treat random number streams as factors is to allocate a dimension for each stream, and to use each distinct seed as a treatment level. Although it is possible to view the random number streams as factors in this way, and to examine their relative effects, it may be best done as a separate study. For the mainstream investigation of the main effects and interactions, we are more interested in using our random number streams to represent sources of natural and unavoidable variability, reserving the dimensions of the Result Matrix for the factors being studied.

The most common approach combines all random number streams into a single pseudo-factor, where each treatment level is a set of distinct seeds for all random number streams. A single dimension, the Replicate Dimension, is then allocated from the Result Matrix for this purpose and is made known to the ANOVA library procedure. This has some similarities with blocking, where all comparisons are performed within each homogeneous environment. The ANOVA library procedure can then treat this Replicate Dimension separately to better estimate the Standard Error of the experiment. The experiment then consists of an experimental "cell" for each treatment combination, with several replicate runs within each cell.

If these approaches are too expensive, or there isn’t a dimension to spare in the Result Matrix, the runs should be “completely randomized”. This means that randomization should not be restricted in any way, and that the random number seeds should not be repeated within a single random number stream. If this procedure fails to provide enough Degrees of Freedom for the analysis, you may need to limit the level of interactions to be included in the statistical model, as well.

13.3 GPSS World Features

We turn now to the to the specific features of GPSS World which can be used in any of the experimentation phases of a simulation project. First, we consider the features provided for the analysis of User Experiments. GPSS World provides for two kinds of automatically generated experiments that are designed for you, and User Experiments that are designed by you.

13.3.1 User Experiments

The immediate goal of a User Experiment is to provide the information needed by the built-in ANOVA library procedure. That means that you must fill a Result Matrix with all the results of the experiment. If you prefer, you can do this directly through the use of Command Lists and manual entries. However, the PLUS Language, which allows you to write programmable experiments can make your life a lot easier if your experiment is complex.

You should begin by selecting the factors whose effects you will be studying. For example, you may want to measure the effect of the number of phone lines on the average waiting time in a telephone exchange. Factor A of your experiment would be the number of phone lines, and you would assign it to the first dimension of your Result Matrix. Next we decide how many treatment levels (i.e. number of phone lines) will be studied. Let's say that we want to simulate exchanges with 4, 8, and 16 lines. We would then need 3 slots in the Result Matrix for these three conditions. Therefore the first dimension in the Result Matrix would have a size of 3.

We continue to add factors to the experiment in this way, using the number of treatment levels as the size of the corresponding dimension in the Result Matrix. Normally, we use the last dimension for replicates. The size of the Replicate Dimension must obviously be large enough to contain the results from any cell (i.e. treatment combination) in the experiment. The only restriction is that the total number of dimensions cannot exceed 6. When you analyze the data, you must explicitly tell the ANOVA routine which dimension is used for replicates.

Remember to initialize the Result Matrix to UNSPECIFIED, before you begin storing results into it. That way, it is clear to the ANOVA routine when a run has not been performed.

When you call the ANOVA routine with your results, you must also decide if you want to exclude 3-way, or even 2-way, interactions from the analysis. If you can justifiably do this, you will improve the statistics in the ANOVA table.

That is a rough outline of the creation of User Experiments. In the remainder of this section we will look at the steps in more detail. We will consider the embedded PLUS programming language, the requirements and restrictions of the Result Matrix, and the use of the ANOVA library procedure. Lesson 19 of the GPSS World Tutorial Manual guides you through an example.

Experiments created using the automatic experiment generators are not considered User Experiments, and are discussed later. However, you may find it useful to study the programming of a generated experiment before you attempt to do it yourself.

13.3.2 PLUS Experiments

PLUS is a simple but powerful programming language that is an important part of the GPSS World simulation environment. It began as a source of custom programmable subroutines to be accessed during GPSS simulations, and has progressed to become a control language that can direct the details of the runs in an experiment, and report on the results. The complete description of PLUS is in Chapter 8 of this manual. Lesson 17 of the GPSS World Tutorial Manual covers it, as well.

An Experiment is a PLUS Procedure where the keyword PROCEDURE is replaced by the keyword EXPERIMENT. Experiments can do everything that normal Procedures can, and a whole lot more. An Experiment is invoked by a CONDUCT Command which names the invoked Experiment and passes it any arguments it might require.

The power of a PLUS Experiment is based on its ability to invoke the "DoCommand" library procedure. Nearly all GPSS Commands can be executed by a DoCommand invocation with a Command String as an argument. Not only can an Experiment invoke DoCommand, but also so can any normal PLUS Procedure that has been called during the execution of an Experiment. This adds a lot of flexibility to the programming of complex experiments.