Thompson - Computing for Scientists and Engineers (523188), страница 37
Текст из файла (страница 37)
Then,since a2(0) andare reciprocal, the second inequality follows immediately. The inequalities are reversed if Sxy < 0, which is (6.41). n6.3 STRAIGHT-LINE LEAST SQUARES197Since a value of the slope intermediate between the two extremes is attained forany ratio of y to x weights, you might guess that the geometric mean of the extreme values of would often give an appropriate value for the slope a2.Exercise 6.13(a) Consider the slopeas a function of the ratio of weights, . By usingthe results in (6.39) and (6.38) for a2(0) andshow that their geometricmean iswith sign that of Sxy,(b) Insert this result for the slope into the defining equation (6.35) for a2 toshow that this choice of the geometric mean corresponds to a specific choice ofthe relative weights, namely(6.42)This value is determined by the data and not by any estimates of the relative errors.
So the geometric-mean choice of slopes is usually not appropriate. nThe weighting-model parameterdefined in (6.19) should be estimated, for example, by repeated measurements of some x and y values in order to estimate theirstandard deviations, which should then be used in (6.19). Because this is a tediousprocess, which scientists in the heat of discovery are seldom willing to spend muchtime on, it is of interest to investigate the dependence of the slope on Within theIDWMC model (Figure 6.2), this can be done without reference to particular databy appropriately scaling the variables. To this end, we introduce the weighted correlation coefficient between x and y variables, p, defined by(6.43)From the Schwartz inequality we have that |p| < 1. Further, introduce the ratiobetween weights and the geometric weight, v, defined by(6.44)By using the dimension-free variables p and v just defined, we can derive from theslope equation (6.35) the more-general form, which is again free of all dimensionalconsiderations:(6.45)198LEAST-SQUARES ANALYSIS OF DATAExercise 6.14(a) In (6.35) eliminate the S variables in favor of the variables appearing in(6.43) and (6.44) in order to derive (6.45).(b) Show algebraically that if the factor greatly exceeds its geometric-mean estimatethen the slope ratio in (6.45) tends to p.(c) Show that if is much less than its geometric-mean estimatethen theslope ratio in (6.45) tends to l/p.
nFrom this exercise we see that the ratio of slopes exhibits complete reflection symmetry with respect to the weighting ratio v about the value v = 1. The relationship(6.45) for interesting values of the correlation coefficient p is illustrated in Figure 6.4. This figure and the formula (6.45) have a simple explanation, as follows.For p = 0.1, x and y are only weakly correlated, so the slope is strongly dependenton the weights. For p = 0.9, they are strongly correlated, so the slope becomes almost independent of the weight ratio.As a final topic in straight-line least squares with errors in both variables, wegive the first form of the minimum value obtained by the y contribution to the objective function,in (6.22), namely(6.46)FIGURE 6.4 Dependence of the least-squares slope on the ratio of x weights to y weights in theIDWMC weighting model (Figure 6.2) according to (6.45).
The slope dependence is shown forthree values of the weighted correlation coefficient p = 0.1 (weakly correlated data), p = 0.5(moderate correlation), and p = 0.9 (strong correlation).6.4 LEAST-SQUARES NORMALIZATION FACTORS199Actually, this formula provides an unstable method (in the sense of Section 4.3) forfor the following reason.
We know that the limitiscomputingjust that for negligible y errors (OLS - x:y in Figure 6.2) and is obtained by usingthe second slope formula (6.37), which is well-behaved asTherefore weare not dealing with an unstable problem in the sense discussed in Section 4.3.
In(6.46), however, the denominator is divergent in the limitPart (b) of Exercise 6.15 shows you how to remove this difficulty.Exercise 6.15(a) Substitute in expressions (6.22) for the objective function and (6.37) for thebest-fit slope a2 in order to derive (6.46).(b) Since, as just discussed, thebehavior in (6.46) cannot hold analytically (as contrasted to numerical behavior), the best way to avoid this unstablemethod is to eliminate from this expression by solving for it from (6.35), thensubstituting its solution in (6.46). By performing some algebraic simplification,show that the y contribution to the minimum objective function can be rewrittensimply as(6.47)(c) Show that if(negligible y errors), then, by using (6.37) for a2, thiscontribution to the objective function vanishes and only the part from any x errors remains.
Is this result expected? nFormula (6.47) is used in the straight-line least-squares-fit function in Program 6.2(Section 6.6). Note that the minimum value of the objective function obtained byusing (6.47) depends upon the overall scale factor for the weights. Even if theweight sum S w given in (6.28) is divided out, one still usually does not have thestatistical chi-squared, because of the weighting scheme used. Therefore, interpretations of the probability content of such an objective function (such as physicists’favorite claim for a good fit that “the chi-squared per point is less than one”) are often not meaningful.Estimating the uncertainties in the slope and the intercept when both variableshave errors involves some advanced concepts from statistics. Derivations of suchestimation formulas are given in the article by Isobe et al., and the formulas arecompared with Monte Carlo simulations in the article by Babu and Feigelson.6.4 LEAST-SQUARES NORMALIZATION FACTORSA common problem when analyzing data is to obtain a realistic estimate of best-fitnormalization factors between data and fitting-function values.
Such a factor may beapplied to the fit values as an overall normalization, Nf, that best matches the data orit may be applied to the data, as Nd, to best match the fitting values. Under mostbest-fit criteria Nd is not the reciprocal of Nf If the normalization is assumed to be200LEAST-SQUARESANALYSISOFDATAindependent of other parameters in the fitting (as when data are scaled to a model tocalibrate an instrument), the procedures that we derive here will decrease the numberof fitting parameters by one. Indeed, I worked up this section of the text after realizing while writing Section 7.2 on world-record sprints that one of the fitting parameters (the acceleration A) was linearly related to the data. The formula is seldom derived in books on statistics or data analysis, and then only for the case of errors inthe y variables.In this section we derive simple expressions for Nf and Nd that can be usedwith weighted least squares when there are errors in both variables.
The formulasare of general applicability and are not restricted to straight-line or linear leastsquares fitting. Also, they do not depend strongly on other assumptions about thefitting procedure, such as the weighting models used. The formulas are not specialcases of determining the slope in a straight-line least-squares fit of the Yj to the yj,or vice versa, because here we constrain the intercept to be zero, whereas in astraight-line fit the analysis determines the intercept. After the best-fit normalizationshave been computed, the corresponding best-fit objective function can be obtainedfrom a simple formula that we derive. We work in the context of bivariate problems(x and y independent variables), then at the end indicate the extension to multivariate analyses.In the first subsection we set up the problem and the method of solution for finding Nf.
The same procedure is then indicated for Nd. We next derive the formulafor the resulting minimized value of the objective function. In the final subsectionwe develop Least Squares Normalization for computing the normalizationsand objective functions, together with suggestions for testing and applying it.Normalizing fitting-function values to dataAs you will recall from (6.6) in Section 6.1, the objective function to be minimizedin a least-squares fit is usually chosen as(6.48)in which the weights may be chosen in various ways, as discussed in Section 6.3.In (6.48) the (x, y) pairs are the data and the (X, Y) pairs are the corresponding fitting values. Suppose that for each of the data points the fitting function is renormalized by a common factor Nf That is,(6.49)where the Y value on the right-hand side is assumed to have normalization factor ofunity, which explains the superscript (1) on it.
We want a formula for a value of Nf,common to all the N data points, so that is minimized. This requires using onlystraightforward calculus and algebra, so why not try it yourself?6.4 LEAST-SQUARES NORMALIZATION FACTORS201Exercise 6.16(a) Substitute (6.49) in (6.48), differentiate the resulting expression with respectto Nf, then equate this derivative to zero. Thus derive the expression for Nf thatproduces an extremum of namely(6.50)(b) Take the second derivative of with respect to Nf and show that this derivative is always positive (assuming, as usual, that all the weights are positive).Thus argue that the extremum found for must be a minimum.
nFormula (6.50) is a general-purpose result for normalizing the fitting function, Y, tobest match the data, y. It is used by the program in Section 7.2 for fitting data onworld-record sprints.A simple check to suggest the correctness of (6.50) is that for a single data point,N = 1, it gives Nf = y1/Y1(1), the fit for the y variables is exact, and only differences in x values (if any) contribute to the objective functionThis seeminglytrivial result is quite useful for checking out the program Least Squares Normalization that is developed below.Normalizing data to fitting valuesIt is important to know that the overall best-fit normalization constant to be applied todata, Nd, is usually not simply obtained as the reciprocal of Nf obtained from (6.50).Rather, if data are to be normalized to the fitting function, we must interchange theroles of yj and Yj in describing the normalization.
Therefore, we set(6.5 1)in which the y data on the right-hand side are assumed to have unity for normalization, as indicated by the superscript (1). The procedure for determining the optimumNd value is very similar to that in the preceding subsection, so it’s left for you towork out.Exercise 6.17(a) Substitute (6.51) in (6.48) and differentiate the expression with respect toNd. By equating this derivative to zero, derive the expression for the normalization of the data, Nd, needed to produce an extremum of namely(6.52)202LEAST-SQUARES ANALYSIS OF DATA(b) Differentiateonce more with respect to Nd to show that this derivative isalways negative if all the weights are positive. Thus show that the extremumfound for is a minimum.(c) Show by comparison of the formulas (6.52) and (6.50) that Nd and Nf aregenerally not reciprocal. Show that a sufficient condition for reciprocity is thatthe fitting-function values Yj and the data values yj are all proportional to eachother with the same proportionality constant.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
