Nash - Compact Numerical Methods for Computers (523163), страница 15
Текст из файла (страница 15)
This is because we use a completecycle of Givens’ rotations using the diagonal elements W[i,j], j := 1 ton, of the work array to zero the first n elements of row nobs of the(implicit) matrix A. In the example, row 1 is rotated from row 4 to row 1since W is originally null. Observation 2 is loaded into row 4 of W, butthe first Givens’ rotation for this observation will zero the first TWOelements because they are the same scalar multiple of the correspondingelements of observation 1. Since W[2,2] is zero, as is W[4,2], the secondGivens’ rotation for observation 2 is omitted, whereas we should move thedata to row 2 of W.
Instead, the third and last Givens’ rotation forobservation 2 zeros element W[4,3] and moves the data to row 3 of W.In the least squares problem such permutations are irrelevant to the finalsolution or sum of squared residuals. However, we do not want therotations which are only used to move data to be incorporated into U.Unfortunately, as shown in the example above, the exact form in which suchrotations arise is not easy to predict. Therefore, we do not recommendthat this algorithm be used via the rotations to compute the svd unlessthe process is restructured as in Algorithm 3.
Note that in any event alarge data array is needed.The main working matrix W must be n+l by n+nRHS in size.Copyright 1988 J. C. Nash}varcount, EstRowRank, i, j, k, m, slimit, sweep, tcol : integer;bb, c, e2, eps, p, q, r, s, tol, trss, vt : real;enddata : boolean;endflag : real;procedure rotnsub; {to allow for rotations using variables localto Givsvd. c and s are cosine and sine ofangle of rotation.}vari: integer;r: real;58Compact numerical methods for computersAlgorithm 4. Givens’ reductions, singular-value decomposition and least-squares solution (cont.)beginfor i := m to tcol do {Note: starts at column m, not column 1.}beginr := W[j,i];W[j,i] := r*c+s*W[k,i];W[k,i] := -r*s+c*W[k,i];end;end; {rotnsub}begin {Givsvd}writeln(‘alg04.pas -- Givens’,chr(39),‘reduction, svd, and least squares solution’);Write(‘Order of 1s problem and no.
of right hand sides:’);readln(infile,n,nRHS); {STEP 0}if infname<>‘con’ then writeln(n,‘ ’,nRHS);write(‘Enter a number to indicate end of data’);readln(infile,endflag);if infname<>‘con’ then writeln(endflag);tcol := n+nRHS; {total columns in the work matrix}k := n+1; {current row of interest in the work array during Givens’ phase}for i := l to n dofor j := 1 to tcol doW[i,j] := 0.0; {initialize the work array}for i := 1 to nRHS do rss[i] := 0.0; {initialize the residual sums of squares}{Note that other quantities, in particular the means and the totalsums of squares will have to be calculated separately if otherstatistics are desired.}eps := calceps; {the machine precision}tol := n*n*eps*eps; {a tolerance for zero}nobs := 0; {initially there are no observations]{STEP 1 -- start of Givens’ reduction}enddata := false; {set TRUE when there is no more data.
Initially FALSE.}while (not enddata) dobegin {main loop for data acquisition and Givens’ reduction}getobsn(n, nRHS, W, k, endflag, enddata); {STEP 2}if (not enddata) thenbegin {We have data, so can proceed.} {STEP 3}nobs := nobs+1; {to count the number of observations}write(‘Obsn’,nobs,‘ ’);for j := 1 to (n+nRHS) dobeginwrite(W[k,j]:10:5,‘ ’);if (7 * (j div 7) = j) and (j<n+nRHS) then writeln;end;writeln;for j := 1 to (n+nRHS) dobegin {write to console file}end;for j := 1 to n do {loop over the rows of the work array tomove information into the triangular part of theGivens’ reduction} {STEP 4}beginm := j; s := W[k,j]; c := W[j,j]; {select elements in rotation}Handling larger problems59Algorithm 4.
Givens’ reductions, singular-value decomposition and least-squares solution (cont.)bb := abs(c); if abs(s)>bb then bb := abs(s);if bb>0.0 thenbegin {can proceed with rotation as at least one non-zero element}c := c/bb; s := s/bb; p := sqrt(c*c+s*s); {STEP 7}s := s/p; {sin of angle of rotation}if abs(s)>=tol thenbegin {not a very small angle} {STEP8}c := c/p; {cosine of angle of rotation}rotnsub; {to perform the rotation}end; {if abs(s)>=tol}end; {if bb>0.0}end; {main loop on j for Givens’ reduction of one observation} {STEP 9}{STEP 10 -- accumulate the residual sums of squares}write(‘Uncorrelated residual(s):’);for j := 1 to nRHS dobeginrss[j] := rss[j]+sqr(W[k,n+j]); write(W[k,n+j]:10,‘ ’);if (7 * (j div 7) = j) and (j < nRHS) thenbeginwriteln;end;end;writeln;{NOTE: use of sqr function which is NOT sqrt.}end; {if (not enddata)}end; {while (not enddata)}{This is the end of the Givens’ reduction part of the program.The residual sums of squares are now in place.
We could find theleast squares solution by back-substitution if the problem is of fullrank. However, to determine the approximate rank, we will continuewith a row-orthogonalisation.}{STEP 11} {Beginning of svd portion of program.}m := 1; {Starting column for the rotation subprogram}slimit := n div 4; if slimit< then slimit := 6; {STEP 12}{This sets slimit, a limit on the number of sweeps allowed.A suggested limit is max([n/4], 6).}sweep := 0; {initialize sweep counter}e2 := l0.0*n*eps*eps; {a tolerance for very small numbers}tol := eps*0.1; {a convergence tolerance}EstRowRank := n; {current estimate of rank};repeatcount := 0; {to initialize the count of rotations performed}for j := 1 to (EstRowRank-1) do {STEP 13}begin {STEP 14}for k := (j+1) to EstRowRank dobegin {STEP 15}p := 0.0; q := 0.0; r := 0.0;for i := 1 to n dobeginp := p+W[j,i]*W[k,i]; q := q+sqr(W[j,i]); r := r+sqr(W[k,i]);end; {accumulation loop}svs[j] := q; svs[k] := r;60Compact numerical methods for computersAlgorithm 4.
Givens’ reductions, singular-value decomposition and least-squares solution (cont.){Now come important convergence test considerations.First we will decide if rotation will exchange order of rows.}{STEP 16 If q<r then goto step 19}{Check if the rows are ordered.}if q>= r thenbegin {STEP 17 Rows are ordered, so try convergence test.}if not ((q<=e2*svs[1]) or (abs(p)c=tol*q)) then{First condition checks for very small row norms in BOTH rows,for which no rotation makes sense. The second conditiondetermines if the inner product is small with respect to thelarger of the rows, which implies a very small rotation angle.}begin {revised STEP 18}{columns are in order, but not converged to smallinner product.Calculate angle and rotate.}p := p/q; r := 1-r/q; vt := sqrt(4*p*p + r*r);c := sqrt(0.5*(1+r/vt)); s := p/(vt*c);rotnsub; {STEP 19 in original algorithm}count := count+1;end;endelse {q<r, columns out of order -- must rotate}{revised STEP 16.
Note: r > q, and cannot be zero since both aresums of squares for the svd. In the case of a real symmetricmatrix, this assumption must be questioned.}beginp := p/r; q := q/r-1; vt := sqrt(4*p*p + q*q);s := sqrt(0.5*(1-q/vt));if p<0 then s := -s;c := p/(vt*s);rotnsub; {STEP 19 in original algorithm}count := count+1;end;{Both angle calculations have been set up so that large numbersdo not occur in intermediate quantities. This is easy in the svdcase, since quantities q and r cannot be negative.}{STEP 20 has been removed, since we now put the number ofrotations in count, and do not count down to zero.}end; {loop on k -- end-loop is STEP 21}end; (loop on j)sweep := sweep +1;writeln( ‘Sweep’,sweep,‘ ’,count,‘rotations performed’);{Set EstColRank to largest column index for whichsvs[column index] > (svs[1]*tol + tol*tol)Note how Pascal expresses this more precisely.}while (EstRowRank >= 3) and (svs[EstRowRank] <= svs[1]*tol+tol*tol)do EstRowRank := EstRowRank-1;until (sweep>slimit) or (count=0); {STEP 22}{Singular value decomposition now ready for extraction of informationand formation of least squares solution.}writeln(‘Singular values and principal components’);for j := 1 to n do {STEP 23}begins := svs[j];Handling larger problems61Algorithm 4.
Givens’ reductions, singular-value decomposition and least-squares solution (cont.)s := sqrt(s); svs[j] := s; {to save the singular value}writeln(‘Singular value[‘,j,’]= ’,s);if s>=tol thenbeginfor i := 1 to n do W[j,i] := W[j,i]/s;for i := 1 to n dobeginif (8 * (i div 8) = i) and (i<n) thenbeginwriteln;end;end; {for i=1...}{principal component is a column of V or a row of V-transpose. Wstores V-transpose at the moment.}writeln;end; {if s>=tol}{Principal components are not defined for very small singular values.}end; {loop on j over the singular values}{STEP 24 -- start least squares solution}q := 0.0; {to ensure one pass of least squares solution}while q>=0.0 dobeginwrite(‘Enter a tolerance for zero (<0 to exit)’);readln(infile,q);if infname<>‘con’ then writeln(q);if q>=0.0 thenbegin{For each value of the tolerance for zero entered as q we mustcalculate the least squares solution and residual sum of squaresfor each right hand side vector.
The current elements in columnsn+1 to n+nRHS of the work array W give the contribution of eachprincipal coordinate to the least squares solution. However, we donot here compute the actual principal coordinates for reasonsoutlined earlier.}for i := 1 to nRHS do {STEP 25}begintrss := rss[i]; {get current sum of squared residuals}for j := 1 to n do {loop over the singular values -- STEP 26}begin {STEP 27}p := 0.0;for k := l to n dobeginif svs[k]>q then p := p+W[k,j]*W[k,n+i]/svs[k];end; {loop over singular values}B[j,i] := p; {to save current solution -- STEP 28}writeln(‘Solution component [‘,j,’]= ’,p);if svs[j]<=q then trss := trss+sqr(W[j,n+i]); {to adjust theresidual sum of squares in the rankdeficient case}end; {loop on j -- end-loop is STEP 29}writeln(‘Residual sum of squares = ’,trss);end; {loop on i -- end-loop is STEP 30}end; {if q>=0.0}end; {while q>=0.0}end; {alg04.pas -- Givens’ reduction, svd, and least squares solution}62Compact numerical methods for computersExample 4.2.
The use of algorithm 4In the first edition, a Hewlett-Packard 9830A desk computer was used to solve aparticular linear least-squares regression problem. This problem is defined by thedata in the file EX04.CNM on the software diskette. Using the driver programDR04.PAS, which is also on the diskette according to the documentation in appendix4, gives rise to the following output.dr04.pas -- run Algorithm 4 problems -- Givens’ reduction,1989/06/03 16:09:47File for input of control data ([cr] for keyboard) ex04.cnmFile for console image ([cr] = nu1) out04.alg04.