Adaptive Filters (Vaseghi - Advanced Digital Signal Processing and Noise Reduction), страница 3

Let wo denote the optimal LSE filter coefficient vector, wedefine a filter coefficients error vector w˜ ( m) as~ (m) = w (m) − wwo(7.85)For a stationary process, the optimal LSE filter wo is obtained from theWiener filter, Equation (5.10), asw o = R −yy1r yx(7.86)Subtracting wo from both sides of Equation (7.84), and then substitutingRyywo for r yx , and using Equation (7.85) yields~ ( m + 1) = [I − µR ] w~ (m)wyy(7.87)It is desirable that the filter error vector w˜ (m) vanishes as rapidly aspossible. The parameter µ, the adaptation step size, controls the stabilityand the rate of convergence of the adaptive filter. Too large a value for µcauses instability; too small a value gives a low convergence rate.

Thestability of the parameter estimation method depends on the choice of theadaptation parameter µ and the autocorrelation matrix. From Equation(7.87), a recursive equation for the error in each individual filter coefficientcan be obtained as follows. The correlation matrix can be expressed interms of the matrices of eigenvectors and eigenvalues asR yy = QΛQ T(7.88)The Steepest-Descent Methodvk (m)2211– µλ kzvk (m+1)–1Figure 7.4 A feedback model of the variation of coefficient error with time.where Q is an orthonormal matrix of the eigenvectors of Ryy, and Λ is adiagonal matrix with its diagonal elements corresponding to theeigenvalues of Ryy.

Substituting Ryy from Equation (7.88) in Equation(7.87) yields~ (m + 1) = [I − µ QΛQ T ] w~ ( m)w(7.89)Multiplying both sides of Equation (7.89) by QT and using the relationQTQ=QQT=I yields~ ( m + 1) = [ I − µ Λ ] Q T w~ (m)QTw(7.90)Let~ ( m)v (m) = Q T wThenv(m + 1) = [I − µ Λ ] v(m)(7.91)(7.92)As Λ and Ι are both diagonal matrices, Equation (7.92) can be expressed interms of the equations for the individual elements of the error vector v(m)asvk (m + 1) =[1− µ λ k ]v k (m)(7.93)where λk is the kth eigenvalue of the autocorrelation matrix of the filterinput y(m). Figure 7.4 is a feedback network model of the time variations ofthe error vector.

From Equation (7.93), the condition for the stability of theadaptation process and the decay of the coefficient error vector is−1<1 − µλ k <1(7.94)Adaptive Filters222Let λmax denote the maximum eigenvalue of the autocorrelation matrix ofy(m) then, from Equation (7.94) the limits on µ for stable adaptation aregiven by20 <µ <(7.95)λ maxConvergence Rate The convergence rate of the filter coefficientsdepends on the choice of the adaptation step size µ, where 0<µ<1/λmax.When the eigenvalues of the correlation matrix are unevenly spread, thefilter coefficients converge at different speeds: the smaller the ktheigenvalue the slower the speed of convergence of the kth coefficients. Thefilter coefficients with maximum and minimum eigenvalues, λmax and λminconverge according to the following equations:v max ( m + 1) = (1− µ λ max )v max ( m )(7.96)v min ( m + 1) = (1− µ λ min )v min ( m )(7.97)The ratio of the maximum to the minimum eigenvalue of a correlationmatrix is called the eigenvalue spread of the correlation matrix:eigenvalue spread =λ maxλ min(7.98)Note that the spread in the speed of convergence of filter coefficients isproportional to the spread in eigenvalue of the autocorrelation matrix of theinput signal.7.5 The LMS FilterThe steepest-descent method employs the gradient of the averaged squarederror to search for the least square error filter coefficients.

Acomputationally simpler version of the gradient search method is the leastmean square (LMS) filter, in which the gradient of the mean square error issubstituted with the gradient of the instantaneous squared error function.The LMS adaptation method is defined asThe LMS Filter223y(m) e(m)µwk(m+1)α w(m)αz–1Figure 7.5 Illustration of LMS adaptation of a filter coefficient. ∂ e 2 (m) w ( m + 1) = w ( m ) + µ  − ∂ w (m) (7.99)where the error signal e(m) is given bye(m)= x(m) − w T (m) x (m)(7.100)The instantaneous gradient of the squared error can be re-expressed as∂e 2 (m)∂=[ x(m) − w T (m) y (m)]2∂w (m) ∂w (m)= − 2 y (m)[ x(m)− w T (m) y (m)]2= − 2 y ( m)e ( m)(7.101)Substituting Equation (7.101) into the recursion update equation of the filterparameters, Equation (7.99) yields the LMS adaptation equation:w (m + 1) = w (m) + µ [ y (m)e(m)](7.102)It can seen that the filter update equation is very simple.

The LMS filter iswidely used in adaptive filter applications such as adaptive equalisation,echo cancellation etc. The main advantage of the LMS algorithm is itssimplicity both in terms of the memory requirement and the computationalcomplexity which is O(P), where P is the filter length.Adaptive Filters224Leaky LMS Algorithm The stability and the adaptability of the recursiveLMS adaptation Equation (7.86) can improved by introducing a so-calledleakage factor α asw ( m + 1) =α w ( m ) + µ [ y ( m ) e ( m ) ](7.103)Note that the feedback equation for the time update of the filter coefficientsis essentially a recursive (infinite impulse response) system with inputµy(m)e(m) and its poles at α. When the parameter α<1, the effect is tointroduce more stability and accelerate the filter adaptation to the changesin input signal characteristics.Steady-State Error: The optimal least mean square error (LSE), Emin, isachieved when the filter coefficients approach the optimum value definedby the block least square error equation w o = R −yy1r yx derived in Chapter 6.The steepest-decent method employs the average gradient of the errorsurface for incremental updates of the filter coefficients towards the optimalvalue.

Hence, when the filter coefficients reach the minimum point of themean square error curve, the averaged gradient is zero and will remain zeroso long as the error surface is stationary. In contrast, examination of theLMS equation shows that for applications in which the LSE is non-zerosuch as noise reduction, the incremental update term µe(m)y(m) wouldremain non-zero even when the optimal point is reached.

Thus at theconvergence, the LMS filter will randomly vary about the LSE point, withthe result that the LSE for the LMS will be in excess of the LSE for Wieneror steepest-descent methods. Note that at, or near, convergence, a gradualdecrease in µ would decrease the excess LSE at the expense of some loss ofadaptability to changes in the signal characteristics.7.6 SummaryThis chapter began with an introduction to Kalman filter theory.

TheKalman filter was derived using the orthogonality principle: for the optimalfilter, the innovation sequence must be an uncorrelated process andorthogonal to the past observations. Note that the same principle can alsobe used to derive the Wiener filter coefficients. Although, like the Wienerfilter, the derivation of the Kalman filter is based on the least squared errorcriterion, the Kalman filter differs from the Wiener filter in two respects.Summary225First, the Kalman filter can be applied to non-stationary processes, andsecond, the Kalman theory employs a model of the signal generationprocess in the form of the state equation.

This is an important advantage inthe sense that the Kalman filter can be used to explicitly model thedynamics of the signal process.For many practical applications such as echo cancellation, channelequalisation, adaptive noise cancellation, time-delay estimation, etc., theRLS and LMS filters provide a suitable alternative to the Kalman filter. TheRLS filter is a recursive implementation of the Wiener filter, and, forstationary processes, it should converge to the same solution as the Wienerfilter.

The main advantage of the LMS filter is the relative simplicity of thealgorithm. However, for signals with a large spectral dynamic range, orequivalently a large eigenvalue spread, the LMS has an uneven and slowrate of convergence. If, in addition to having a large eigenvalue spread asignal is also non-stationary (e.g. speech and audio signals) then the LMScan be an unsuitable adaptation method, and the RLS method, with itsbetter convergence rate and less sensitivity to the eigenvalue spread,becomes a more attractive alternative.BibliographyALEXANDER S.T.