Transforms and Filters for Stochastic Processes (779449), страница 2
Текст из файла (страница 2)
Approximation0.20.81'00.20.40.6t -10.8t+Figure 5.2. Examples of sample functions; (a) typical signal contours; (b) twosample functions and their approximations.and-N(5.30)Observing the a priori probabilities of the two classes, p1 and p 2 , a process2=PlZl+(5.31)P222can be defined. The covariance matrix R,, can be estimated asNR,, = E { x x ~ M}CN+1,P1-NixlixTa= 1P2+CN + 1 ,a= 1ix2 ix;,(5.32)where i x l and ix2 are realizations of the zero-mean processes x1 andrespectively.The first ten eigenvalues computed from a training set are:X1X2X3X5X4X6X7X8X922,X10968 2551 3139We see that by using only a few eigenvectors a good approximation canbe expected.
To give an example,Figure 5.2 shows two signals andtheir1095.3. The KLT of Real-Valued AR(1) Processesapproximations(5.33)with the basis{ul,u2,u3,~4}.In general, the optimality andusefulness of extracted featuresfor discrimination is highly dependent on the algorithm that is used to carry out thediscrimination. Thus, the feature extraction methoddescribed in this exampleis not meant to be optimal for all applications.
However, it shows how a highproportion of information about a process can be stored within afew features.For more details onclassification algorithms and further transformsfor featureextraction, see [59, 44, 167, 581.5.3The KLT of Real-Valued AR(1) ProcessesAn autoregressiwe process of order p (AR(p) process) is generated by excitinga recursive filter of order p with a zero-mean, stationary white noise process.The filter has the system function1H ( z )=c p ( i ) z-iP1-P ( P ) # 0.>(5.34)i=lThus, an AR(p) process ).(Xis described by the difference equationV+ C p ( i ) X(.= W(.)).(X- i),(5.35)i=lwhere W(.)is white noise. The AR(1) process with difference equation).(X= W(.)+ p X(.- 1)(5.36)is often used as a simple model.
It is also known as a first-order Markowprocess. From (5.36) we obtain by recursion:).(X=c p iW(.- i).(5.37)i=OFor determining the variance of the process X(.),mw = E { w ( n ) }= 0+we use the propertiesm, = E { z ( n ) }= 0(5.38)110Chapter 5. Tkansforms and FiltersStochasticfor Processesand?-,,(m)= E {w(n)w(n+ m)} = 02smo,where SmO is the Kronecker delta. Supposing IpI(5.39)< 1, we geti=O-U21- p2’For the autocorrelation sequence we obtaini=OWe see thatthe autocorrelationsequence is infinitely long.
However,henceforth only the values rzz(-N l),.... T,,(N - 1) shall be considered.Because of the stationarity of the input process, the covariance matrix of theAR(1) process is a Toeplitz matrix. It is given by+o2R,, = -(5.42)1 - p2The eigenvectors of R,, form the basis of the KLT.
For real signals andeven N , the eigenvalues Xk, Ic = 0,. ... N - 1 and the eigenvectors wereanalytically derived by Ray and Driver [123]. The eigenvalues are1Xk=I1- 2 p cos(ak)+ p2 ’k=O,.... N - 1 ,(5.43)1115.4. Whitening Transformswhereak,5 = 0 , . . . ,N- 1 denotestan(Nak) = -the real positive roots of(1 - p’) sin(ak)cos(ak) - 2p p COS(Qk).+(5.44)The components of the eigenvectors u k , k = 0 , . .
. ,N - 1 are given by5.4Whitening TransformsIn this section we are concerned with the problem of transforming a colorednoise process into a white noise process. That is, the coefficients of therepresentation should not only be uncorrelated (as for the KLT), they shouldalso have the same variance. Such transforms, known as whitening transforms,are mainly applied in signal detection and pattern recognition, because theylead t o a convenient process representation with additive white noise.Let n be a process with covariance matrixRnn = E { n n H }#a21.(5.46)We wish t o find a linear transform T which yields an equivalent processii=Tn(5.47)wit hE{iiiiH}= E { T n n H T H=} TR,,TH = I .(5.48)We already see that the transform cannot be unique since by multiplying analready computed matrix T with an arbitrary unitary matrix,property (5.48)is preserved.The covariance matrix can be decomposed as follows (KLT):R,, = U A U H = U X E H U H .For A and X we have(5.49)Chapter 5.
Tkansforms and FiltersStochasticfor Processes112Possible transforms areT = zP1UH(5.50)T =U T I U H .(5.51)orThis can easily be verified by substituting (5.50) into (5.48):(5.52)Alternatively, we can apply the Cholesky decompositionR,, = L L H ,(5.53)where L is a lower triangular matrix. The whitening transform isT = L-l.(5.54)For the covariance matrix we again haveE{+inH)= T R , , T ~= L - ~ L L H L H - '= I .(5.55)In signal analysis, one often encounters signals of the formr=s+n,(5.56)where S is a known signal and n is an additive colored noise processes. Thewhitening transforms transfer (5.56) into an equivalent modelF=I+k(5.57)withF= Tr,(5.58)I = Ts,ii= Tn,where n is a white noise process of varianceIS:1.=1135.5.
Linear Estimation5.5Linear EstimationInestimationthegoal is to determine a set of parametersas preciselyas possible from noisy observations. Wewill focus on the case where theestimators are linear, that is, the estimates for the parameters are computedas linear combinations of the observations. This problem is closely related tothe problem of computing the coefficients of a series expansion of a signal, asdescribed in Chapter 3.Linear methods do not require precise knowledge of the noise statistics;only moments up to the second order are taken into account.
Therefore theyare optimal only under the linearity constraint, and, in general, non-linearestimators with better properties may be found. However, linear estimatorsconstitutethe globally optimal solution as far as Gaussian processes areconcerned [ 1491.5.5.1Least-Squares EstimationWe consider the modelr=Sa+n,(5.59)where r is our observation, a is the parameter vector in question, and n is anoise process. Matrix S can be understood as a basis matrix that relates theparameters to the clean observation S a .The requirement to have an unbiased estimate can be written asE { u ( r ) l a }= a,(5.60)where a is understood as an arbitrarynon-random parameter vector.
Becauseof the additive noise, the estimates u ( r ) l aagain form a random process.The linear estimation approach is given byh(.) = A r .(5.61)If we assume zero-mean noise n , matrix A must satisfyA S = I(5.62)114Chapter 5. Tkansforms and FiltersStochasticfor Processesin order to ensure unbiased estimates. This is seen fromE{h(r)la} = E { A rla}= A E{rla}= AE{Sa+n}=(5.63)A S aThe generalized least-squares estimator is derived from the criterion!.= mm,(5.64)CY = &(r)where an arbitrary weighting matrix G may be involved in the definition ofthe inner product that induces the norm in (5.64). Here the observation r isconsidered as a single realization of the stochastic process r .
Making use ofthe fact that orthogonal projections yield a minimal approximation error,wegeta(r) = [SHGS]-lSHGr(5.65)according to (3.95). Assuming that [SHGS]-lexists, the requirement (5.65)to have an unbiased estimator is satisfied for arbitrary weighting matrices, ascan easily be verified.If we choose G = I , we speak of a least-squares estimator. For weightingmatrices G # I , we speak of a generalized least-squares estimator. However,the approach leaves open the question of how a suitable G is found.5.5.2The Best Linear Unbiased Estimator (BLUE)As will be shown below, choosing G = R;:, whereR,, = E { n n H }(5.66)is the correlation matrix of the noise, yields an unbiased estimatorwithminimal variance.
The estimator, which is known as the best linear unbiasedestimator (BLUE), then isA = [SHR;AS]-'SHR;A.(5.67)The estimate is given byu ( r )=[s~R;AsS]-~S~R;Ar.(5.68)1155.5. Linear EstimationThe variances of the individual estimates can be found on the main diagonalof the covariance matrix of the error e = u ( r )- a, given byR,, = [SHRiAS]-'.Proof of (5.69) and theoptimalityAS = I we haveh ( r )--la(5.69)of (5.67). First, observe that with=A S a+A n-a=An.(5.70)Thus,R,,=AE { n n H } A H=AR,,A~=[SHR;AS]-'SHR;ARn,R;AS[SHR;AS]-'=[SHR;AS]-'.(5.71)In order to see whether A according to (5.67) is optimal, an estimation(5.72)with(5.73)will be considered. The urlbiasedness constraint requires thatAs==.(5.74)Because of A S = I this meansDS=O(null matrix).(5.75)For the covariance matrix of the error E(r) = C(r)- a we obtain=AR,,A-H=[ A D]Rnn[A DIH+++ ARnnDH+ DRnnAH+ DRnnDH.= ARnnAH(5.76)116Chapter 5.
Tkansforms and Filters for Stochastic ProcessesWith(AR,nDH)H= DRn,AH= DRnnR$S[SHRiAS]-'=DSISHR;AS]-lv0(5.77)= o(5.76) reduces toR22 = ARn,AH + DRnnDH.(5.78)We see that Rc2 is the sum of two non-negative definite expressions so thatminimal main diagonal elementsof Rgc are yielded for D = 0 and thus for Aaccording to (5.67). 0In the case of a white noise process n , (5.68) reduces toS(r) = [s~s]-~s~~.(5.79)Otherwise the weighting with G = R;; can be interpreted as animplicitwhitening of the noise. This can beseen by using the Cholesky decompositionR,, = LLH and and by rewriting the model aswhereF= L-'r,F=Sa+fi,(5.80)3 = L - ~ s ,fi = L-ln.(5.81)The transformedprocess n is a white noise process.
The equivalent estimatorthen is-H- HU(?) = [ S ~ 1 - l r ~.(5.82)I5.5.3MinimumMeanSquareErrorEstimationThe advantage of the linear estimators considered in the previous sectionis their unbiasedness. If we dispensewith this property,estimateswithsmaller mean square error maybe found. Wewill start the discussion onthe assumptionsE { r } = 0,E { a } = 0.(5.83)Again, the linear estimator is described by a matrix A:S(r) = A r .(5.84)1175.5. Linear EstimationHere, r is somehow dependent on a , but the inner relationship between rand a need not be known however.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
