Transforms and Filters for Stochastic Processes (779449), страница 3
Текст из файла (страница 3)
The matrix A which yields minimal maindiagonal elements of the correlation matrix of the estimation error e = a - Uis called the minimum mean square error (MMSE) estimator.In order to find the optimal A , observe thatR,,{=E [ U - U ] [U - U ]=E{aaH}-E{UaH}-E{aUH}+E{UUH}.(5.85)Substituting (5.84) into (5.85) yieldsR,, = R,, - AR,, - R,,AH + AR,,AH(5.86)withR,,=E {aaH},R,,=R: = E { r a H } ,R,,= E{mH}.(5.87)Assuming the existence of R;:, (5.86) can be extended byR,, R;: R,,- R,, R;:R,,and be re-written asR,, = [ A - R,,RF:] R,, [AH- RF:Ra,]-RTaRF:Ra,+ Raa.(5.88)Clearly, R,, has positive diagonal elements.
Since only the first term on theright-hand side of (5.88) is dependenton A , we have a minimum of thediagonal elements of R,, forA = R,, R;:.(5.89)The correlation matrix of the estimation error is then given byRe,= R a a - RTaR;:RaT*(5.90)Orthogonality Principle. In Chapter 3 we saw that approximations D ofsignals z are obtained with minimal error if the error D - z is orthogonal toD. A similar relationship holds between parameter vectors a and their MMSEestimates.
With A according to (5.89) we haveR,, = A R,,,i.e.E { a r H }= A E { r r H }.(5.91)118Chapter 5. TkansformsFiltersandStochasticfor ProcessesThis means that the following orthogonality relations hold:E [S - a] SH{}- R&,==[AR,, - R,,] A"=0.(5.92)With A r = S the right part of (5.91) can also be written asE { a r H }= E {Sr"} ,(5.93)E { [ S - a] r H }= 0.(5.94)which yieldsThe relationship expressed in (5.94) is referred to as the orthogonalityprinciple.
The orthogonality principle states that we get an MMSE estimateif the error S(r)- a is uncorrelated to all components of the input vector rused for computing S ( r ) .Singular Correlation Matrix. Thereare cases where the correlationmatrix R,, becomes singular and the linear estimator cannot be written asA = R,, R;:.(5.95)A more general solution, which involves the replacement of the inverse by thepseudoinverse, isA = R,, R:,.(5.96)In order to show the optimality of (5.96), the estimatorA=A+D(5.97)with A according to (5.96) and an arbitrary matrix D is considered. Usingthe properties of the pseudoinverse, we derive from (5.97) and (5.86):R,,- H- H= R,, - AR,, - R,,A= R,, - R,,R:,R,,+ AR,,A+DR:,D~.(5.98)Since R:, is at least positive semidefinite, we get a minimum of the diagonalelements of R,, for D = 0, and (5.96) constitutes oneof the optimalsolutions.Additive Uncorrelated Noise.
So far, nothing has beensaid about possibledependencies between a and the noise contained in r . Assuming thatr=Sa+n,(5.99)1195.5. Linear Estimationwhere n is an additive, uncorrelated noise process, we have(5.100)and A according to (5.89) becomes+ R,,]-1.(5.101)+ SHRLAS]-1 SHRLA.(5.102)A = R,,SH[SR,,SHAlternatively, A can be written asA = [R;:This isverifiedbyequating (5.101) and (5.102), and by multiplying theobtained expression with [R;: +SHR;AS]from the left and with [SR,,SH+R,,] from the right, respectively:[R;:+ SHRLAS]R,,SH = SHRLA[SR,,SH+ R , , ] .The equality of both sides is easily seen.
The matrices tobe inverted in (5.102),except R,,, typically have a much smaller dimension than those in (5.101). Ifthe noise is white, R;; can be immediatelystated, and(5.102) is advantageousin terms of computational cost.For R,, we get from (5.89), (5.90), (5.100) and (5.102):Ree= Raa - ARar= R,, - [R;;Multiplying (5.103) with [R;;+ S H R ; ; S ] - ~SHR;;SR,,.(5.103)+ SHR;;S] from the left yields= I,(5.104)so that the following expression is finally obtained:+Re, = [R,-,' S H R i A S ] - l .(5.105)Equivalent Estimation Problems.
We partition A and a into(5.106)120Chapter 5. TkansformsFiltersandStochasticfor Processessuch that(5.107)If we assume that the processes a l , a2 and n are independent of one another,the covariance matrix R,, and its inverse R;: have the formand A according to (5.102) can be written aswhere S = [SI,5'21. Applying the matrix equation€3€-l+E-132)-1BE-14 - 1 3 2 ) - 12)-l2)(5.110)= 3c - &?€-l3yieldswithRn1n1+ SzRazazSf,Rnn + S1Ra1a1Sf.= Rnn(5.113)=(5.114)The inverses R;:nl and R;inz can be written asR;:nz=[R;:=[R;: -- RiiS2(SfRiAS2+ R;:az)-1 SfRiA] ,(5.115)+(SyR;AS1 R;:al)- 1 SyR;:] .(5.116)Equations (5.111) and (5.112) describe estimations of a1 and a2 in themodelsr = S l a l + nl,(5.117)1215.5.
Linear Estimationr = S2a2+n2(5.118)with(5.119)Thus, each parameter to be estimated can be understood as noise in theestimation of the remaining parameters.An exception is given if SFR$S2 =0 , which means that S1 and S2 are orthogonal to each other with respect tothe weighting matrix R;:. Then we getandRe,,, = [SBRLASI+and we observe that the second signal component Sza2 has no influence onthe estimate.Nonzero-Mean Processes. One could imagine that the precision of linearestimations with respect to nonzero-mean processes r and a can be increasedcompared to thesolutions above if an additional term taking careof the meanvalues of the processes is considered.
In order to describe this more generalcase, let us denote the mean of the parameters as=E{a}.(5.120)The estimate is now written asiL=Ar+c%+c,(5.121)where c is yet unknown. Using the shorthandb = a--,b = h--,M=[c,A](5.122)(5.121) can be rewritten as:b =Mx.(5.123)122Chapter 5. TkansformsFiltersandStochasticfor ProcessesThe relationship between b and X is linear as usual, so that the optimal Mcan be given according to (5.89):M = R,bR$.(5.124)Now let us express R , b and R;: through correlation matrices of the processesa and T . From (5.122) and E { b } = 0 we derive(5.125)withR,b= E{[U-si]= E{[a-si]T"}(5.126)[T-e]"},whereF=E{r}.(5.127)R,, writes1= [FFH(5.128)RT,]'Using (5.110) we obtainR;: =1+ e"[R,, - ee"] - le- [R,, - ee"1-l e-e" [R,, - FP"]-l[R,, - FP"]-l(5.129)From (5.122) - (5.129) and[R,, - Fe"] = E { [T - e] [T - e]"}(5.130)we finally concludeU = E { [a- s i ][T- e]"}E { [T - e] [T - e]"}-l[T-e] + a.(5.131)Equation (5.131) can be interpreted asfollows: the nonzero-mean processesa and r are first modified so as to become zero-mean processes a - si andr - e.
For the zero-mean processes the estimation problem can be solved asusual. Subsequently the mean value si is added in order to obtain the finalestimate U.Unbiasedness for Random Parameter Vectors. So far the parametervector to be estimated was assumed to be non-random.
If we consider a tobe a random process, various other weaker definitions of unbiasedness arepossible.1235.5. Linear EstimationThe straightforward requirementis meaningless, because itis satisfied for any A as faras r and a are zero-mean.A useful definition of unbiasedness in the case of random parameters is toconsider one of the parameters contained in a (e.g. a k ) as non-random and toregard all other parameters a l , . . .
, ak-1, ak+l etc. as random variables:In order to obtain an estimator which is unbiased in the sense of (5.133), theequivalences discussed above may be applied. Starting with the modelr=Sa+n=skak(5.134)+n ,in which n contains the additive noise n and the signal component producedby all random parameters a j , j # k, we can write the unbiased estimate as6, = h,H r(5.135)A = [hl,h z , .. .IH(5.137)Then,is an estimator which is unbiased in the sense of (5.133).The Relationship between MMSE Estimation and the BLUE. IfR,, = E { a a H }is unknown, R;: = 0 is substituted into (5.102), and weobtain the BLUE (cf.
(5.67)):A = [SHRiAS]-lSHnnR P 1(5.138)In the previous discussion it became obvious that it is possible to obtainunbiased estimates of some of the parameters and to estimate the otherswithminimummeansquareerror.Thisresult is of special interest if nounbiased estimator can be statedfor all parameters because of a singularmatrix SHR;AS.124Chapter 5. Tkansforms and FiltersStochasticfor Processes5.6LinearOptimal5.6.1Wiener FiltersFiltersWe consider the problem depictedin Figure 5.3. By linear filtering of the noisysignal r ( n ) = z(n) w ( n ) we wish to make y(n) = r ( n ) * h(n) as similar aspossible to a desired signal d ( n ) .
The quality criterion used for designing theoptimal causal linear filter h(n)is+The solution to this optimization problem can easily be stated by applyingthe orthogonalityprinciple. Assuming a causal FIR filter h(n) of length p , wehavecP-1y(n) =h(i)r ( n - i).(5.140)i=OThus, according to (5.94), the following orthogonality condition mustbesatisfied by the optimal filter:For stationary processes r ( n ) and d ( n ) this yields the discrete form of theso-called Wiener-Hopf equation:cP-1j = 0, 1 , . .
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
