Probability Models (Vaseghi - Advanced Digital Signal Processing and Noise Reduction), страница 3

The loudness and thefrequency composition of speech changes over time, and sometimes thechange can be quite abrupt. Time-varying processes may be modelled by acombination of stationary random models as illustrated in Figure 3.5. InFigure 3.5(a) a non-stationary process is modelled as the output of a timevarying system whose parameters are controlled by a stationary process. InFigure 3.5(b) a time-varying process is modelled by a chain of timeinvariant states, with each state having a different set of statistics or57Expected Values of a Random ProcessState excitationS1(Stationary)State modelSignalexcitationNoiseTime-varyingsignal modelS3S2(a)(b)Figure 3.5 Two models for non-stationary processes: (a) a stationary processdrives the parameters of a continuously time-varying model; (b) a finite-statemodel with each state having a different set of statistics.probability distributions.

Finite state statistical models for time-varyingprocesses are discussed in detail in Chapter 5.3.4 Expected Values of a Random ProcessExpected values of a process play a central role in the modelling andprocessing of signals. Furthermore, the probability models of a randomprocess are usually expressed as functions of the expected values.

Forexample, a Gaussian pdf is defined as an exponential function of the meanand the covariance of the process, and a Poisson pdf is defined in terms ofthe mean of the process. In signal processing applications, we often have asuitable statistical model of the process, e.g. a Gaussian pdf, and to completethe model we need the values of the expected parameters. Furthermore inmany signal processing algorithms, such as spectral subtraction for noisereduction described in Chapter 11, or linear prediction described in Chapter8, what we essentially need is an estimate of the mean or the correlationfunction of the process.

The expected value of a function, h(X(m1), X(m2), ...,X(mM)), of a random process X is defined asE [h( X (m1 ), , X (m M ))] =∫ ∫ h( x1 ,, x M ) f X (m ) X (m∞∞−∞−∞1M, x) ( x1 ,M) dx1 dxM(3.34)The most important, and widely used, expected values are the mean value,the correlation, the covariance, and the power spectrum.58Probability Models3.4.1 The Mean ValueThe mean value of a process plays an important part in signal processingand parameter estimation from noisy observations. For example, in Chapter3 it is shown that the optimal linear estimate of a signal from a noisyobservation, is an interpolation between the mean value and the observedvalue of the noisy signal.

The mean value of a random vector [X(m1), ...,X(mM)] is its average value across the ensemble of the process defined asE[ X (m1 ), , X (m M )] =∞∞∫ ∫ ( x ,, x1−∞−∞M, x) f X ( m1 ) X ( mM ) ( x1 ,M) dx1 dxM(3.35)3.4.2 AutocorrelationThe correlation function and its Fourier transform, the power spectraldensity, are used in modelling and identification of patterns and structures ina signal process. Correlators play a central role in signal processing andtelecommunication systems, including predictive coders, equalisers, digitaldecoders, delay estimators, classifiers and signal restoration systems. Theautocorrelation function of a random process X(m), denoted by rxx(m1,m2), isdefined asrxx (m1 ,m2 ) = E[ x(m1 ) x(m2 )]=∞ ∞∫ ∫ x(m1 ) x(m2 ) f X (m ), X (m ) (x(m1 ), x(m2 )) dx(m1 ) dx(m2 )− ∞ −∞11(3.36)The autocorrelation function rxx(m1,m2) is a measure of the similarity, or themutual relation, of the outcomes of the process X at time instants m1 and m2.If the outcome of a random process at time m1 bears no relation to that attime m2 then X(m1) and X(m2) are said to be independent or uncorrelatedand rxx(m1,m2)=0.

For a wide-sense stationary process, the autocorrelationfunction is time-invariant and depends on the time difference m= m1–m2:rxx (m1 + τ , m2 + τ ) = rxx (m1 , m2 ) = rxx (m1 − m2 ) = rxx (m)(3.37)Expected Values of a Random Process59The autocorrelation function of a real-valued wide-sense stationary processis a symmetric function with the following properties:rxx(–m) = rxx (m)rxx (m) ≤ rxx (0)(3.38)(3.39)Note that for a zero-mean signal, rxx(0) is the signal power.Example 3.5 Autocorrelation of the output of a linear time-invariant (LTI)system.

Let x(m), y(m) and h(m) denote the input, the output and the impulseresponse of a LTI system respectively. The input–output relation is given byy (m)=∑ hk x(m − k )(3.40)kThe autocorrelation function of the output signal y(m) can be related to theautocorrelation of the input signal x(m) byryy (k ) = E [ y (m) y (m + k )]=∑∑ hi h j E[ x(m − i ) x(m + k − j )]i(3.41)j=∑∑ hi h j rxx (k + i − j )ijWhen the input x(m) is an uncorrelated random signal with a unit variance,Equation (3.41) becomesryy (k ) =∑ hi hk +i(3.42)i3.4.3 AutocovarianceThe autocovariance function cxx(m1,m2) of a random process X(m) is measureof the scatter, or the dispersion, of the random process about the mean value,and is defined asc xx (m1 , m2 ) = E [(x(m1 ) − µ x (m1 ) )(x(m2 ) − µ x (m2 ) )]= rxx (m1 , m2 ) − µ x (m1 ) µ x (m2 )(3.43)60Probability Modelswhere µx(m) is the mean of X(m). Note that for a zero-mean process theautocorrelation and the autocovariance functions are identical.

Note also thatcxx(m1,m1) is the variance of the process. For a stationary process theautocovariance function of Equation (3.43) becomesc xx (m1 , m2 ) = c xx (m1 − m2 ) = rxx (m1 − m2 ) − µ x2(3.44)3.4.4 Power Spectral DensityThe power spectral density (PSD) function, also called the power spectrum,of a random process gives the spectrum of the distribution of the poweramong the individual frequency contents of the process. The powerspectrum of a wide sense stationary process X(m) is defined, by the Wiener–Khinchin theorem in Chapter 9, as the Fourier transform of theautocorrelation function:PXX ( f ) = E [ X ( f ) X * ( f )]=∞∑ r xx ( k ) e − j 2πfm(3.45)m= − ∞where rxx(m) and PXX(f) are the autocorrelation and power spectrum of x(m)respectively, and f is the frequency variable.

For a real-valued stationaryprocess, the autocorrelation is symmetric, and the power spectrum may bewritten as∞PXX ( f ) = r xx (0) + ∑ 2rxx (m)cos(2π fm)(3.46)m=1The power spectral density is a real-valued non-negative function, expressedin units of watts per hertz. From Equation (3.45), the autocorrelationsequence of a random process may be obtained as the inverse Fouriertransform of the power spectrum as1/ 2r xx (m) =∫ PXX ( f ) e j2 π fm df(3.47)−1/ 2Note that the autocorrelation and the power spectrum represent the secondorder statistics of a process in the time and frequency domains respectively.61Expected Values of a Random ProcessPXX(f)rxx(m)mfFigure 3.6 Autocorrelation and power spectrum of white noise.Example 3.6 Power spectrum and autocorrelation of white noise(Figure3.6).

A noise process with uncorrelated independent samples iscalled a white noise process. The autocorrelation of a stationary white noisen(m) is defined as: Noisepower k = 0rnn (k ) = E [n(m)n(m + k )] = (3.48)k≠00Equation (3.48) is a mathematical statement of the definition of anuncorrelated white noise process. The equivalent description in thefrequency domain is derived by taking the Fourier transform of rnn(k):PNN ( f ) =∞∑ rnn (k )e − j 2πfk = rnn (0) =noise power(3.49)k = −∞The power spectrum of a stationary white noise process is spread equallyacross all time instances and across all frequency bins.

White noise is one ofthe most difficult types of noise to remove, because it does not have alocalised structure either in the time domain or in the frequency domain.Example 3.7 Autocorrelation and power spectrum of impulsive noise.Impulsive noise is a random, binary-state (“on/off”) sequence of impulses ofrandom amplitudes and random time of occurrence. In Chapter 12, a randomimpulsive noise sequence ni(m) is modelled as an amplitude-modulatedrandom binary sequence asni (m) = n(m) b(m)(3.50)where b(m) is a binary-state random sequence that indicates the presence orthe absence of an impulse, and n(m) is a random noise process.

Assuming62Probability Modelsthat impulsive noise is an uncorrelated process, the autocorrelation ofimpulsive noise can be defined as a binary-state process asrnn (k, m) = E [ni (m)ni (m + k )] = σ n2 δ (k )b(m)(3.51)where σ n2 is the noise variance. Note that in Equation (3.51), theautocorrelation is expressed as a binary-state function that depends on theon/off state of impulsive noise at time m. The power spectrum of animpulsive noise sequence is obtained by taking the Fourier transform of theautocorrelation function:PNN ( f , m) = σ n2 b(m)(3.52)3.4.5 Joint Statistical Averages of Two Random ProcessesIn many signal processing problems, for example in processing the outputsof an array of sensors, we deal with more than one random process.

Jointstatistics and joint distributions are used to describe the statistical interrelationship between two or more random processes. For two discrete-timerandom processes x(m) and y(m), the joint pdf is denoted by, xf X ( m1 ) X ( mM ),Y ( n1 )Y ( nN ) ( x1 ,M, y1 , , y N )(3.53)When two random processes, X(m) and Y(m) are uncorrelated, the joint pdfcan be expressed as product of the pdfs of each process as, xf X ( m1 ) X ( mM ),Y ( n1 )Y ( nN ) ( x1 ,, x= f X ( m1 ) X ( mM ) ( x1 ,MM, y1 , , y N )) f Y ( n1 )Y ( nN ) ( y1 , , y N )(3.54)3.4.6 Cross-Correlation and Cross-CovarianceThe cross-correlation of two random process x(m) and y(m) is defined asrxy (m1 ,m2 ) = E[ x(m1 ) y (m2 )]=∞ ∞∫ ∫ x(m1 ) y (m2 ) f X (m )Y (m ) (x(m1 ), y(m2 ) ) dx(m1 ) dy(m2 )−∞ −∞12(3.55)63Expected Values of a Random ProcessFor wide-sense stationary processes, the cross-correlation functionrxy(m1,m2) depends only on the time difference m=m1–m2:rxy ( m1 + τ , m2 + τ ) = rxy ( m1 , m2 ) = rxy ( m1 − m2 ) = rxy ( m)(3.56)The cross-covariance function is defined asc xy ( m1 , m 2 ) = E [( x ( m1 ) − µ x ( m1 ) )(y ( m2 ) − µ y ( m 2 ) )]= rxy ( m1 , m 2 ) − µ x ( m1 ) µ y ( m2 )(3.57)Note that for zero-mean processes, the cross-correlation and the crosscovariance functions are identical.