Probability Models (Vaseghi - Advanced Digital Signal Processing and Noise Reduction), страница 5

A non-Gaussian pdf may be approximated by a weighted sum (i.e.a mixture) of a number of Gaussian densities of appropriate mean vectorsand covariance matrices. An M-mixture Gaussian density is defined asMf X ( x) = ∑ Pi N i ( x, µ x i , Σ xx i )(3.92)i =1f (x)µ1µ2µ3µ4µ5Figure 3.9 A mixture Gaussian pdf.x72Probability Modelswhere N i (x, µ x i , Σ xxi ) is a multivariate Gaussian density with mean vectorµ x i and covariance matrix Σ xx i , and Pi are the mixing coefficients.

Theparameter Pi is the prior probability of the ith mixture component, and isgiven byNPi = M i(3.93)∑ Njj=1where Ni is the number of observations associated with the mixture i. Figure3.9 shows a non-Gaussian pdf modelled as a mixture of five Gaussian pdfs.Algorithms developed for Gaussian processes can be extended to mixtureGaussian densities.3.5.4 A Binary-State Gaussian ProcessConsider a random process x(m) with two statistical states: such that in thestate s0 the process has a Gaussian pdf with mean µ x ,0 and variance σ 2x,0 ,and in the state s1 the process is also Gaussian with mean µ x ,1 and varianceσ 2x,1 (Figure 3.10). The state-dependent pdf of x(m) can be expressed asf X S ( x ( m) s i ) =1x(m) − µ x,i ] 2  ,exp −[2π σ x,i 2σ x2,i1i=0, 1xf X ,S (x, s )s0s1sFigure 3.10 Illustration of a binary-state Gaussian process(3.94)Some Useful Classes of Random Processes73The joint probability distribution of the binary-valued state si and thecontinuous-valued signal x(m) can be expressed asf X ,S (x(m), si ) = f X S (x(m) si ) PS (si )11[x(m) − µ x,i ] 2  PS (si )exp −2π σ x,i 2σ x2,i=(3.95)where PS ( si ) is the state probability.

For a multistate process we have thefollowing probabilistic relations between the joint and marginalprobabilities:∑ f X ,S (x(m), si ) = f X (x(m))(3.96)∫ f X ,S (x(m), si ) dx = PS (si )(3.97)∑ ∫ f X ,S (x(m), si ) dx = 1(3.98)SXandS XNote that in a multistate model, the statistical parameters of the processswitch between a number of different states, whereas in a single-statemixture pdf, a weighted combination of a number of pdfs models theprocess.

In Chapter 5 on hidden Markov models we consider multistatemodels with a mixture pdf per state.3.5.5 Poisson ProcessThe Poisson process is a continuous-time, integer-valued counting process,used for modelling the occurrence of a random event in various timeintervals. An important area of application of the Poisson process is inqueuing theory for the analysis and modelling of the distributions of demandon a service facility such as a telephone network, a shared computer system,a financial service, a petrol station, etc.

Other applications of the Poissondistribution include the counting of the number of particles emitted inphysics, the number of times that a component may fail in a system, andmodelling of radar clutter, shot noise and impulsive noise. Consider anevent-counting process X(t), in which the probability of occurrence of the74Probability Modelsevent is governed by a rate function λ(t), such that the probability that anevent occurs in a small time interval ∆t isProb(1 occurrencein the interval (t , t + ûW ) ) = λ (t ) ûW(3.99)Assuming that in the small interval ∆t, no more than one occurrence of theevent is possible, the probability of no occurrence of the event in a timeinterval of ∆t is given byProb(0 occurrence in the interval(t , t + ûW ))=1 − λ (t ) ûW(3.100)when the parameter λ(t) is independent of time, λ(t)=λ, and the process iscalled a homogeneous Poisson process. Now, for a homogeneous Poissonprocess, consider the probability of k occurrences of an event in a timeinterval of t+∆t, denoted by P(k, (0, t+∆t)):P(k ,(0, t + ûW ) ) = P(k , (0, t ) )P(0, (t , t + ûW ) ) + P(k − 1,(0, t ) )P(1, (t , t + ûW ) )= P(k , (0, t ) )(1 − λûW ) + P(k − 1,(0, t ) )λûW(3.101)Rearranging Equation (3.101), and letting ∆t tend to zero, we obtain thefollowing linear differential equation:dP (k , t )= − λP ( k , t ) + λP ( k − 1, t )dt(3.102)where P(k,t)=P(k,(0, t)).

The solution of this differential equation is givenbyP(k , t ) = λe−λtt∫ P(k − 1,τ )eλτdτ(3.103)0Equation (3.103) can be solved recursively: starting with P(0,t)=e–λt andP(1,t)=λt e-λt, we obtain the Poisson densityP(k , t ) =(λt ) k −λtek!(3.104)75Some Useful Classes of Random ProcessesFrom Equation (3.104), it is easy to show that for a homogenous Poissonprocess, the probability of k occurrences of an event in a time interval (t1, t2)is given by[λ (t 2 − t1 )] k −λ (t2 −t1 )(3.105)eP [ k , (t1 , t 2 )] =k!A Poisson counting process X(t) is incremented by one every time the eventoccurs. From Equation (3.104), the mean and variance of a Poisson countingprocess X(t) areE [ X (t )] = λtrXX (t1 ,t 2 ) = E [X (t1 ) X (t 2 )] = λ 2 t1 t 2 +λ min(t1 ,t 2 )[]Var [ X (t )] = E X 2 (t ) −E 2 [X (t )]= λt(3.106)(3.107)(3.108)Note that the variance of a Poisson process is equal to its mean value.3.5.6 Shot NoiseShot noise happens when there is randomness in a directional flow ofparticles: as in the flow of electrons from the cathode to the anode of acathode ray tube, the flow of photons in a laser beam, the flow andrecombination of electrons and holes in semiconductors, and the flow ofphotoelectrons emitted in photodiodes.

Shot noise has the form of a randompulse sequence. The pulse sequence can be modelled as the response of alinear filter excited by a Poisson-distributed binary impulse input sequence(Figure 3.11). Consider a Poisson-distributed binary-valued impulse processx(t). Divide the time axis into uniform short intervals of ∆t such that onlyone occurrence of an impulse is possible within each time interval. Letx(m∆t) be “1” if an impulse is present in the interval m∆t to (m+1)∆t, and“0” otherwise. For x(m∆t), we haveE [x(mût )] = 1× P(x(mût ) = 1) + 0 × P(x(mût ) = 0) = λûWand(3.109)76Probability Modelsh(m)Figure 3.11 Shot noise is modelled as the output of a filter excited with a process.1 × P( x( mût ) = 1)= λûW ,E [x(mût ) x(nût )] = m=n21 × P( x( mût ) = 1)) × P (x ( nût ) = 1)= (λûW ) , m ≠ n(3.110)A shot noise process y(m) is defined as the output of a linear system with animpulse response h(t), excited by a Poisson-distributed binary impulse inputx(t):y (t ) ==∞∫ x(τ )h(t − τ )dτ−∞∞(3.111)∑ x(mût )h(t − mût )k = −∞where the binary signal x(m∆t) can assume a value of 0 or 1.

In Equation(3.111) it is assumed that the impulses happen at the beginning of eachinterval. This assumption becomes more valid as ∆t becomes smaller. Theexpectation of y(t) is obtained asE [ y ( t ) ]==∞∑ E [x ( mût ) ]h (t − mût )k = −∞∞(3.112)∑ λût h (t − mût )k =−∞andryy (t1 , t 2 ) = E [ y ( t1 ) y ( t 2 ) ]=∞∞∑ ∑ E [x ( mût ) x ( nût ) ]h (t1 − nût )h (t 2 − mût )m = −∞ n = −∞(3.113)77Some Useful Classes of Random ProcessesUsing Equation (3.110), the autocorrelation of y(t) can be obtained asr yy (t1 , t 2 ) =∞∞∞∑ (λ ût )h(t1 − mût )h(t 2 − mût ) + ∑ ∑ (λ ût ) 2 h(t1 − mût )h(t 2 − nût )m = −∞m = −∞ n = −∞n≠m(3.114)3.5.7 Poisson–Gaussian Model for Clutters and Impulsive NoiseAn impulsive noise process consists of a sequence of short-duration pulsesof random amplitude and random time of occurrence whose shape andduration depends on the characteristics of the channel through which theimpulse propagates.

A Poisson process can be used to model the randomtime of occurrence of impulsive noise, and a Gaussian process can be usedto model the random amplitude of the impulses. Finally, the finite durationcharacter of real impulsive noise may be modelled by the impulse responseof linear filter. The Poisson–Gaussian impulsive noise model is given byx ( m )=∞∑ Ak h(m − τ k )(3.115)k =−∞where h(m) is the response of a linear filter that models the shape ofimpulsive noise, Ak is a zero-mean Gaussian process of variance σ 2 and τk isa Poisson process.

The output of a filter excited by a Poisson-distributedsequence of Gaussian amplitude impulses can also be used to model cluttersin radar. Clutters are due to reflection of radar pulses from a multitude ofbackground surfaces and objects other than the radar target.3.5.8 Markov ProcessesA first-order discrete-time Markov process is defined as one in which thestate of the process at time m depends only on its state at time m–1 and isindependent of the process history before m–1. In probabilistic terms, a firstorder Markov process can be defined asf X (x(m) = x m x(m − 1) = x m−1 ,, x(m − N ) = x m− N )= f X (x(m) = x m x(m − 1) = x m−1 )(3.116)78Probability Modelsx(m)e(m)aFigure 3.12 A first order autoregressive (Markov) process.The marginal density of a Markov process at time m can be obtained byintegrating the conditional density over all values of x(m–1):f X ( x ( m) = x m ) =∞∫ f X (x(m) = xm x(m − 1) = xm−1 )) f X (x(m − 1) = xm−1 ) dxm−1−∞(3.117)A process in which the present state of the system depends on the past nstates may be described in terms of n first-order Markov processes and isknown as an nth order Markov process.

The term “Markov process” usuallyrefers to a first order process.Example 3.10 A simple example of a Markov process is a first-order autoregressive process (Figure 3.12) defined asx(m) = a x(m − 1) + e(m)(3.118)In Equation (3.118), x(m) depends on the previous value x(m–1) and theinput e(m). The conditional pdf of x(m) given the previous sample value canbe expressed asf X (x( m) x ( m − 1), ..., x( m − N ) ) = f X (x ( m) x ( m − 1) )= f E (e( m) = x ( m) −ax ( m − 1) )(3.119)where fE(e(m)) is the pdf of the input signal e(m). Assuming that input e(m)is a zero-mean Gaussian process with variance σ e2 , we have79Some Useful Classes of Random Processesa 00State 0a 03a 01a 30a33a 02a 13a 10a 31State 3State 1a 20a 32aa 11a 12a 2123State 2a 22Figure 3.13 A Markov chain model of a four-state discrete-time Markov process.f X (x(m) x( m − 1)..., x(m − N ) ) = f X (x( m) x( m − 1) )= f E (x(m) −ax(m − 1) )= 1(x(m) −ax(m − 1) )2 exp −22π σ e 2σ e1(3.120)When the input to a Markov model is a Gaussian process the output isknown as a Gauss–Markov process.3.5.9 Markov Chain ProcessesA discrete-time Markov process x(m) with N allowable states may bemodelled by a Markov chain of N states (Figure 3.13).

Each state can beassociated with one of the N values that x(m) may assume. In a Markovchain, the Markovian property is modelled by a set of state transitionprobabilities defined as80Probability Modelsaij ( m − 1,m) = Prob(x( m) = j x ( m − 1) = i )(3.121)where aij(m,m–1) is the probability that at time m–1 the process is in thestate i and then at time m it moves to state j.