Probability Models (Vaseghi - Advanced Digital Signal Processing and Noise Reduction), страница 2

This space can be partitionedin different ways; for example, the two subspaces shown in Figure 3.3 areassociated with the pair of numbers that add up to less than or equal to 8,and to greater than 8. In this example, assuming the dice are not loaded, allnumbers are equally likely, and the probability of each event is proportionalto the total number of outcomes in the space of the event.3.2.1 Probability Mass Function (pmf)For a discrete random variable X that can only assume discrete values from afinite set of N numbers {x1, x2, ..., xN}, each outcome xi may be consideredas an event and assigned a probability of occurrence.

The probability that a50Probability Modelsdiscrete-valued random variable X takes on a value of xi, P(X= xi), is calledthe probability mass function (pmf). For two such random variables X and Y,the probability of an outcome in which X takes on a value of xi and Y takeson a value of yj, P(X=xi, Y=yj), is called the joint probability mass function.The joint pmf can be described in terms of the conditional and the marginalprobability mass functions asPX ,Y ( xi , y j ) = PY | X ( y j | xi ) PX ( xi )= PX |Y ( xi | y j ) PY ( y j )(3.6)where PY | X (y j | xi ) is the probability of the random variable Y taking on avalue of yj conditioned on X having taken a value of xi, and the so-calledmarginal pmf of X is obtained asMPX ( xi ) = ∑ PX ,Y ( xi , y j )j =1(3.7)M= ∑ PX |Y ( xi | y j ) PY ( y j )j =1where M is the number of values, or outcomes, in the space of the discreterandom variable Y. From Equations (3.6) and (3.7), we have Bayes’ rule forthe conditional probability mass function, given byPX |Y ( xi | y j ) ==1PY | X ( y j | xi ) PX ( xi )PY ( y j )PY | X ( y j | xi ) PX ( xi )(3.8)M∑ PY | X ( y j | xi ) PX ( xi )i =13.2.2 Probability Density Function (pdf)Now consider a continuous-valued random variable.

A continuous-valuedvariable can assume an infinite number of values, and hence, the probabilitythat it takes on a given value vanishes to zero. For a continuous-valued51Probabilistic Modelsrandom variable X the cumulative distribution function (cdf) is defined asthe probability that the outcome is less than x as:FX (x) = Prob( X ≤ x )(3.9)where Prob(· ) denotes probability. The probability that a random variable Xtakes on a value within a band of ∆ centred on x can be expressed as1û1Prob ( x − û / 2 ≤ X ≤ x + û / 2 ) = [ Prob ( X ≤ x + û / 2 ) − Prob ( X ≤ x − û / 2 )]û=1û[ F X ( x + û / 2 ) − F X ( x − û / 2 )]( 3.10 )As ∆ tends to zero we obtain the probability density function (pdf) as1[ FX ( x + û / 2) − FX ( x − û / 2)]û→0 û∂ FX ( x )=∂xf X ( x) = lim(3.11)Since FX (x) increases with x, the pdf of x, which is the rate of change ofFX(x) with x, is a non-negative-valued function; i.e.

f X ( x ) ≥ 0. The integralof the pdf of a random variable X in the range ± ∞ is unity:∞∫ f X ( x)dx =1(3.12)−∞The conditional and marginal probability functions and the Bayes rule, ofEquations (3.6)–(3.8), also apply to probability density functions ofcontinuous-valued variables.Now, the probability models for random variables can also be applied torandom processes. For a continuous-valued random process X(m), thesimplest probabilistic model is the univariate pdf fX(m)(x), which is theprobability density function that a sample from the random process X(m)takes on a value of x.

A bivariate pdf fX(m)X(m+n)(x1, x2) describes theprobability that the samples of the process at time instants m and m+n takeon the values x1, and x2 respectively. In general, an M-variate pdf52Probability Modelsf(x , x , , x M ) describes the pdf of M samples of a X(m1 ) X( m2 )X (m M ) 1 2random process taking specific values at specific time instants. For an Mvariate pdf, we can write∞∫ f X ( m ) X ( m−∞M1) ( x1 , , x M) dx M = f X ( m1 ) X (mM −1 ) ( x1 , , x M −1 )(3.13)and the sum of the pdfs of all possible realisations of a random process isunity, i.e.∞∫−∞∞∫ f X ( m ) X ( m−∞1M) ( x1 , , x M) dx1 dx M =1(3.14)The probability of a realisation of a random process at a specified timeinstant may be conditioned on the value of the process at some other timeinstant, and expressed in the form of a conditional probability densityfunction asf X ( m ) | X ( n ) (x m x n )=f X ( n )| X ( m ) (x n x m ) f X ( m ) ( x m )(3.15)f X ( n) ( x n )If the outcome of a random process at any time is independent of itsoutcomes at other time instants, then the random process is uncorrelated.For an uncorrelated process a multivariate pdf can be written in terms of theproducts of univariate pdfs asMf [ X ( m1 ) X ( mM ) X ( n1 ) X ( nN ) ] (x m1 , , x mM x n1 , , x nN )= ∏ f X ( mi ) ( x mi )i =1(3.16)Discrete-valued stochastic processes can only assume values from a finiteset of allowable numbers [x1, x2, ..., xn].

An example is the output of abinary message coder that generates a sequence of 1s and 0s. Discrete-time,discrete-valued, stochastic processes are characterised by multivariateprobability mass functions (pmf) denoted as , x(mP[x ( m1 ) x ( mM ) ] (x ( m1 )= xi ,M)= x k )(3.17)53Stationary and Non-Stationary Random ProcessesThe probability that a discrete random process X(m) takes on a value of xmat time instant m can be conditioned on the process taking on a value xn atsome other time instant n, and expressed in the form of a conditional pmf asPX ( m )| X ( n ) (x m x n ) =PX ( n ) | X ( m ) (x n x m )PX ( m ) ( x m )PX ( n ) ( x n )(3.18)and for a statistically independent process we haveMP[ X ( m1 ) X ( mM )| X ( n1 ) X ( nN )] (x m1 , , x mM x n1 , , x nN ) = ∏ PX ( mi ) ( X ( mi ) = x mi )i =1(3.19)3.3 Stationary and Non-Stationary Random ProcessesAlthough the amplitude of a signal x(m) fluctuates with time m, thecharacteristics of the process that generates the signal may be time-invariant(stationary) or time-varying (non-stationary).

An example of a nonstationary process is speech, whose loudness and spectral compositionchanges continuously as the speaker generates various sounds. A process isstationary if the parameters of the probability model of the process are timeinvariant; otherwise it is non-stationary (Figure 3.4). The stationarityproperty implies that all the parameters, such as the mean, the variance, thepower spectral composition and the higher-order moments of the process,are time-invariant. In practice, there are various degrees of stationarity: itmay be that one set of the statistics of a process is stationary, whereasanother set is time-varying.

For example, a random process may have atime-invariant mean, but a time-varying power.Figure 3.4 Examples of a quasistationary and a non-stationary speech segment.54Probability ModelsExample 3.3 In this example, we consider the time-averaged values of themean and the power of: (a) a stationary signal Asinωt and (b) a transientsignal Ae-αt.The mean and power of the sinusoid are1Asin ω t dt = 0 ,T∫constant(3.20)A21 2 2,A sin ω t dt =∫T2constant(3.21)Mean( Asin ωt ) =TPower( Asin ω t ) =TWhere T is the period of the sine wave.

The mean and the power of thetransient signal are given by:Mean( Ae−αt1)=Tt +T∫ Ae−ατtdτ =A(1 − e −αT )e −αt ,αTtime-varying(3.22)Power( Ae−αt ) =+A21 t T 2 −2 ατ(1 − e −2 αT )e −2 αt , time-varyingτ=AedT ∫2α Tt(3.23)In Equations (3.22) and (3.23), the signal mean and power are exponentiallydecaying functions of the time variable t.Example 3.4 Consider a non-stationary signal y(m) generated by a binarystate random process described by the following equation:y (m) = s (m) x0 (m)+ s (m) x1 (m)(3.24)where s(m) is a binary-valued state indicator variable and s (m) denotes thebinary complement of s(m). From Equation (3.24), we have x ( m)y ( m) =  0 x1 (m)if s (m) = 0if s (m) = 1(3.25)Stationary and Non-Stationary Random Processes55Let µ x 0 and Px0 denote the mean and the power of the signal x0(m), andµ x1 and Px1 the mean and the power of x1(m) respectively. The expectationof y(m), given the state s(m), is obtained asE [y (m) s (m)] = s (m)E [x0 (m)] + s (m)E [x1 (m)]= s ( m) µ x0 + s ( m) µ x1(3.26)In Equation (3.26), the mean of y(m) is expressed as a function of the stateof the process at time m.

The power of y(m) is given byE [y 2 (m) s(m)]= s (m)E [x02 (m)]+ s(m)E [x12 (m)]= s (m) Px0 + s (m) Px1(3.27)Although many signals are non-stationary, the concept of a stationaryprocess has played an important role in the development of signalprocessing methods. Furthermore, even non-stationary signals such asspeech can often be considered as approximately stationary for a shortperiod of time. In signal processing theory, two classes of stationaryprocesses are defined: (a) strict-sense stationary processes and (b) widesense stationary processes, which is a less strict form of stationarity, in thatit only requires that the first-order and second-order statistics of the processshould be time-invariant.3.3.1 Strict-Sense Stationary ProcessesA random process X(m) is stationary in a strict sense if all its distributionsand statistical parameters are time-invariant. Strict-sense stationarity impliesthat the nth order distribution is translation-invariant for all n=1, 2,3, … :Prob[ x(m1 ) ≤ x1 , x(m2 ) ≤ x2 , , x(mn ) ≤ x n )]= Prob[ x(m1 + τ ) ≤ x1 , x(m2 + τ ) ≤ x 2 , , x(mn + τ ) ≤ x n )](3.28)From Equation (3.28) the statistics of a strict-sense stationary processincluding the mean, the correlation and the power spectrum, are timeinvariant; therefore we haveE[ x(m)] = µ x(3.29)56Probability ModelsE[ x(m) x(m + k )] = rxx (k )(3.30)andE [| X ( f , m) | 2 ] = E [| X ( f ) | 2 ] = PXX ( f )(3.31)where µx, rxx(m) and PXX(f) are the mean value, the autocorrelation and thepower spectrum of the signal x(m) respectively, and X(f,m) denotes thefrequency–time spectrum of x(m).3.3.2 Wide-Sense Stationary ProcessesThe strict-sense stationarity condition requires that all statistics of theprocess should be time-invariant.

A less restrictive form of a stationaryprocess is so-called wide-sense stationarity. A process is said to be widesense stationary if the mean and the autocorrelation functions of the processare time invariant:(3.32)E[ x(m)] = µ xE[ x(m) x(m + k )] = rxx (k )(3.33)From the definitions of strict-sense and wide-sense stationary processes, it isclear that a strict-sense stationary process is also wide-sense stationary,whereas the reverse is not necessarily true.3.3.3 Non-Stationary ProcessesA random process is non-stationary if its distributions or statistics vary withtime. Most stochastic processes such as video signals, audio signals,financial data, meteorological data, biomedical signals, etc., are nonstationary, because they are generated by systems whose environments andparameters vary over time. For example, speech is a non-stationary processgenerated by a time-varying articulatory system.