Hidden Markov Models (779805), страница 4
Текст из файла (страница 4)
In a state–time trellis diagram, such as Figure 5.8, thenumber of paths diverging from each state of a trellis can growexponentially by a factor of N at successive time instants. The Viterbi166Hidden Markov Modelsmethod prunes the trellis by selecting the most likely path to each state.
Ateach time instant t, for each state i, the algorithm selects the most probablepath to state i and prunes out the less likely branches. This procedureensures that at any time instant, only a single path survives into each state ofthe trellis.For each time instant t and for each state i, the algorithm keeps a recordof the state j from which the maximum-likelihood path branched into i, andalso records the cumulative probability of the most likely path into state i attime t. The Viterbi algorithm is given on the next page, and Figure 5.11gives a network illustration of the algorithm.Viterbi Algorithmδ t (i ) records the cumulative probability of the best path to state i at time t.ψ t (i ) records the best state sequence to state i at time t.Step 1: Initialisation, at time t=0, for states i=1, …, Nδ 0 (i )=π i f i ( x (0))ψ 0 (i )=0Step 2: Recursive calculation of the ML state sequences and theirprobabilitiesFor time t =1, …, T–1For states i = 1, …, Nδ t (i )=max [δ t −1 ( j )a ji ] f i ( x (t ))jψ t (i )=arg max [δ t −1 ( j )a ji ]jStep 3: Termination, retrieve the most likely final states MAP (T − 1) =arg max [δ T −1 (i )]iProbmax =max [δ T −1 (i )]iStep 4: Backtracking through the most likely state sequence:For t = T–2, …, 0s MAP (t )=ψ t +1 s MAP (t + 1) .[]HMM-Based Estimation of Signals in Noise167The backtracking routine retrieves the most likely state sequence of themodel M.
Note that the variable Probmax, which is the probability of theobservation sequence X=[x(0), ..., x(T–1)] and the most likely statesequence of the model M, can be used as the probability score for the modelM and the observation X. For example, in speech recognition, for eachcandidate word model the probability of the observation and the most likelystate sequence is calculated, and then the observation is labelled with theword that achieves the highest probability score.5.5 HMM-Based Estimation of Signals in NoiseIn this section, and the following two sections, we consider the use ofHMMs for estimation of a signal x(t) observed in an additive noise n(t), andmodelled asy ( t ) = x ( t )+ n ( t )(5.43)From Bayes’ rule, the posterior pdf of the signal x(t) given the noisyobservation y(t) is defined asf X |Y ( x (t ) y (t ) ) =fY | X ( y (t ) x (t ) ) f X ( x (t ))f Y ( y (t ))1f N ( y (t ) − x (t ) ) f X ( x (t ) )=fY ( y (t ))(5.44)For a given observation, fY(y(t)) is a constant, and the maximum a posteriori(MAP) estimate is obtained asxˆ MAP (t ) = arg max f N ( y (t ) − x (t ) ) f X ( x (t ) )x (t )(5.45)The computation of the posterior pdf, Equation (5.44), or the MAP estimateEquation (5.45), requires the pdf models of the signal and the noiseprocesses.
Stationary, continuous-valued, processes are often modelled by aGaussian or a mixture Gaussian pdf that is equivalent to a single-stateHMM. For a non-stationary process an N-state HMM can model the time-168Hidden Markov Modelsvarying pdf of the process as a Markovian chain of N stationary Gaussiansubprocesses. Now assume that we have an Ns-state HMM M for the signal,and another Nn-state HMM η for the noise.
For signal estimation, we needestimates of the underlying state sequences of the signal and the noiseTprocesses. For an observation sequence of length T, there are N s possibleTsignal state sequences and N n possible noise state sequences that couldhave generated the noisy signal. Since it is assumed that the signal and noiseare uncorrelated, each signal state may be observed in any noisy state;TTtherefore the number of noisy signal states is on the order of N s × N n .Given an observation sequence Y=[y(0), y(1), ..., y(T–1)], the mostprobable state sequences of the signal and the noise HMMs maybeexpressed asMAP= arg max max f Y (Y , ssignal , s noise M ,η )s signals signal s noise(5.46)MAP= arg max max f Y (Y , ssignal , s noise M ,η )s noises noise s signal(5.47)andGiven the state sequence estimates for the signal and the noise models, theMAP estimation Equation (5.45) becomes(xˆ MAP (t ) = arg max f NxS ,ηMAPMAP,η ) f X S ,M (x (t ) ssignal(y(t ) − x(t ) snoise, M )) (5.48)Implementation of Equations (5.46)–(5.48) is computationally prohibitive.In Sections 5.6 and 5.7, we consider some practical methods for theestimation of signal in noise.Example Assume a signal, modelled by a binary-state HMM, is observedin an additive stationary Gaussian noise.
Let the noisy observation bemodelled asy ( t ) = s ( t ) x 0 ( t )+ s ( t ) x1 ( t )+ n ( t )(5.49)where s (t ) is a hidden binary-state process such that: s (t ) = 0 indicates that169HMM-Based Estimation of Signals in Noisethe signal is from the state S0 with a Gaussian pdf of N ( x ( t ), µ x 0 , Σ x 0 x 0 ) ,and s(t) = 1 indicates that the signal is from the state S1 with a Gaussian pdfof N ( x ( t ), µ x 1 , Σ x1 x1 ) .
Assume that a stationary Gaussian processN ( n ( t ), µ n , Σ nn ) , equivalent to a single-state HMM, can model the noise.Using the Viterbi algorithm the maximum a posteriori (MAP) statesequence of the signal model can be estimated as[]MAPs signal=arg max f Y |S ,M (Y s, M )PS M (s M )s(5.50)For a Gaussian-distributed signal and additive Gaussian noise, theobservation pdf of the noisy signal is also Gaussian. Hence, the stateobservation pdfs of the signal model can be modified to account for theadditive noise asandf Y |s 0 ( y ( t ) s 0 ) = N ( y ( t ), ( µ x 0 + µ n ), ( Σ x 0 x 0 + Σ nn ) )(5.51)f Y |s1 ( y ( t ) s1 ) =N ( y ( t ), ( µ x1 + µ n ), ( Σ x1 x1 + Σ nn ) )(5.52)where N ( y (t ), µ , Σ ) denotes a Gaussian pdf with mean vector µ andcovariance matrix Σ .
The MAP signal estimate, given a state sequenceestimate sMAP, is obtained from[()]xˆ MAP (t )=arg max f X |S ,M x ( t ) s MAP , M f N ( y ( t ) − x ( t ) )x(5.53)Substitution of the Gaussian pdf of the signal from the most likely statesequence, and the pdf of noise, in Equation (5.53) results in the followingMAP estimate:−1−1xˆ MAP (t ) = (Σ xx ,s(t ) + Σ nn ) Σ xx,s(t ) ( y( t ) − µ n ) + (Σ xx , s( t) + Σ nn ) Σ nn µ x, s(t )(5.54)where µ x,s(t) and Σ xx ,s( t) are the mean vector and covariance matrix of thesignal x(t) obtained from the most likely state sequence [s(t)].170Hidden Markov ModelsSpeechHMMsNoisy speechSpeechstatesNoisy speechHMMsModelStatecombinationdecompositionSpeechWienerfilterNoisestatesNoiseNoiseHMMsFigure 5.12 Outline configuration of HMM-based noisy speech recognition andenhancement.5.6 Signal and Noise Model Combination and DecompositionFor Bayesian estimation of a signal observed in additive noise, we need tohave an estimate of the underlying statistical state sequences of the signaland the noise processes.
Figure 5.12 illustrates the outline of an HMMbased noisy speech recognition and enhancement system. The systemperforms the following functions:(1) combination of the speech and noise HMMs to form the noisyspeech HMMs;(2) estimation of the best combined noisy speech model given thecurrent noisy speech input;(3) state decomposition, i.e. the separation of speech and noise statesgiven noisy speech states;(4) state-based Wiener filtering using the estimates of speech and noisestates.5.6.1 Hidden Markov Model CombinationThe performance of HMMs trained on clean signals deteriorates rapidly inthe presence of noise, since noise causes a mismatch between the cleanHMMs and the noisy signals. The noise-induced mismatch can be reduced:either by filtering the noise from the signal (for example using the Wienerfiltering and the spectral subtraction methods described in Chapters 6 and11) or by combining the noise and the signal models to model the noisySignal and Noise Model Combination and Decomposition171signal.
The model combination method was developed by Gales and Young.In this method HMMs of speech are combined with an HMM of noise toform HMMs of noisy speech signals. In the power-spectral domain, themean vector and the covariance matrix of the noisy speech can beapproximated by adding the mean vectors and the covariance matrices ofspeech and noise models:µ y = µ x + gµ n(5.55)Σ yy = Σ xx + g 2Σ nn(5.56)Model combination also requires an estimate of the current signal-to-noiseratio for calculation of the scaling factor g in Equations (5.55) and (5.56). Incases such as speech recognition, where the models are trained on cepstralfeatures, the model parameters are first transformed from cepstral featuresinto power spectral features before using the additive linear combinationEquations (5.55) and (5.56).
Figure 5.13 illustrates the combination of a 4state left–right HMM of a speech signal with a 2-state ergodic HMM ofnoise. Assuming that speech and noise are independent processes, eachspeech state must be combined with every possible noise state to give thenoisy speech model. It is assumed that the noise process only affects themean vectors and the covariance matrices of the speech model; hence thetransition probabilities of the speech model are not modified.5.6.2 Decomposition of State Sequences of Signal and NoiseThe HMM-based state decomposition problem can be stated as follows:given a noisy signal and the HMMs of the signal and the noise processes,estimate the underlying states of the signal and the noise.HMM state decomposition can be obtained using the following method:(a) Given the noisy signal and a set of combined signal and noisemodels, estimate the maximum-likelihood (ML) combined noisyHMM for the noisy signal.(b) Obtain the ML state sequence of from the ML combined model.(c) Extract the signal and noise states from the ML state sequence of theML combined noisy signal model.The ML state sequences provide the probability density functions for thesignal and noise processes.
The ML estimates of the speech and noise pdfs172Hidden Markov Modelsa33a22a11a12a44a23a34s2s1+s4s3Speech model=a22sasbNoise modela11s1aa11a22a12s2aa23s2bs1ba44a33s3aa34s3bs4as4bNoisy speech modelFigure 5.13 Outline configuration of HMM-based noisy speech recognition andenhancement. Sij is a combination of the state i of speech with the state j of noise.may then be used in Equation (5.45) to obtain a MAP estimate of the speechsignal. Alternatively the mean spectral vectors of the speech and noise fromthe ML state sequences can be used to program a state-dependent Wienerfilter as described in the next section.5.7 HMM-Based Wiener FiltersThe least mean square error Wiener filter is derived in Chapter 6. For astationary signal x(m), observed in an additive noise n(m), the Wiener filterequations in the time and the frequency domains are derived as :w = ( R xx + Rnn )−1 r xx(5.55)andW( f ) =PXX ( f )PXX ( f ) + PNN ( f )(5.56)173HMM-Based Wiener FiltersNoise HMMSignal HMMModel CombinationNoisy SignalML Model Estimation andState DecompositionPXX ( f )PNN ( f )W( f ) =PXX ( f )PXX ( f ) + PNN ( f )Wiener Filter SequenceFigure 5.14 Illustrations of HMMs with state-dependent Wiener filters.where Rxx, rxx and PXX(f) denote the autocorrelation matrix, theautocorrelation vector and the power-spectral functions respectively.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
