Hidden Markov Models (Vaseghi - Advanced Digital Signal Processing and Noise Reduction), страница 5

Theimplementation of the Wiener filter, Equation (5.56), requires the signal andthe noise power spectra. The power-spectral variables may be obtained fromthe ML states of the HMMs trained to model the power spectra of the signaland the noise. Figure 5.14 illustrates an implementation of HMM-basedstate-dependent Wiener filters. To implement the state-dependent Wienerfilter, we need an estimate of the state sequences for the signal and thenoise. In practice, for signals such as speech there are a number of HMMs;one HMM per word, phoneme, or any other elementary unit of the signal.

Insuch cases it is necessary to classify the signal, so that the state-basedWiener filters are derived from the most likely HMM. Furthermore the noiseprocess can also be modelled by an HMM. Assuming that there are VHMMs {M1, ..., MV} for the signal process, and one HMM for the noise, thestate-based Wiener filter can be implemented as follows:174Hidden Markov ModelsStep 1: Combine the signal and noise models to form the noisy signalmodels.Step 2: Given the noisy signal, and the set of combined noisy signalmodels, obtain the ML combined noisy signal model.Step 3: From the ML combined model, obtain the ML state sequence ofspeech and noise.Step 4: Use the ML estimate of the power spectra of the signal and thenoise to program the Wiener filter Equation (5.56).Step 5: Use the state-dependent Wiener filters to filter the signal.5.7.1 Modelling Noise CharacteristicsThe implicit assumption in using an HMM for noise is that noise statisticscan be modelled by a Markovian chain of N different stationary processes.A stationary noise process can be modelled by a single-state HMM.

For anon-stationary noise, a multi-state HMM can model the time variations ofthe noise process with a finite number of quasi-stationary states. In general,the number of states required to accurately model the noise depends on thenon-stationary character of the noise.An example of a non-stationary noise process is the impulsive noise ofFigure 5.15. Figure 5.16 shows a two-state HMM of the impulsive noisesequence where the state S0 models the “off” periods between the impulsesand the state S1 models an impulse.

In cases where each impulse has a welldefined temporal structure, it may be beneficial to use a multistate HMM tomodel the pulse itself. HMMs are used in Chapter 12 for modellingimpulsive noise, and in Chapter 15 for channel equalisation.5.8 SummaryHMMs provide a powerful method for the modelling of non-stationaryprocesses such as speech, noise and time-varying channels.

An HMM is aBayesian finite-state process, with a Markovian state prior, and a statelikelihood function that can be either a discrete density model or acontinuous Gaussian pdf model. The Markovian prior models the timeevolution of a non-stationary process with a chain of stationary subprocesses. The state observation likelihood models the space of the processwithin each state of the HMM.175SummaryFigure 5.15 Impulsive noise.a =α12a =α22S1a =1-α11S2a =1 - α21Figure 5.16 A binary-state model of an impulsive noise process.In Section 5.3, we studied the Baum–Welch method for the training ofthe parameters of an HMM to model a given data set, and derived theforward–backward method for efficient calculation of the likelihood of anHMM given an observation signal. In Section 5.4, we considered the use ofHMMs in signal classification and in the decoding of the underlying statesequence of a signal.

The Viterbi algorithm is a computationally efficientmethod for estimation of the most likely sequence of an HMM. Given anunlabelled observation signal, the decoding of the underlying state sequenceand the labelling of the observation with one of number of candidate HMMsare accomplished using the Viterbi method. In Section 5.5, we consideredthe use of HMMs for MAP estimation of a signal observed in noise, andconsidered the use of HMMs in implementation of state-based Wiener filtersequence.BibliographyBAHL L.R., BROWN P.F., de SOUZA P.V. and MERCER R.L. (1986) MaximumMutual Information Estimation of Hidden Markov Model Parameters forSpeech Recognition.

IEEE Proc. Acoustics, Speech and Signal176Hidden Markov ModelsProcessing, ICASSP-86 Tokyo, pp. 40–43.BAHL L.R., JELINEK F. and MERCER R.L. (1983) A Maximum LikelihoodApproach to Continuous Speech Recognition. IEEE Trans. PatternAnalysis and Machine Intelligence, 5, pp. 179–190.BAUM L.E. and EAGON J.E. (1967) An Inequality with Applications toStatistical Estimation for Probabilistic Functions of a Markov Processand to Models for Ecology. Bull. AMS, 73, pp. 360-363.BAUM L.E. and PETRIE T.

(1966) Statistical Inference for ProbabilisticFunctions of Finite State Markov Chains. Ann. Math. Stat. 37, pp. 1554–1563.BAUM L.E., PETRIE T., SOULES G. and WEISS N. (1970) A MaximisationTechnique Occurring in the Statistical Analysis of ProbabilisticFunctions of Markov Chains.

Ann. Math. Stat., 41, pp. 164–171.CONNER P.N. (1993) Hidden Markov Model with Improved Observation andDuration Modelling. PhD. Thesis, University of East Anglia, England.EPHARAIM Y., MALAH D. and JUANG B.H.(1989) On Application of HiddenMarkov Models for Enhancing Noisy Speech. IEEE Trans.

AcousticsSpeech and Signal Processing, 37(12), pp. 1846-1856, Dec.FORNEY G.D. (1973) The Viterbi Algorithm. Proc. IEEE, 61, pp. 268–278.GALES M.J.F. and YOUNG S.J. (1992) An Improved Approach to the HiddenMarkov Model Decomposition of Speech and Noise. Proc. IEEE, Int.Conf. on Acoust., Speech, Signal Processing, ICASSP-92, pp. 233–235.GALES M.J.F. and YOUNG S.J. (1993) HMM Recognition in Noise usingParallel Model Combination. Eurospeech-93, pp. 837–840.HUANG X.D., ARIKI Y. and JACK M.A. (1990) Hidden Markov Models forSpeech Recognition. Edinburgh University Press, Edinburgh.HUANG X.D.

and JACK M.A. (1989) Unified Techniques for VectorQuantisation and Hidden Markov Modelling using Semi-ContinuousModels. IEEE Proc. Acoustics, Speech and Signal Processing, ICASSP89 Glasgow, pp. 639–642.JELINEK F. and MERCER R. (1980) Interpolated Estimation of Markov SourceParameters from Sparse Data. Proc. of the Workshop on PatternRecognition in Practice. North-Holland, Amesterdam.JELINEK F, (1976) Continuous Speech Recognition by Statistical Methods.Proc.

of IEEE, 64, pp. 532–555.JUANG B.H. (1985) Maximum-Likelihood Estimation for Mixture MultiVariate Stochastic Observations of Markov Chain. AT&T Belllaboratories Tech J., 64, pp. 1235–1249.JUANG B.H. (1984) On the Hidden Markov Model and Dynamic TimeBibliography177Warping for Speech Recognition- A unified Overview.

AT&T TechnicalJ., 63, pp. 1213–1243.KULLBACK S. and LEIBLER R.A. (1951) On Information and Sufficiency.Ann. Math. Stat., 22, pp. 79–85.LEE K.F. (1989) Automatic Speech Recognition: the Development ofSPHINX System. MA: Kluwer Academic Publishers, Boston.LEE K.F. (1989) Hidden Markov Model: Past, Present and Future.Eurospeech-89, Paris.LIPORACE L.R. (1982) Maximum Likelihood Estimation for Multi-VariateObservations of Markov Sources. IEEE Trans. IT, IT-28, pp. 729–735.MARKOV A.A. (1913) An Example of Statistical Investigation in the text ofEugen Onyegin Illustrating Coupling of Tests in Chains.

Proc. Acad. Sci.St Petersburg VI Ser., 7, pp. 153–162.MILNER B.P. (1995) Speech Recognition in Adverse Environments, PhD.Thesis, University of East Anglia, England.PETERIE T. (1969) Probabilistic Functions of Finite State Markov Chains.Ann. Math. Stat., 40, pp. 97–115.RABINER L.R. and JUANG B.H. (1986) An Introduction to Hidden MarkovModels. IEEE ASSP.

Magazine, pp. 4–15.RABINER L.R., JUANG B.H., LEVINSON S.E. and SONDHI M.M., (1985)Recognition of Isolated Digits using Hidden Markov Models withContinuous Mixture Densities. AT&T Technical Journal, 64, pp. 12111235.RABINER L.R. and JUANG B.H. (1993) Fundamentals of Speech Recognition.Prentice-Hall, Englewood Cliffs, NJ.YOUNG S.J. (1999), HTK: Hidden Markov Model Tool Kit. CambridgeUniversity Engineering Department.VARGA A.

and MOORE R.K., Hidden Markov Model Decomposition ofSpeech and Noise. in Proc. IEEE Int., Conf. on Acoust., Speech, SignalProcessing, 1990, pp. 845–848VITERBI A.J. (1967) Error Bounds for Convolutional Codes and anAsymptotically Optimum Decoding Algorithm. IEEE Trans. onInformation theory, IT-13, pp. 260–269..