Bayesian Estimation (779797), страница 8
Текст из файла (страница 8)
The steps in K-Means method are as follows:Modelling the Space of a Random Signal139Select initial centroids andform cluster partitionsUpdate cluster centroidsUpdate cluster partitionsUpdate cluster centroidsFigure 4.18 Illustration of the K-means clustering method.Step 1: Initialisation Use a suitable method to choose a set of K initialcentroids [ci]. For m = 1, 2, . .
.Step 2: Classification Classify the training vectors {x} into K clusters {[x1],[x2], ... [xK]} using the so-called nearest-neighbour rule Equation(4.154).Step 3: Centroid computation Use the vectors [xi] associated with the ithcluster to compute an updated cluster centroid ci, and calculate thecluster distortion defined asDi ( m) =1NiNi∑ d ( x i ( j ), ci (m))(4.155)j =1where it is assumed that a set of Ni vectors [xi(j) j=0, ..., Ni] areassociated with cluster i.
The total distortion is given byKD ( m) = ∑ Di ( m)i =1(4.156)Bayesian Estimation140Step 4: Convergence test:ifD( m − 1) − D(m ) ≥ Threshold stop,elsegoto Step 2.A vector quantiser models the regions, or the clusters, of the signal spacewith a set of cluster centroids. A more complete description of the signalspace can be achieved by modelling each cluster with a Gaussian density asdescribed in the next chapter.4.8 SummaryThis chapter began with an introduction to the basic concepts in estimationtheory; such as the signal space and the parameter space, the prior andposterior spaces, and the statistical measures that are used to quantify theperformance of an estimator. The Bayesian inference method, with itsability to include as much information as is available, provides a generalframework for statistical signal processing problems.
The minimum meansquare error, the maximum-likelihood, the maximum a posteriori, and theminimum absolute value of error methods were derived from the Bayesianformulation. Further examples of the applications of Bayesian type modelsin this book include the hidden Markov models for non-stationary processesstudied in Chapter 5, and blind equalisation of distorted signals studied inChapter 15.We considered a number of examples of the estimation of a signalobserved in noise, and derived the expressions for the effects of using priorpdfs on the mean and the variance of the estimates. The choice of the priorpdf is an important consideration in Bayesian estimation.
Many processes,for example speech or the response of a telecommunication channel, are notuniformly distributed in space, but are constrained to a particular region ofsignal or parameter space. The use of a prior pdf can guide the estimator tofocus on the posterior space that is the subspace consistent with both thelikelihood and the prior pdfs. The choice of the prior, depending on howwell it fits the process, can have a significant influence on the solutions.The iterative estimate-maximise method, studied in Section 4.3,provides a practical framework for solving many statistical signalprocessing problems, such as the modelling of a signal space with a mixtureGaussian densities, and the training of hidden Markov models in Chapter 5.In Section 4.4 the Cramer–Rao lower bound on the variance of an estimatorBibliography141was derived, and it was shown that the use of a prior pdf can reduce theminimum estimator variance.Finally we considered the modelling of a data space with a mixtureGaussian process, and used the EM method to derive a solution for theparameters of the mixture Gaussian model.BibliographyANDERGERG M.R.
(1973) Cluster Analysis for Applications. Academic Press,New York.ABRAMSON N. (1963) Information Theory and Coding. McGraw Hill, NewYork.BAUM L.E., PETRIE T., SOULES G. and WEISS N. (1970) A MaximisationTechnique occurring in the Statistical Analysis of ProbabilisticFunctions of Markov Chains. Ann. Math. Stat. 41, pp.164–171.BAYES T. (1763) An Essay Towards Solving a Problem in the Doctrine ofChanges, Phil.
Trans. Royal Society of London, 53, pp. 370–418,(reprinted in 1958 in Biometrika, 45, pp. 293–315).CHOU P. LOOKABAUGH T. and GRAY R. (1989) Entropy-Constrained VectorQuantisation. IEEE Trans. Acoustics, Speech and Signal Processing,ASSP-37, pp. 31–42.BEZDEK J.C. (1981) Pattern Recognition with Fuzzy Objective FunctionAlgorithms. Plenum Press, New York.CRAMER H. (1974) Mathematical Methods of Statistics. Princeton UniversityPress.DEUTSCH R. (1965) Estimation Theory. Prentice-Hall, Englewood Cliffs, NJ.DEMPSTER A.P., LAIRD N.M. and RUBIN D.B. (1977) Maximum Likelihoodfrom Incomplete Data via the EM Algorithm.
J. R. Stat. Soc. Ser. B,39, pp. 1-38.DUDA R.O. and HART R.E. (1973) Pattern Classification. Wiley, New York.FEDER M. and WEINSTEIN E. (1988) Parameter Estimation of SuperimposedSignals using the EM algorithm. IEEE Trans. Acoustics, Speech andSignal Processing, ASSP-36(4), pp. 477.489.FISHER R.A. (1922) On the Mathematical Foundations of the TheoreticalStatistics. Phil Trans. Royal. Soc. London, 222, pp.
309–368.GERSHO A. (1982) On the Structure of Vector Quantisers. IEEE Trans.Information Theory, IT-28, pp. 157–166.GRAY R.M. (1984) Vector Quantisation. IEEE ASSP Magazine, p. 4-29.GRAY R.M. and KARNIN E.D (1982), Multiple local Optima in VectorQuantisers. IEEE Trans. Information Theory, IT-28, pp. 256–261.142Bayesian EstimationJEFFREY H. (1961) Scientific Inference, 3rd ed. Cambridge University Press.LARSON H.J. and BRUNO O.S.
(1979) Probabilistic Models in EngineeringSciences. I and II. Wiley, New York.LINDE Y., BUZO A. and GRAY R.M. (1980) An Algorithm for VectorQuantiser Design. IEEE Trans. Comm. COM-28, pp. 84–95.MAKHOUL J., ROUCOS S., and GISH H. (1985) Vector Quantisation in SpeechCoding. Proc. IEEE, 73, pp. 1551–1588.MOHANTY N. (1986) Random Signals, Estimation and Identification. VanNostrand, New York.RAO C.R. (1945) Information and Accuracy Attainable in the Estimation ofStatistical Parameters.
Bull Calcutta Math. Soc., 37, pp. 81–91.RENDER R.A. and WALKER H.F.(1984) Mixture Densities, MaximumLikelihood and the EM algorithm. SIAM review, 26, pp. 195–239.SCHARF L.L. (1991) Statistical Signal Processing: Detection, Estimation,and Time Series Analysis. Addison Wesley, Reading, MA..
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
