Bayesian Estimation (779797)
Текст из файла
Advanced Digital Signal Processing and Noise Reduction, Second Edition.Saeed V. VaseghiCopyright © 2000 John Wiley & Sons LtdISBNs: 0-471-62692-9 (Hardback): 0-470-84162-1 (Electronic)f(y,θ)f(θ|y2 )4θf(θ|y1 )y2y1yBAYESIAN ESTIMATION4.14.24.34.44.54.64.74.8Bayesian Estimation Theory: Basic DefinitionsBayesian EstimationThe Estimate–Maximise MethodCramer–Rao Bound on the Minimum Estimator VarianceDesign of Mixture Gaussian ModelsBayesian ClassificationModeling the Space of a Random ProcessSummaryBayesian estimation is a framework for the formulation of statisticalinference problems.
In the prediction or estimation of a randomprocess from a related observation signal, the Bayesian philosophy isbased on combining the evidence contained in the signal with priorknowledge of the probability distribution of the process. Bayesianmethodology includes the classical estimators such as maximum a posteriori(MAP), maximum-likelihood (ML), minimum mean square error (MMSE)and minimum mean absolute value of error (MAVE) as special cases. Thehidden Markov model, widely used in statistical signal processing, is anexample of a Bayesian model. Bayesian inference is based on minimisationof the so-called Bayes’ risk function, which includes a posterior model ofthe unknown parameters given the observation and a cost-of-error function.This chapter begins with an introduction to the basic concepts of estimationtheory, and considers the statistical measures that are used to quantify theperformance of an estimator. We study Bayesian estimation methods andconsider the effect of using a prior model on the mean and the variance of anestimate.
The estimate–maximise (EM) method for the estimation of a set ofunknown parameters from an incomplete observation is studied, and appliedto the mixture Gaussian modelling of the space of a continuous randomvariable. This chapter concludes with an introduction to the Bayesianclassification of discrete or finite-state signals, and the K-means clusteringmethod.90Bayesian Estimation4.1 Bayesian Estimation Theory: Basic DefinitionsEstimation theory is concerned with the determination of the best estimateof an unknown parameter vector from an observation signal, or the recoveryof a clean signal degraded by noise and distortion. For example, given anoisy sine wave, we may be interested in estimating its basic parameters(i.e.
amplitude, frequency and phase), or we may wish to recover the signalitself. An estimator takes as the input a set of noisy or incompleteobservations, and, using a dynamic model (e.g. a linear predictive model)and/or a probabilistic model (e.g. Gaussian model) of the process, estimatesthe unknown parameters. The estimation accuracy depends on the availableinformation and on the efficiency of the estimator. In this chapter, theBayesian estimation of continuous-valued parameters is studied. Themodelling and classification of finite-state parameters is covered in the nextchapter.Bayesian theory is a general inference framework. In the estimation orprediction of the state of a process, the Bayesian method employs both theevidence contained in the observation signal and the accumulated priorprobability of the process. Consider the estimation of the value of a randomparameter vector θ, given a related observation vector y.
From Bayes’ rulethe posterior probability density function (pdf) of the parameter vector θgiven y, f Θ |Y (θ | y) , can be expressed asfΘ |Y (θ | y ) =fY |Θ ( y | θ ) fΘ (θ )fY ( y )(4.1)where for a given observation, fY(y) is a constant and has only a normalisingeffect. Thus there are two variable terms in Equation (4.1): one termfY|Θ(y|θ) is the likelihood that the observation signal y was generated by theparameter vector θ and the second term is the prior probability of theparameter vector having a value of θ.
The relative influence of thelikelihood pdf fY|Θ(y|θ) and the prior pdf fΘ(θ) on the posterior pdf fΘ|Y(θ|y)depends on the shape of these function, i.e. on how relatively peaked eachpdf is. In general the more peaked a probability density function, the more itwill influence the outcome of the estimation process. Conversely, a uniformpdf will have no influence.The remainder of this chapter is concerned with different forms of Bayesianestimation and its applications. First, in this section, some basic concepts ofestimation theory are introduced.91Basic Definitions4.1.1 Dynamic and Probability Models in EstimationOptimal estimation algorithms utilise dynamic and statistical models of theobservation signals.
A dynamic predictive model captures the correlationstructure of a signal, and models the dependence of the present and futurevalues of the signal on its past trajectory and the input stimulus. A statisticalprobability model characterises the random fluctuations of a signal in termsof its statistics, such as the mean and the covariance, and most completely interms of a probability model. Conditional probability models, in addition tomodelling the random fluctuations of a signal, can also model thedependence of the signal on its past values or on some other related process.As an illustration consider the estimation of a P-dimensional parametervector θ =[θ0,θ1, ..., θP–1] from a noisy observation vector y=[y(0), y(1), ...,y(N–1)] modelled asy = h (θ ,x ,e )+ n(4.2)where, as illustrated in Figure 4.1, the function h(·) with a random input e,output x, and parameter vector θ, is a predictive model of the signal x, and nis an additive random noise process.
In Figure 4.1, the distributions of therandom noise n, the random input e and the parameter vector θ are modelledby probability density functions, fN(n), fE(e), and fΘ(θ) respectively. The pdfmodel most often used is the Gaussian model. Predictive and statisticalmodels of a process guide the estimator towards the set of values of theunknown parameters that are most consistent with both the prior distributionof the model parameters and the noisy observation. In general, the moremodelling information used in an estimation process, the better the results,provided that the models are an accurate characterisation of the observationand the parameter process.Noise processfN(n)Parameter processfΘ (θ)θExcitation processfE(e)ePredictive modelhΘ (θ, x, e)nxy =x+nFigure 4.1 A random process y is described in terms of a predictive model h(· ),and statistical models fE(· ), fΘ(· ) and fN(· ).92Bayesian Estimation4.1.2 Parameter Space and Signal SpaceConsider a random process with a parameter vector θ.
For example, eachinstance of θ could be the parameter vector for a dynamic model of a speechsound or a musical note. The parameter space of a process Θ is thecollection of all the values that the parameter vector θ can assume. Theparameters of a random process determine the “character” (i.e. the mean, thevariance, the power spectrum, etc.) of the signals generated by the process.As the process parameters change, so do the characteristics of the signalsgenerated by the process.
Each value of the parameter vector θ of a processhas an associated signal space Y; this is the collection of all the signalrealisations of the process with the parameter value θ. For example,consider a three-dimensional vector-valued Gaussian process with parametervector θ =[µ, Σ ], where µ is the mean vector and Σ is the covariance matrixof the Gaussian process. Figure. 4.2 illustrates three mean vectors in a threedimensional parameter space. Also shown is the signal space associatedwith each parameter. As shown, the signal space of each parameter vector ofa Gaussian process contains an infinite number of points, centred on themean vector µ, and with a spatial volume and orientation that aredetermined by the covariance matrix Σ. For simplicity, the variances are notshown in the parameter space, although they are evident in the shape of theGaussian signal clusters in the signal space.µ1y1Parameter spaceMappingMappingN ( y, µ 2 , Σ 2 )MappingSignal spaceN ( y, µ1, Σ1)µ3y3N ( y, µ3 , Σ 3 )µ2y2Figure 4.2 Illustration of three points in the parameter space of a Gaussian processand the associated signal spaces, for simplicity the variances are not shown inparameter space.93Basic Definitions4.1.3 Parameter Estimation and Signal RestorationParameter estimation and signal restoration are closely related problems.The main difference is due to the rapid fluctuations of most signals incomparison with the relatively slow variations of most parameters.
Forexample, speech sounds fluctuate at speeds of up to 20 kHz, whereas theunderlying vocal tract and pitch parameters vary at a relatively lower rate ofless than 100 Hz. This observation implies that normally more averagingcan be done in parameter estimation than in signal restoration.As a simple example, consider a signal observed in a zero-mean randomnoise process. Assume we wish to estimate (a) the average of the cleansignal and (b) the clean signal itself.
As the observation length increases, theestimate of the signal mean approaches the mean value of the clean signal,whereas the estimate of the clean signal samples depends on the correlationstructure of the signal and the signal-to-noise ratio as well as on theestimation method used.As a further example, consider the interpolation of a sequence of lostsamples of a signal given N recorded samples, as illustrated in Figure 4.3.Assume that an autoregressive (AR) process is used to model the signal asy = Xθ + e + n(4.3)where y is the observation signal, X is the signal matrix, θ is the ARparameter vector, e is the random input of the AR model and n is therandom noise. Using Equation (4.3), the signal restoration process involvesthe estimation of both the model parameter vector θ and the random input efor the lost samples.
Assuming the parameter vector θ is time-invariant, theestimate of θ can be averaged over the entire N observation samples, and asN becomes infinitely large, a consistent estimate should approach the trueLostsamplesInput signal ySignal estimator(Interpolator)Restored signal x^θParameterestimatorFigure 4.3 Illustration of signal restoration using a parametric model of thesignal process.94Bayesian Estimationparameter value.
Характеристики
Тип файла PDF
PDF-формат наиболее широко используется для просмотра любого типа файлов на любом устройстве. В него можно сохранить документ, таблицы, презентацию, текст, чертежи, вычисления, графики и всё остальное, что можно показать на экране любого устройства. Именно его лучше всего использовать для печати.
Например, если Вам нужно распечатать чертёж из автокада, Вы сохраните чертёж на флешку, но будет ли автокад в пункте печати? А если будет, то нужная версия с нужными библиотеками? Именно для этого и нужен формат PDF - в нём точно будет показано верно вне зависимости от того, в какой программе создали PDF-файл и есть ли нужная программа для его просмотра.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
