Meta tags:

Headings (most frequently used words):

bayes, estimators, example, posterior, mean, estimator, contents, definition, examples, generalized, empirical, properties, practical, of, see, also, notes, references, external, links, minimum, square, error, estimation, for, conjugate, priors, alternative, risk, functions, admissibility, asymptotic, efficiency, median, and, other, quantiles, mode, lp, estimating, in, binomial, distribution,

Text of the page (most frequently used words):
the (221), theta (125), displaystyle (119), bayes (70), and (60), #estimator (49), for (49), prior (46), posterior (42), distribution (37), widehat (34), this (33), that (27), edit (26), are (25), estimation (22), estimators (22), mean (21), example (21), with (19), bayesian (19), frac (19), risk (19), sigma (19), function (18), loss (18), which (18), statistics (17), model (16), analysis (15), generalized (15), can (15), then (14), from (13), probability (13), data (13), under (12), information (12), regression (12), empirical (12), average (12), also (12), case (12), mse (12), all (11), test (11), variance (11), error (11), rating (11), conjugate (11), likelihood (10), normal (10), statistical (10), parameter (10), weight (10), given (10), other (10), int (10), linear (9), maximum (9), theorem (9), expected (9), where (9), has (9), one (9), measurement (9), minimizes (9), correlation (8), interval (8), decision (8), plot (8), deviation (8), ratings (8), estimate (8), following (8), measurements (8), when (8), sim (8), article (8), wikipedia (7), use (7), lehmann (7), median (7), parametric (7), minimum (7), theory (7), transformation (7), chart (7), see (7), not (7), number (7), weighted (7), used (7), distributed (7), value (7), delta (7), unknown (7), rules (7), admissible (7), rule (7), main (7), toggle (6), non (6), rank (6), time (6), squared (6), over (6), there (6), such (6), based (6), improper (6), priors (6), some (6), examples (6), functions (6), page (5), log (5), general (5), binomial (5), inference (5), alternative (5), way (5), unbiased (5), moments (5), family (5), location (5), casella (5), its (5), films (5), votes (5), have (5), formula (5), since (5), particular (5), same (5), known (5), let (5), properties (5), expectation (5), observations (5), common (5), cases (5), square (5), table (4), contents (4), search (4), using (4), citations (4), different (4), design (4), models (4), limit (4), density (4), box (4), statistic (4), specific (4), series (4), distributions (4), multivariate (4), factor (4), standard (4), most (4), power (4), confidence (4), estimating (4), optimal (4), experiment (4), approximation (4), random (4), normalization (4), mode (4), arithmetic (4), links (4), isbn (4), new (4), top (4), definition (4), film (4), above (4), more (4), centered (4), another (4), beta (4), assume (4), each (4), mle (4), large (4), suppose (4), ldots (4), admissibility (4), parameters (4), yields (4), defined (4), mbox (4), tau (4), hide (4), move (4), sidebar (4), subsection (4), view (3), about (3), text (3), terms (3), was (3), articles (3), index (3), mathematics (3), portal (3), population (3), control (3), methods (3), applications (3), product (3), survival (3), least (3), squares (3), vector (3), conditional (3), cross (3), partial (3), tests (3), exponential (3), simple (3), adaptive (3), sum (3), sample (3), score (3), distance (3), method (3), efficiency (3), sampling (3), study (3), max (3), scatter (3), continuous (3), 1998 (3), references (3), imdb (3), 250 (3), section (3), posteriori (3), approach (3), many (3), movie (3), true (3), but (3), weights (3), account (3), alpha (3), often (3), thus (3), asymptotically (3), end (3), samples (3), asymptotic (3), performance (3), iid (3), belongs (3), set (3), typically (3), called (3), minimizing (3), form (3), proper (3), however (3), evidence (3), principle (3), tools (3), languages (2), contact (2), privacy (2), policy (2), available (2), commons (2), last (2), categories (2), lacking (2), november (2), 2009 (2), short (2), description (2), wikidata (2), system (2), science (2), engineering (2), studies (2), clinical (2), first (2), spectral (2), frequency (2), domain (2), autoregressive (2), structural (2), seasonal (2), adjustment (2), stationarity (2), equation (2), cluster (2), principal (2), manova (2), contingency (2), categorical (2), anova (2), poisson (2), robust (2), nonparametric (2), equations (2), validation (2), errors (2), coefficient (2), determination (2), variable (2), pearson (2), moment (2), credible (2), uniformly (2), bootstrap (2), point (2), scale (2), shape (2), natural (2), designs (2), trial (2), randomized (2), controlled (2), size (2), missing (2), effect (2), reduction (2), cleaning (2), scaling (2), fisher (2), transform (2), cox (2), dispersion (2), range (2), geometric (2), press (2), encyclopedia (2), external (2), 2nd (2), springer (2), 387 (2), berger (2), 2012 (2), gaussian (2), doi (2), arxiv (2), ieee (2), mathematical (2), print (2), cambridge (2), notes (2), utility (2), only (2), would (2), surpasses (2), vote (2), approaches (2), closer (2), will (2), across (2), while (2), their (2), practical (2), does (2), details (2), becomes (2), small (2), combined (2), assuming (2), independent (2), moreover (2), words (2), take (2), exactly (2), still (2), problem (2), bits (2), reason (2), described (2), get (2), consider (2), efficient (2), between (2), shown (2), left (2), right (2), conditions (2), sequence (2), interested (2), undefined (2), inadmissible (2), difficult (2), discrete (2), every (2), unique (2), therefore (2), having (2), finite (2), calculated (2), estimated (2), assumed (2), next (2), compute (2), law (2), total (2), past (2), derived (2), related (2), sometimes (2), been (2), expression (2), constant (2), especially (2), here (2), well (2), referred (2), resulting (2), cannot (2), define (2), measures (2), occasionally (2), real (2), hat (2), begin (2), geq (2), quantiles (2), chosen (2), how (2), simplicity (2), pareto (2), gamma (2), sequential (2), current (2), taken (2), equivalently (2), nested (2), laplace (2), learn (2), help (2), appearance (2), upload (2), file (2), changes (2), history (2), read (2), create (2), donate (2), menu (2), add, topic, mobile, cookie, statement, developers, code, conduct, legal, safety, contacts, disclaimers, additional, may, apply, site, you, agree, registered, trademark, profit, organization, wikimedia, foundation, inc, creative, attribution, sharealike, license, rendered, parsoid, edited, september, 2026, utc, hidden, retrieved, https, org, php, title, bayes_estimator, oldid, 1376990397, wikiproject, category, kriging, geostatistics, geographic, environmental, cartography, spatial, psychometrics, official, national, accounts, jurimetrics, econometrics, demography, crime, census, actuarial, social, identification, reliability, quality, process, probabilistic, chemometrics, medical, epidemiology, trials, bioinformatics, biostatistics, nelson, aalen, hazard, hitting, accelerated, failure, aft, proportional, hazards, kaplan, meier, whittle, wavelet, fourier, autoregression, var, heteroskedasticity, arch, arima, jenkins, arma, xcf, pacf, autocorrelation, acf, breusch, godfrey, durbin, watson, ljung, johansen, dickey, fuller, granger, causality, break, cointegration, smoothing, trend, decomposition, elliptical, classification, discriminant, canonical, components, cochran, mantel, haenszel, mcnemar, graphical, cohen, kappa, degrees, freedom, ancova, partition, regressions, logistic, bernoulli, families, homoscedasticity, heteroscedasticity, isotonic, semiparametric, nonlinear, predictors, ordinary, template, splines, mars, simultaneous, mixed, effects, residuals, confounding, van, der, waerden, ordered, jonckheere, terpstra, friedman, kruskal, wallis, mann, whitney, hodges, signed, wilcoxon, sign, bic, aic, selection, normality, shapiro, wilk, jarque, bera, lilliefors, anderson, darling, kolmogorov, smirnov, chi, goodness, fit, student, wald, lagrange, multiplier, ratio, multiple, comparisons, randomization, permutation, powerful, tails, testing, hypotheses, jackknife, resampling, tolerance, prediction, pivot, plug, scheffé, rao, blackwellization, frequentist, sensitivity, robustness, asymptotics, divergence, functional, sufficiency, completeness, monotone, space, specification, order, quasi, sectional, cohort, observational, down, stochastic, scientific, assignment, interaction, factorial, blocking, experiments, questionnaire, opinion, poll, stratified, survey, methodology, replication, collection, detrending, differencing, preprocessing, component, dimensionality, truncation, winsorizing, outlier, unit, min, standardization, feature, anscombe, stabilizing, yeo, johnson, transformations, processing, line, ecdf, matrix, heatmap, violin, stem, leaf, display, run, radar, pie, histogram, forest, fan, correlogram, biplot, bar, graphics, spearman, kendall, dependence, grouped, summary, tables, count, skewness, kurtosis, central, percentile, interquartile, variation, absolute, lehmer, heinz, heronian, harmonic, cubic, contraharmonic, center, descriptive, outline, ems, 2001, 1994, pilz, jürgen, 1991, chichester, john, wiley, sons, 117, 471, 91732, experimental, 98502, 1985, york, verlag, 0804611, 96098, james, archived, original, 1980, dytso, bustin, tuninetti, devroye, poor, shamai, shitz, 2018, pth, noise, channels, 2037, 1109, tit, 2017, 2782786, 1607, 01461, transactions, bassett, robert, deride, julio, march, 2019, 144, 1007, s10107, 018, 1241, 1611, 05917, 129, 174, programming, jaynes, 2007, univ, 978, 521, 59271, 172, logic, recursive, ensures, few, godfather, 500, 000, note, just, straight, zero, simpler, fewer, cast, skew, towards, approaching, pure, whole, pool, currently, deemed, necessary, validity, uses, calculating, comparing, users, including, claimed, give, initially, calculate, though, changed, rated, titles, internet, database, compare, exact, depend, difference, matches, match, times, combining, results, plays, role, made, advance, phenomena, normally, being, were, extra, into, hand, relevant, affects, relative, brings, bit, shows, knows, very, little, fine, coincides, possible, interpretation, calculation, pathological, gives, equal, implies, close, denotes, success, according, relations, follows, sqrt, customary, regard, deterministic, whose, values, approximately, negligible, converges, increasing, analyzing, contrast, these, verification, stein, phenomenon, stated, theorems, conclude, finally, obtain, marginal, depends, determine, both, through, enable, auxiliary, development, done, assumption, obtained, performed, improved, identical, except, replaced, must, minimize, equivalent, subjective, typical, type, recall, application, applied, uncommon, valid, means, consequence, longer, meaningful, speak, nevertheless, infty, restrictive, requirement, covering, numbers, equally, likely, yet, sense, seems, like, choice, imply, preference, any, infinite, mass, informative, far, conceived, although, widely, validated, particularly, characterize, closed, they, share, similar, those, certain, assigns, sub, generalization, previous, quantile, tfrac, bias, depending, primarily, due, several, alternatives, denote, overline, variables, useful, unless, complex, added, usually, without, resorting, numerical, inherent, prefer, belonging, important, property, etc, mmse, simply, joint, mathrm, defines, said, among, maximizes, formulating, within, action, predictive, averaging, schwarz, criterion, evaluation, lower, bound, approximate, computation, variational, integrated, approximations, markov, chain, monte, carlo, hierarchical, building, entropy, indifference, cromwell, coherence, bernstein, von, mises, background, part, includes, list, remove, message, please, precise, introducing, improve, lacks, sufficient, corresponding, inline, free, item, projects, printable, version, download, pdf, export, switch, legacy, parser, shortened, url, cite, permanent, link, what, actions, english, talk, tiếng, việt, українська, српски, srpski, русский, 한국어, italiano, فارسی, deutsch, català, personal, special, pages, recent, community, contribute, events, navigation, jump, content,

Text of the page (random words):
nding inline citations please help improve this article by introducing more precise citations november 2009 learn how and when to remove this message part of a series on bayesian statistics posterior likelihood prior evidence background bayesian inference bayesian probability bayes theorem bernstein von mises theorem coherence cox s theorem cromwell s rule likelihood principle principle of indifference principle of maximum entropy model building conjugate prior linear regression empirical bayes hierarchical model posterior approximation markov chain monte carlo laplace s approximation integrated nested laplace approximations variational inference approximate bayesian computation estimators bayes estimator credible interval maximum a posteriori estimation evidence approximation evidence lower bound nested sampling model evaluation bayes factor schwarz criterion model averaging posterior predictive mathematics portal v t e in estimation theory and decision theory a bayes estimator or a bayes action is an estimator or decision rule that minimizes the posterior expected value of a loss function i e the posterior expected loss equivalently it maximizes the posterior expectation of a utility function an alternative way of formulating an estimator within bayesian statistics is maximum a posteriori estimation definition edit suppose an unknown parameter θ displaystyle theta is known to have a prior distribution π displaystyle pi let θ θ x displaystyle widehat theta widehat theta x be an estimator of θ displaystyle theta based on some measurements x and let l θ θ displaystyle l theta widehat theta be a loss function such as squared error the bayes risk of θ displaystyle widehat theta is defined as e π l θ θ displaystyle e_ pi l theta widehat theta where the expectation is taken over the probability distribution of θ displaystyle theta this defines the risk function as a function of θ displaystyle widehat theta an estimator θ displaystyle widehat theta is said to be a bayes estimator if it minimizes the bayes risk among all estimators equivalently the estimator which minimizes the posterior expected loss e l θ θ x displaystyle e l theta widehat theta x for each x displaystyle x also minimizes the bayes risk and therefore is a bayes estimator 1 if the prior is improper then an estimator which minimizes the posterior expected loss for each x displaystyle x is called a generalized bayes estimator 2 examples edit minimum mean square error estimation edit main article minimum mean square error the most common risk function used for bayesian estimation is the mean square error mse also called squared error risk the mse is defined by m s e e θ x θ 2 displaystyle mathrm mse e left widehat theta x theta 2 right where the expectation is taken over the joint distribution of θ displaystyle theta and x displaystyle x posterior mean edit using the mse as risk the bayes estimate of the unknown parameter is simply the mean of the posterior distribution 3 θ x e θ x θ p θ x d θ displaystyle widehat theta x e theta x int theta p theta x d theta this is known as the minimum mean square error mmse estimator bayes estimators for conjugate priors edit main article conjugate prior if there is no inherent reason to prefer one prior probability distribution over another a conjugate prior is sometimes chosen for simplicity a conjugate prior is defined as a prior distribution belonging to some parametric family for which the resulting posterior distribution also belongs to the same family this is an important property since the bayes estimator as well as its statistical properties variance confidence interval etc can all be derived from the posterior distribution conjugate priors are especially useful for sequential estimation where the posterior of the current measurement is used as the prior in the next measurement in sequential estimation unless a conjugate prior is used the posterior distribution typically becomes more complex with each added measurement and the bayes estimator cannot usually be calculated without resorting to numerical methods following are some examples of conjugate priors if x θ displaystyle x theta is normal x θ n θ σ 2 displaystyle x theta sim n theta sigma 2 and the prior is normal θ n μ τ 2 displaystyle theta sim n mu tau 2 then the posterior is also normal and the bayes estimator under mse is given by θ x σ 2 σ 2 τ 2 μ τ 2 σ 2 τ 2 x displaystyle widehat theta x frac sigma 2 sigma 2 tau 2 mu frac tau 2 sigma 2 tau 2 x if x 1 x n displaystyle x_ 1 x_ n are iid poisson random variables x i θ p θ displaystyle x_ i theta sim p theta and if the prior is gamma distributed θ g a b displaystyle theta sim g a b then the posterior is also gamma distributed and the bayes estimator under mse is given by θ x n x a n b displaystyle widehat theta x frac n overline x a n b if x 1 x n displaystyle x_ 1 x_ n are iid uniformly distributed x i θ u 0 θ displaystyle x_ i theta sim u 0 theta and if the prior is pareto distributed θ p a θ 0 a displaystyle theta sim pa theta _ 0 a then the posterior is also pareto distributed and the bayes estimator under mse is given by θ x a n max θ 0 x 1 x n a n 1 displaystyle widehat theta x frac a n max theta _ 0 x_ 1 x_ n a n 1 alternative risk functions edit risk functions are chosen depending on how one measures the distance between the estimate and the unknown parameter the mse is the most common risk function in use primarily due to its simplicity however alternative risk functions are also occasionally used the following are several examples of such alternatives we denote the posterior generalized distribution function by f displaystyle f posterior median and other quantiles edit main article bias of an estimator median unbiased estimators a linear loss function with a 0 displaystyle a 0 which yields the posterior median as the bayes estimate l θ θ a θ θ displaystyle l theta widehat theta a theta widehat theta f θ x x 1 2 displaystyle f widehat theta x x tfrac 1 2 another linear loss function which assigns different weights a b 0 displaystyle a b 0 to over or sub estimation it yields a quantile from the posterior distribution and is a generalization of the previous loss function l θ θ a θ θ for θ θ 0 b θ θ for θ θ 0 displaystyle l theta widehat theta begin cases a theta widehat theta mbox for theta widehat theta geq 0 b theta widehat theta mbox for theta widehat theta 0 end cases f θ x x a a b displaystyle f widehat theta x x frac a a b posterior mode edit the following loss function yields the posterior mode in the limit of k 0 displaystyle k 0 under certain conditions on the posterior 4 l θ θ 0 for θ θ k l for θ θ k displaystyle l theta widehat theta begin cases 0 mbox for theta widehat theta k l mbox for theta widehat theta geq k end cases where l 0 displaystyle l 0 is a constant lp estimators edit one can also consider l p displaystyle l p risk for which the loss is given by l θ θ θ θ p p 0 displaystyle l theta hat theta theta hat theta p p 0 while optimal l p displaystyle l p estimators can be difficult to characterize in closed form they do share many similar properties to those in l 2 displaystyle l 2 case 5 other loss functions can be conceived although the mean squared error is the most widely used and validated other loss functions are used in statistics particularly in robust statistics generalized bayes estimators edit see also admissible decision rule bayes rules and generalized bayes rules the prior distribution p displaystyle p has thus far been assumed to be a true probability distribution in that p θ d θ 1 displaystyle int p theta d theta 1 however occasionally this can be a restrictive requirement for example there is no distribution covering the set r of all real numbers for which every real number is equally likely yet in some sense such a distribution seems like a natural choice for a non informative prior i e a prior distribution which does not imply a preference for any particular value of the unknown parameter one can still define a function p θ 1 displaystyle p theta 1 but this would not be a proper probability distribution since it has infinite mass p θ d θ displaystyle int p theta d theta infty such measures p θ displaystyle p theta which are not probability distributions are referred to as improper priors the use of an improper prior means that the bayes risk is undefined since the prior is not a probability distribution and we cannot take an expectation under it as a consequence it is no longer meaningful to speak of a bayes estimator that minimizes the bayes risk nevertheless in many cases one can define the posterior distribution p θ x p x θ p θ p x θ p θ d θ displaystyle p theta x frac p x theta p theta int p x theta p theta d theta this is a definition and not an application of bayes theorem since bayes theorem can only be applied when all distributions are proper however it is not uncommon for the resulting posterior to be a valid probability distribution in this case the posterior expected loss l θ a p θ x d θ displaystyle int l theta a p theta x d theta is typically well defined and finite recall that for a proper prior the bayes estimator minimizes the posterior expected loss when the prior is improper an estimator which minimizes the posterior expected loss is referred to as a generalized bayes estimator 2 example edit a typical example is estimation of a location parameter with a loss function of the type l a θ displaystyle l a theta here θ displaystyle theta is a location parameter i e p x θ f x θ displaystyle p x theta f x theta it is common to use the improper prior p θ 1 displaystyle p theta 1 in this case especially when no other more subjective information is available this yields p θ x p x θ p θ p x f x θ p x displaystyle p theta x frac p x theta p theta p x frac f x theta p x so the posterior expected loss e l a θ x l a θ p θ x d θ 1 p x l a θ f x θ d θ displaystyle e l a theta x int l a theta p theta x d theta frac 1 p x int l a theta f x theta d theta the generalized bayes estimator is the value a x displaystyle a x that minimizes this expression for a given x displaystyle x this is equivalent to minimizing l a θ f x θ d θ displaystyle int l a theta f x theta d theta for a given x displaystyle x 1 in this case it can be shown that the generalized bayes estimator has the form x a 0 displaystyle x a_ 0 for some constant a 0 displaystyle a_ 0 to see this let a 0 displaystyle a_ 0 be the value minimizing 1 when x 0 displaystyle x 0 then given a different value x 1 displaystyle x_ 1 we must minimize l a θ f x 1 θ d θ l a x 1 θ f θ d θ displaystyle int l a theta f x_ 1 theta d theta int l a x_ 1 theta f theta d theta 2 this is identical to 1 except that a displaystyle a has been replaced by a x 1 displaystyle a x_ 1 thus the expression minimizing is given by a x 1 a 0 displaystyle a x_ 1 a_ 0 so that the optimal estimator has the form a x a 0 x displaystyle a x a_ 0 x empirical bayes estimators edit main article empirical bayes method a bayes estimator derived through the empirical bayes method is called an empirical bayes estimator empirical bayes methods enable the use of auxiliary empirical data from observations of related parameters in the development of a bayes estimator this is done under the assumption that the estimated parameters are obtained from a common prior for example if independent observations of different parameters are performed then the estimation performance of a particular parameter can sometimes be improved by using data from other observations there are both parametric and non parametric approaches to empirical bayes estimation 6 example edit the following is a simple example of parametric empirical bayes estimation given past observations x 1 x n displaystyle x_ 1 ldots x_ n having conditional distribution f x i θ i displaystyle f x_ i theta _ i one is interested in estimating θ n 1 displaystyle theta _ n 1 based on x n 1 displaystyle x_ n 1 assume that the θ i displaystyle theta _ i s have a common prior π displaystyle pi which depends on unknown parameters for example suppose that π displaystyle pi is normal with unknown mean μ π displaystyle mu _ pi and variance σ π displaystyle sigma _ pi we can then use the past observations to determine the mean and variance of π displaystyle pi in the following way first we estimate the mean μ m displaystyle mu _ m and variance σ m displaystyle sigma _ m of the marginal distribution of x 1 x n displaystyle x_ 1 ldots x_ n using the maximum likelihood approach μ m 1 n x i displaystyle widehat mu _ m frac 1 n sum x_ i σ m 2 1 n x i μ m 2 displaystyle widehat sigma _ m 2 frac 1 n sum x_ i widehat mu _ m 2 next we use the law of total expectation to compute μ m displaystyle mu _ m and the law of total variance to compute σ m 2 displaystyle sigma _ m 2 such that μ m e π μ f θ displaystyle mu _ m e_ pi mu _ f theta σ m 2 e π σ f 2 θ e π μ f θ μ m 2 displaystyle sigma _ m 2 e_ pi sigma _ f 2 theta e_ pi mu _ f theta mu _ m 2 where μ f θ displaystyle mu _ f theta and σ f θ displaystyle sigma _ f theta are the moments of the conditional distribution f x i θ i displaystyle f x_ i theta _ i which are assumed to be known in particular suppose that μ f θ θ displaystyle mu _ f theta theta and that σ f 2 θ k displaystyle sigma _ f 2 theta k we then have μ π μ m displaystyle mu _ pi mu _ m σ π 2 σ m 2 σ f 2 σ m 2 k displaystyle sigma _ pi 2 sigma _ m 2 sigma _ f 2 sigma _ m 2 k finally we obtain the estimated moments of the prior μ π μ m displaystyle widehat mu _ pi widehat mu _ m σ π 2 σ m 2 k displaystyle widehat sigma _ pi 2 widehat sigma _ m 2 k for example if x i θ i n θ i 1 displaystyle x_ i theta _ i sim n theta _ i 1 and if we assume a normal prior which is a conjugate prior in this case we conclude that θ n 1 n μ π σ π 2 displaystyle theta _ n 1 sim n widehat mu _ pi widehat sigma _ pi 2 from which the bayes estimator of θ n 1 displaystyle theta _ n 1 based on x n 1 displaystyle x_ n 1 can be calculated properties edit admissibility edit see also admissible decision rule bayes rules having finite bayes risk are typically admissible the following are some specific examples of admissibility theorems if a bayes rule is unique then it is admissible 7 for example as stated above under mean squared error mse the bayes rule is unique and therefore admissible if θ belongs to a discrete set then all bayes rules are admissible if θ belongs to a continuous non discrete set and if the risk function r θ δ is continuous in θ for every δ then all bayes rules are admissible by contrast generalized bayes rules often have undefined bayes risk in the case of improper priors these rules are often inadmissible and the verification of their admissibility can be difficult for example the generalized bayes estimator of a location parameter θ based on gaussian samples described in the generalized bayes estimator section above is inadmissible for p 2 displaystyle p 2 this is known as stein s phenomenon asymptotic efficiency e...