Meta tags:

Headings (most frequently used words):

properties, error, consistency, estimator, contents, discussion, definition, quantified, behavioral, see, also, references, further, reading, external, links, mean, squared, sampling, deviation, variance, bias, unbiasedness, relationships, among, the, quantities, example, fisher, asymptotic, normality, efficiency, robustness,

Text of the page (most frequently used words):
the (327), displaystyle (130), theta (119), #estimator (117), widehat (63), and (59), variance (36), for (34), operatorname (34), unbiased (33), mean (31), are (30), error (29), distribution (29), with (28), that (27), mathrm (26), edit (25), estimators (24), bias (24), parameter (22), estimates (22), value (21), can (21), estimate (21), mse (18), var (17), bar (17), also (16), sample (16), properties (15), not (15), arrows (15), squared (14), used (14), function (14), there (13), this (12), estimation (12), biased (12), true (12), from (11), data (11), right (11), where (11), random (11), statistics (10), may (10), theory (10), point (10), probability (10), between (10), being (10), hat (10), then (10), target (10), variable (9), square (9), low (9), left (9), have (9), asymptotic (9), consistency (9), sigma (9), see (8), which (8), good (8), two (8), efficiency (8), example (8), only (8), normal (8), all (8), consistent (8), deviation (8), sampling (8), color (8), rgb (8), cdot (8), average (8), wikipedia (7), main (7), article (7), called (7), bad (7), relatively (7), would (7), high (7), single (7), means (7), under (6), isbn (6), often (6), will (6), than (6), following (6), aligned (6), quantity (6), particular (6), limit (6), standard (6), defined (6), estimated (6), given (6), 17254901960784313 (6), frac (6), even (6), expected (6), using (5), page (5), springer (5), statistical (5), theorem (5), smaller (5), relationship (5), interest (5), zero (5), converges (5), same (5), number (5), but (5), bull (5), eye (5), toggle (4), contents (4), search (4), different (4), links (4), 387 (4), 978 (4), empirical (4), minimum (4), best (4), such (4), bound (4), among (4), one (4), some (4), absolute (4), curve (4), graph (4), more (4), frequency (4), formula (4), based (4), normality (4), asymptotically (4), however (4), when (4), note (4), around (4), size (4), fisher (4), shown (4), infinite (4), samples (4), density (4), quantified (4), model (4), over (4), quantities (4), thus (4), looking (4), unbiasedness (4), valued (4), clustered (4), collection (4), observed (4), space (4), functions (4), interval (4), rule (4), parametric (4), hide (4), move (4), sidebar (4), view (3), about (3), terms (3), use (3), casella (3), press (3), further (3), filter (3), method (3), maximum (3), robust (3), rao (3), exists (3), having (3), lower (3), efficient (3), comparison (3), while (3), large (3), these (3), difference (3), equation (3), term (3), represents (3), identified (3), suppose (3), above (3), begin (3), end (3), any (3), conditions (3), central (3), centered (3), check (3), sequence (3), how (3), usually (3), behavioral (3), exact (3), green (3), values (3), its (3), 6274509803921569 (3), 4980392156862745 (3), 054901960784313725 (3), consider (3), below (3), too (3), should (3), type (3), 1ex (3), median (3), they (3), does (3), dispersed (3), distance (3), far (3), still (3), definition (3), dimensional (3), tools (3), languages (2), table (2), contact (2), privacy (2), policy (2), non (2), was (2), short (2), description (2), wikidata (2), retrieved (2), index (2), external (2), 1998 (2), 2nd (2), lehmann (2), kosorok (2), 2008 (2), series (2), 74978 (2), introduction (2), inference (2), 2007 (2), university (2), jaynes (2), berger (2), 2002 (2), encyclopedia (2), reading (2), pdf (2), section (2), tukey (2), analysis (2), references (2), statistic (2), moments (2), markov (2), measure (2), blue (2), information (2), robustness (2), cramér (2), lowest (2), mvue (2), simply (2), has (2), through (2), 2cm (2), better (2), whether (2), general (2), both (2), loss (2), weak (2), likelihood (2), found (2), convergence (2), necessarily (2), finite (2), whose (2), grows (2), denote (2), sqrt (2), xrightarrow (2), long (2), functional (2), easy (2), divided (2), occurs (2), scale (2), small (2), lim (2), varepsilon (2), without (2), other (2), words (2), population (2), case (2), original (2), histogram (2), known (2), along (2), 2ex (2), 12156862745098039 (2), 4666666666666667 (2), 7058823529411765 (2), cdots (2), sum (2), fitting (2), performance (2), related (2), relationships (2), because (2), obtain (2), produce (2), satisfy (2), equations (2), overline (2), expectation (2), preferred (2), property (2), tendency (2), larger (2), variances (2), problem (2), starchy (2), leaves (2), express (2), version (2), half (2), people (2), their (2), errors (2), yet (2), position (2), off (2), indicate (2), grossly (2), miss (2), hit (2), depends (2), highly (2), fixed (2), denoted (2), symbol (2), corresponding (2), itself (2), variables (2), estimating (2), problems (2), decision (2), unknown (2), sometimes (2), models (2), semi (2), estimand (2), discussion (2), rules (2), hold (2), result (2), calculating (2), appearance (2), upload (2), file (2), changes (2), history (2), read (2), english (2), norsk (2), subsection (2), log (2), create (2), account (2), donate (2), menu (2), add, topic, mobile, cookie, statement, developers, code, conduct, legal, safety, contacts, disclaimers, text, available, additional, apply, site, you, agree, registered, trademark, profit, organization, wikimedia, foundation, inc, creative, commons, attribution, sharealike, license, rendered, parsoid, last, edited, june, 2026, utc, hidden, categories, articles, category, https, org, php, title, oldid, 1359131939, fundamentals, shao, jun, 98674, mathematical, 98502, michael, 1007, doi, processes, semiparametric, 521, 59271, cambridge, logic, science, george, roger, duxbury, bol, shev, login, nikolaevich, 2001, 1994, ems, mathematics, 471, lauritzen, steffen, oxford, 2023, december, dekking, frederik, michel, kraaikamp, cornelis, lopuhaä, hendrik, paul, meester, ludolf, erwin, 2005, texts, 85233, 896, modern, 172, mosteller, 1987, 1968, vol, crc, 601, 720, 633, via, google, books, 534, 05101, collected, works, john, philosophy, principles, 1965, 1986, including, well, behaved, wiener, testimator, state, observer, signal, processing, shrinkage, sensitivity, specificity, pitman, closeness, criterion, particle, mmse, generalized, map, posteriori, mcmc, chain, monte, carlo, kalman, invariant, linear, regression, concerning, blackwell, scheffé, gauss, cases, addition, satisfies, put, narrow, plotting, curves, shared, axis, becomes, obvious, gentle, relates, first, second, third, quality, expressed, formulas, besides, identify, center, sides, manner, reality, explicit, choice, reflected, naturally, desirable, minimal, cannot, satisfied, simultaneously, mild, implies, generally, fairly, regularity, simplest, examples, lies, boundary, allowable, region, asymptotics, formulation, authors, call, occurred, therefore, approximation, weakly, dirac, delta, approaches, shrinking, proportion, considered, theoretical, respectively, confirm, ssd, made, into, multiplying, namely, frequently, measures, dispersion, parameters, factor, multiple, mathematically, matter, almost, surely, strongly, infty, leq, mathbb, increasing, increases, close, converge, red, 20000, ensures, associated, picture, blacksquare, bullet, phi, follows, except, verify, relation, illustrated, taking, bienaymé, 5ex, qquad, underbrace, cov, leqslant, degenerate, sim, mathcal, tradeoff, complexity, mainly, field, diagnose, algorithms, predictive, modelling, supervised, learning, root, equals, dividing, negative, mentioned, though, reverse, similarly, find, follow, solving, previous, figure, despite, distributions, overlapped, were, actually, desired, trait, systematic, additionally, closer, smallest, practical, always, genetic, states, leaf, modeled, show, np_, bin, alternative, agrees, run, applies, immediately, scalar, extended, refer, really, talking, confuse, fact, astronomical, preclude, instance, ideal, situation, try, extreme, few, essential, just, little, permitted, fewer, outlier, outliers, since, analogous, accuracy, precision, neq, kinds, saying, every, deviations, cluster, diffuse, finally, nevertheless, weighted, analogy, process, shooting, individual, likely, procedure, definitions, attributes, relevant, needs, maps, set, convenient, treated, symbolised, abbreviated, notation, interpreted, directly, cause, confusion, algebra, arises, applications, firstly, secondly, thought, time, spectral, word, qualifier, refers, another, subsets, places, virtually, restrictions, attractiveness, judged, etc, construction, subjects, context, evaluated, infer, common, way, phrasing, selected, either, traditionally, written, adding, realization, interchangeably, circumflex, concerned, defining, compare, creating, determine, circumstances, goes, balance, tightly, assumptions, worse, wider, yield, results, contrast, range, plausible, includes, vector, distinguished, commonly, free, item, projects, printable, download, print, export, switch, legacy, parser, get, shortened, url, cite, permanent, link, what, here, actions, talk, tiếng, việt, українська, svenska, sunda, simple, русский, português, polski, bokmål, nynorsk, nederlands, 한국어, қазақша, 日本語, italiano, հայերեն, galego, français, suomi, فارسی, euskara, español, esperanto, ελληνικά, deutsch, čeština, català, বাংলা, العربية, top, personal, special, pages, recent, community, portal, learn, help, contribute, current, events, navigation, jump, content,

Text of the page (random words):
parametric models 2 if the parameter is denoted θ displaystyle theta then the estimator is traditionally written by adding a circumflex over the symbol θ displaystyle widehat theta being a function of the data the estimator is itself a random variable a particular realization of this random variable is called the estimate sometimes the words estimator and estimate are used interchangeably the definition places virtually no restrictions on which functions of the data can be called the estimators the attractiveness of different estimators can be judged by looking at their properties such as unbiasedness mean square error consistency asymptotic distribution etc the construction and comparison of estimators are the subjects of the estimation theory in the context of decision theory an estimator is a type of decision rule and its performance may be evaluated through the use of loss functions when the word estimator is used without a qualifier it usually refers to point estimation the estimate in this case is a single point in the parameter space there also exists another type of estimator interval estimators where the estimates are subsets of the parameter space the problem of density estimation arises in two applications firstly in estimating the probability density functions of random variables and secondly in estimating the spectral density function of a time series in these problems the estimates are functions that can be thought of as point estimates in an infinite dimensional space and there are corresponding interval estimation problems definition edit suppose a fixed parameter θ displaystyle theta needs to be estimated then an estimator is a function that maps the sample space to a set of sample estimates an estimator of θ displaystyle theta is usually denoted by the symbol θ displaystyle widehat theta it is often convenient to express the theory using the algebra of random variables thus if x displaystyle x is used to denote a random variable corresponding to the observed data the estimator itself treated as a random variable is symbolised as a function of that random variable θ x displaystyle widehat theta x the estimate for a particular observed data value x displaystyle x i e for x x displaystyle x x is then θ x displaystyle widehat theta x which is a fixed value often an abbreviated notation is used in which θ displaystyle widehat theta is interpreted directly as a random variable but this can cause confusion quantified properties edit the following definitions and attributes are relevant 3 error edit for a given sample x displaystyle x the error of the estimator θ displaystyle widehat theta is defined as e x θ x θ displaystyle e x widehat theta x theta where θ displaystyle theta is the parameter being estimated the error e depends not only on the estimator the estimation formula or procedure but also on the sample mean squared error edit the mean squared error of θ displaystyle widehat theta is defined as the expected value probability weighted average over all samples of the squared errors that is mse θ e θ x θ 2 displaystyle operatorname mse widehat theta operatorname e widehat theta x theta 2 it is used to indicate how far on average the collection of estimates are from the single parameter being estimated consider the following analogy suppose the parameter is the bull s eye of a target the estimator is the process of shooting arrows at the target and the individual arrows are estimates samples then high mse means the average distance of the arrows from the bull s eye is high and low mse means the average distance from the bull s eye is low the arrows may or may not be clustered for example even if all arrows hit the same point yet grossly miss the target the mse is still relatively large however if the mse is relatively low then the arrows are likely more highly clustered than highly dispersed around the target sampling deviation edit for a given sample x displaystyle x the sampling deviation of the estimator θ displaystyle widehat theta is defined as d x θ x e θ x θ x e θ displaystyle d x widehat theta x operatorname e widehat theta x widehat theta x operatorname e widehat theta where e θ x displaystyle operatorname e widehat theta x is the expected value of the estimator the sampling deviation d displaystyle d depends not only on the estimator but also on the sample variance edit the variance of θ displaystyle widehat theta is the expected value of the squared sampling deviations that is var θ e θ e θ 2 displaystyle operatorname var widehat theta operatorname e widehat theta operatorname e widehat theta 2 it is used to indicate how far on average the collection of estimates are from the expected value of the estimates note the difference between mse and variance if the parameter is the bull s eye of a target and the arrows are estimates then a relatively high variance means the arrows are dispersed and a relatively low variance means the arrows are clustered even if the variance is low the cluster of arrows may still be far off target and even if the variance is high the diffuse collection of arrows may still be unbiased finally even if all arrows grossly miss the target if they nevertheless all hit the same point the variance is zero bias edit the bias of θ displaystyle widehat theta is defined as b θ e θ θ displaystyle b widehat theta operatorname e widehat theta theta it is the distance between the average of the collection of estimates and the single parameter being estimated the bias of θ displaystyle widehat theta is a function of the true value of θ displaystyle theta so saying that the bias of θ displaystyle widehat theta is b displaystyle b means that for every θ displaystyle theta the bias of θ displaystyle widehat theta is b displaystyle b there are two kinds of estimators biased estimators and unbiased estimators whether an estimator is biased or not can be identified by the relationship between e θ θ displaystyle operatorname e widehat theta theta and 0 if e θ θ 0 displaystyle operatorname e widehat theta theta neq 0 θ displaystyle widehat theta is biased if e θ θ 0 displaystyle operatorname e widehat theta theta 0 θ displaystyle widehat theta is unbiased the bias is also the expected value of the error since e θ θ e θ θ displaystyle operatorname e widehat theta theta operatorname e widehat theta theta if the parameter is the bull s eye of a target and the arrows are estimates then a relatively high absolute value for the bias means the average position of the arrows is off target and a relatively low absolute bias means the average position of the arrows is on target they may be dispersed or may be clustered the relationship between bias and variance is analogous to the relationship between accuracy and precision the estimator θ displaystyle widehat theta is an unbiased estimator of θ displaystyle theta if and only if b θ 0 displaystyle b widehat theta 0 bias is a property of the estimator not of the estimate often people refer to a biased estimate or an unbiased estimate but they really are talking about an estimate from a biased estimator or an estimate from an unbiased estimator also people often confuse the error of a single estimate with the bias of an estimator that the error for one estimate is large does not mean the estimator is biased in fact even if all estimates have astronomical absolute values for their errors if the expected value of the error is zero the estimator is unbiased also an estimator s being biased does not preclude the error of an estimate from being zero in a particular instance the ideal situation is to have an unbiased estimator with low variance and also try to limit the number of samples where the error is extreme that is to have few outliers yet unbiasedness is not essential often if just a little bias is permitted then an estimator can be found with lower mean squared error and or fewer outlier sample estimates an alternative to the version of unbiased above is median unbiased where the median of the distribution of estimates agrees with the true value thus in the long run half the estimates will be too low and half too high while this applies immediately only to scalar valued estimators it can be extended to any measure of central tendency of a distribution see median unbiased estimators in a practical problem θ displaystyle widehat theta can always have functional relationship with θ displaystyle theta for example if a genetic theory states there is a type of leaf starchy green that occurs with probability p 1 1 4 θ 2 displaystyle p_ 1 1 4 cdot theta 2 with 0 θ 1 displaystyle 0 theta 1 then for n displaystyle n leaves the random variable n 1 displaystyle n_ 1 or the number of starchy green leaves can be modeled with a b i n n p 1 displaystyle mathrm bin n p_ 1 distribution the number can be used to express the following estimator for θ displaystyle theta θ 4 n n 1 2 displaystyle widehat theta 4 n cdot n_ 1 2 one can show that θ displaystyle widehat theta is an unbiased estimator for θ displaystyle theta e θ e 4 n n 1 2 4 n e n 1 2 4 n n p 1 2 4 p 1 2 4 1 4 θ 2 2 θ 2 2 θ displaystyle begin aligned operatorname e widehat theta operatorname e 4 n cdot n_ 1 2 4 n cdot operatorname e n_ 1 2 1ex 4 n cdot np_ 1 2 4 cdot p_ 1 2 1ex 4 cdot 1 4 cdot theta 2 2 theta 2 2 1ex theta end aligned unbiasedness edit difference between estimators an unbiased estimator θ 2 displaystyle theta _ 2 is centered around θ displaystyle theta vs a biased estimator θ 1 displaystyle theta _ 1 a desired property for estimators is the unbiased trait where an estimator is shown to have no systematic tendency to produce estimates larger or smaller than the true parameter additionally unbiased estimators with smaller variances are preferred over larger variances because it will be closer to the true value of the parameter the unbiased estimator with the smallest variance is known as the minimum variance unbiased estimator mvue to find if an estimator θ displaystyle widehat theta is unbiased it is easy to follow along the equation e θ θ 0 displaystyle operatorname e widehat theta theta 0 with estimator t with and parameter of interest θ displaystyle theta solving the previous equation so it is shown as e t θ displaystyle operatorname e t theta the estimator is unbiased looking at the figure to the right despite θ 2 displaystyle hat theta _ 2 being the only unbiased estimator if the distributions overlapped and were both centered around θ displaystyle theta then distribution θ 1 displaystyle hat theta _ 1 would actually be the preferred unbiased estimator expectation when looking at quantities in the interest of expectation for the model distribution there is an unbiased estimator which should satisfy the two equations below x n 1 n x 1 x 2 x n displaystyle overline x _ n frac 1 n left x_ 1 x_ 2 cdots x_ n right e1 e x n μ displaystyle operatorname e left overline x _ n right mu e2 variance similarly when looking at quantities in the interest of variance as the model distribution there is also an unbiased estimator that should satisfy the two equations below s n 2 1 n 1 i 1 n x i x 2 displaystyle s_ n 2 frac 1 n 1 sum _ i 1 n left x_ i bar x right 2 v1 e s n 2 σ 2 displaystyle operatorname e left s_ n 2 right sigma 2 v2 note we are dividing by n 1 because if we divided with n we would obtain an estimator with a negative bias which would thus produce estimates that are too small for σ 2 displaystyle sigma 2 it should also be mentioned that even though s n 2 displaystyle s_ n 2 is unbiased for σ 2 displaystyle sigma 2 the reverse is not true 4 relationships among the quantities edit the mean squared error variance and bias are related mse θ var θ b θ 2 displaystyle operatorname mse widehat theta operatorname var widehat theta b widehat theta 2 i e mean squared error variance square of bias in particular for an unbiased estimator the variance equals the mean squared error the standard deviation of an estimator θ displaystyle widehat theta of θ displaystyle theta the square root of the variance or an estimate of the standard deviation of an estimator θ displaystyle widehat theta of θ displaystyle theta is called the standard error of θ displaystyle widehat theta the bias variance tradeoff will be used in model complexity over fitting and under fitting it is mainly used in the field of supervised learning and predictive modelling to diagnose the performance of algorithms example edit consider a random variable following a normal probability distribution x n μ σ 2 displaystyle x sim mathcal n mu sigma 2 and a biased estimator of the mean μ θ displaystyle mu theta of that distribution θ x x n b 1 n i 1 n x i b displaystyle hat theta x bar x _ n b frac 1 n sum _ i 1 n x_ i b where b displaystyle b follows a degenerate distribution i e p b b 1 displaystyle p b b 1 such that e θ x e x n e b μ b displaystyle mathrm e hat theta x mathrm e bar x _ n mathrm e b mu b b θ x b displaystyle mathrm b hat theta x b v a r θ x e x n b μ b 2 e x n μ 2 e b b 2 2 e x n μ b b v a r x n μ e x n μ 2 v a r b b e b b 2 2 c o v x n μ b b 2 v a r x n μ v a r b b 0 2 e x n μ e b b σ 2 n displaystyle begin aligned mathrm var hat theta x mathrm e bar x _ n b mu b 2 2ex color rgb 0 12156862745098039 0 4666666666666667 0 7058823529411765 mathrm e bar x _ n mu 2 color rgb 1 0 4980392156862745 0 054901960784313725 mathrm e b b 2 color rgb 0 17254901960784313 0 6274509803921569 0 17254901960784313 2 mathrm e bar x _ n mu b b 2ex color rgb 0 12156862745098039 0 4666666666666667 0 7058823529411765 mathrm var bar x _ n mu mathrm e bar x _ n mu 2 color rgb 1 0 4980392156862745 0 054901960784313725 mathrm var b b mathrm e b b 2 0 5ex qquad color rgb 0 17254901960784313 0 6274509803921569 0 17254901960784313 2 underbrace mathrm cov bar x _ n mu b b _ cdots 2 leqslant mathrm var bar x _ n mu mathrm var b b 0 2 mathrm e bar x _ n mu mathrm e b b frac sigma 2 n end aligned where all the terms are zero except v a r x n μ v a r x n σ 2 n displaystyle mathrm var bar x _ n mu mathrm var bar x _ n frac sigma 2 n using the bienaymé formula and m s e θ x e x n b μ 2 e x n μ 2 e b 2 σ 2 n b 2 displaystyle mathrm mse hat theta x mathrm e bar x _ n b mu 2 mathrm e bar x _ n mu 2 mathrm e b 2 frac sigma 2 n b 2 we verify the relation between the mean square error the variance and the bias below are illustrated the quantified properties of the estimation of the probability distribution mean taking μ 0 displaystyle mu 0 σ 1 displaystyle sigma 1 and b 1 2 displaystyle b 1 2 probability density function ϕ displaystyle phi of the standard normal distribution blue with a sample x i n displaystyle x_ i _ n of n 10 displaystyle n 10 values displaystyle color rgb 1 0 4980392156862745 0 054901960784313725 bullet and the associated estimate θ x i n displaystyle hat theta x_ i _ n displaystyle color rgb 0 17254901960784313 0 6274509803921569 0 17254901960784313 blacksquare the mean θ 0 displaystyle theta 0 of the original distribution and the mean e θ 1...