Meta tags:

Headings (most frequently used words):

of, the, vector, distribution, distributions, and, entropy, parameter, unknown, estimation, example, exponential, family, examples, cumulants, sufficient, statistic, formulation, normal, mean, variance, contents, nomenclature, difficulty, definition, interpretation, properties, table, moments, role, in, statistics, see, also, footnotes, references, further, reading, external, links, scalar, factorization, variables, involved, variable, measure, theoretic, known, binomial, normalization, moment, generating, function, relative, maximum, derivation, classical, sufficiency, bayesian, conjugate, unbiased, hypothesis, testing, uniformly, most, powerful, tests, generalized, linear, models, citations, sources, support, must, be, independent, valued, canonical, differential, identities, for,

Text of the page (most frequently used words):
the (541), displaystyle (458), eta (417), log (405), boldsymbol (190), theta (165), frac (162), right (150), left (149), distribution (145), bmatrix (142), #exponential (140), and (138), begin (119), end (118), aligned (94), mathbf (94), family (91), with (76), exp (74), are (72), gamma (71), that (71), parameter (70), for (70), function (67), this (65), distributions (60), families (55), sum (49), can (49), edit (45), alpha (45), partial (44), sigma (43), beta (42), sufficient (42), parameters (42), statistics (41), natural (40), normal (40), form (40), statistic (37), probability (36), chi (33), 1ex (33), mathsf (32), tfrac (32), not (31), limits (31), data (30), mid (30), vector (29), where (29), one (28), number (28), lambda (27), cdot (27), dfrac (27), textstyle (27), binomial (26), vdots (26), prior (25), all (24), example (23), statistical (22), which (21), int (21), 5pt (21), using (20), variance (20), then (20), conjugate (20), from (19), entropy (19), value (19), operatorname (19), discrete (18), when (18), inverse (17), generalized (17), variable (17), posterior (17), mean (17), known (17), examples (17), functions (17), above (17), case (17), model (16), has (16), only (16), fixed (16), terms (15), continuous (15), analysis (15), measure (15), written (15), same (15), variant (15), multivariate (14), support (14), density (14), partition (14), moments (14), more (14), matrix (13), logistic (13), trials (13), test (13), likelihood (13), regression (13), divergence (13), normalization (13), other (13), have (13), unknown (13), scalar (13), sqrt (13), estimation (12), random (12), ldots (12), set (12), must (12), poisson (11), valued (11), negative (11), gaussian (11), models (11), general (11), factor (11), bayesian (11), these (11), often (11), defined (11), however (11), psi (11), table (10), non (10), maximum (10), information (10), first (10), product (10), canonical (10), most (10), some (10), curved (10), used (10), see (10), many (10), over (10), does (10), equivalently (10), given (10), respect (10), any (10), 2pt (10), wishart (9), multinomial (9), whose (9), bernoulli (9), categorical (9), linear (9), sample (9), isbn (9), but (9), bregman (9), there (9), dimension (9), two (9), constant (9), both (9), independent (9), values (9), toggle (8), citations (8), uniform (8), dirichlet (8), weibull (8), squared (8), type (8), interval (8), correlation (8), standard (8), simple (8), minimum (8), theory (8), plot (8), doi (8), mathematical (8), article (8), also (8), because (8), cases (8), important (8), single (8), expression (8), following (8), properties (8), cumulants (8), wikipedia (7), use (7), articles (7), pareto (7), geometric (7), real (7), estimator (7), time (7), equations (7), moment (7), way (7), parametric (7), shape (7), size (7), transformation (7), chart (7), nielsen (7), relative (7), forms (7), prod (7), definition (7), normalized (7), consider (7), reference (7), normalizer (7), fact (7), called (7), generating (7), below (7), exponent (7), subsection (7), may (6), page (6), last (6), 2010 (6), different (6), references (6), wrapped (6), mixture (6), kaniadakis (6), student (6), power (6), components (6), inference (6), uniformly (6), unbiased (6), theorem (6), space (6), further (6), common (6), follows (6), will (6), arbitrary (6), another (6), either (6), variables (6), such (6), even (6), bracket (6), each (6), big (6), links (5), location (5), sources (5), von (5), mises (5), laplace (5), cauchy (5), univariate (5), rank (5), domain (5), var (5), tests (5), alternative (5), powerful (5), variational (5), uses (5), pdf (5), how (5), american (5), koopman (5), darmois (5), class (5), shows (5), update (5), simply (5), seen (5), into (5), its (5), observations (5), distributed (5), vary (5), need (5), what (5), expectation (5), formulas (5), integration (5), would (5), useful (5), since (5), sometimes (5), iverson (5), 4pt (5), 10pt (5), possible (5), contents (4), search (4), about (4), missing (4), scale (4), fisher (4), mixed (4), noncentral (4), supported (4), line (4), logit (4), limit (4), least (4), box (4), cross (4), series (4), bayes (4), lagrange (4), testing (4), sufficiency (4), order (4), experiment (4), 387 (4), 978 (4), arxiv (4), convex (4), jstor (4), pitman (4), shown (4), computed (4), involves (4), particular (4), than (4), needed (4), sets (4), thus (4), very (4), cannot (4), determined (4), multiplied (4), derivation (4), now (4), equal (4), chosen (4), derivatives (4), generally (4), writing (4), related (4), between (4), derivative (4), closed (4), reverse (4), identities (4), similar (4), equiv (4), variants (4), resulting (4), constraint (4), equivalent (4), tau (4), vec (4), note (4), 15pt (4), binom (4), failures (4), main (4), mass (4), defining (4), allowed (4), always (4), formulation (4), cdots (4), said (4), hide (4), move (4), sidebar (4), view (3), statement (3), text (3), additional (3), commons (3), cs1 (3), unsourced (3), 2011 (3), types (3), org (3), index (3), pearson (3), circular (3), bivariate (3), stable (3), varies (3), maxwell (3), 2nd (3), kind (3), infinite (3), finite (3), population (3), control (3), design (3), survival (3), squares (3), specific (3), adaptive (3), lehmann (3), median (3), ratio (3), estimating (3), point (3), sampling (3), study (3), scatter (3), dispersion (3), range (3), link (3), based (3), frank (3), barndorff (3), wiley (3), learn (3), help (3), corresponding (3), journal (3), association (3), formula (3), oclc (3), 1936 (3), society (3), well (3), include (3), decreasing (3), provided (3), exists (3), hypothesis (3), points (3), various (3), been (3), expressed (3), while (3), priors (3), kernel (3), propto (3), assume (3), represented (3), here (3), corresponds (3), automatically (3), mathbb (3), bounds (3), less (3), identically (3), nor (3), being (3), role (3), associated (3), absolutely (3), read (3), dual (3), second (3), parallel (3), calculate (3), difficult (3), listed (3), part (3), therefore (3), differentiate (3), identity (3), substitutions (3), again (3), step (3), dimensional (3), cumulant (3), generalization (3), true (3), dot (3), why (3), discussion (3), means (3), node (3), hence (3), directly (3), lebesgue (3), cumulative (3), involved (3), factorized (3), factors (3), observed (3), special (3), loosely (3), referred (3), tools (3), languages (2), contact (2), privacy (2), policy (2), apply (2), was (2), september (2), 2026 (2), categories (2), template (2), lacking (2), november (2), maint (2), publisher (2), statements (2), french (2), short (2), description (2), wikidata (2), needing (2), january (2), 2025 (2), retrieved (2), https (2), category (2), tweedie (2), elliptical (2), compound (2), singular (2), delta (2), degenerate (2), lévy (2), asymmetric (2), directional (2), joint (2), shifted (2), erlang (2), skew (2), johnson (2), hyperbolic (2), gumbel (2), wigner (2), phase (2), kolmogorov (2), watson (2), half (2), gompertz (2), logarithmic (2), scaled (2), benktander (2), reciprocal (2), hall (2), bounded (2), extended (2), zipf (2), hypergeometric (2), list (2), mathematics (2), portal (2), system (2), econometrics (2), methods (2), engineering (2), studies (2), clinical (2), applications (2), spectral (2), frequency (2), autoregressive (2), structural (2), seasonal (2), adjustment (2), stationarity (2), equation (2), cluster (2), principal (2), manova (2), graphical (2), contingency (2), anova (2), nonparametric (2), ordinary (2), validation (2), coefficient (2), determination (2), van (2), selection (2), score (2), multiplier (2), multiple (2), hypotheses (2), bootstrap (2), distance (2), method (2), optimal (2), decision (2), functional (2), monotone (2), designs (2), approximation (2), trial (2), randomized (2), controlled (2), effect (2), collection (2), reduction (2), cleaning (2), scaling (2), transform (2), processing (2), dependence (2), deviation (2), arithmetic (2), archived (2), 2013 (2), words (2), external (2), casella (2), keener (2), robert (2), springer (2), theoretical (2), topics (2), core (2), course (2), reading (2), nock (2), entropies (2), garcia (2), vincent (2), 2009 (2), digest (2), flash (2), cards (2), bibcode (2), 0911 (2), 4863 (2), ole (2), john (2), sons (2), ltd (2), 1978 (2), includes (2), remove (2), message (2), please (2), improve (2), physics (2), 2604 (2), 22752 (2), divergences (2), s2cid (2), 1017 (2), cite (2), 1989758 (2), 2307 (2), 1935 (2), 2284291 (2), themselves (2), footnotes (2), encompasses (2), commonly (2), sim (2), products (2), exactly (2), match (2), produces (2), specifically (2), actual (2), termed (2), grow (2), compute (2), observation (2), derive (2), belong (2), normalised (2), makes (2), particularly (2), similarly (2), convenient (2), say (2), pseudo (2), contribute (2), remaining (2), serves (2), qquad (2), conditions (2), namely (2), suppose (2), their (2), increases (2), neither (2), obtained (2), among (2), classical (2), constraints (2), multipliers (2), calculation (2), expected (2), out (2), question (2), conventionally (2), equals (2), computing (2), much (2), latter (2), expectations (2), involving (2), hphantom (2), digamma (2), symmetric (2), calculus (2), facilitate (2), differentiation (2), substitution (2), extremely (2), rewritten (2), making (2), work (2), just (2), find (2), holds (2), higher (2), cov (2), differential (2), normalizing (2), actually (2), rather (2), they (2), technically (2), make (2), setting (2), effectively (2), causes (2), usual (2), entire (2), nonidentifiable (2), adding (2), added (2), relation (2), three (2), due (2), notation (2), like (2), softmax (2), 2k_ (2), logarithm (2), mathrm (2), base (2), mapping (2), easily (2), differentiating (2), convert (2), quad (2), representation (2), result (2), compounding (2), previous (2), mentioned (2), varying (2), bound (2), critical (2), section (2), subset (2), approximating (2), large (2), networks (2), property (2), regarding (2), definitions (2), interpretation (2), alternatively (2), write (2), stieltjes (2), member (2), integrals (2), theoretic (2), place (2), once (2), factorizes (2), inside (2), required (2), factorization (2), free (2), importantly (2), certain (2), nomenclature (2), difficulty (2), term (2), appearance (2), upload (2), file (2), changes (2), history (2), create (2), account (2), donate (2), menu (2), add, topic, mobile, cookie, developers, code, conduct, legal, safety, contacts, disclaimers, available, under, site, you, agree, registered, trademark, profit, organization, wikimedia, foundation, inc, creative, attribution, sharealike, license, rendered, parsoid, edited, utc, hidden, webarchive, archiveis, june, language, exponentials, php, title, exponential_family, oldid, 1376458876, cantor, dirac, bingham, toroidal, kent, spherical, stiefel, manifold, complex, lkj, ewens, rectified, tukey, marchenko, pastur, extreme, voigt, tracy, widom, slash, landau, secant, holtsmark, whole, wilks, truncated, rice, relativistic, breit, rayleigh, poly, nakagami, mittag, leffler, jüttner, boltzmann, lomax, hartman, hotelling, fréchet, folded, hypoexponential, hyperexponential, hyper, davis, dagum, burr, prime, 1st, benini, semi, semicircle, quadratic, triangular, raised, cosine, pert, kumaraswamy, irwin, rectangular, bates, balding, nichols, argus, arcsine, zeta, yule, simon, skellam, parabolic, fractal, panjer, gauss, kuzmin, flory, schulz, delaporte, conway, borel, mandelbrot, soliton, rademacher, benford, wikiproject, kriging, geostatistics, geographic, environmental, cartography, spatial, psychometrics, official, national, accounts, jurimetrics, demography, crime, census, actuarial, science, social, identification, reliability, quality, process, probabilistic, chemometrics, medical, epidemiology, bioinformatics, biostatistics, nelson, aalen, hazard, hitting, accelerated, failure, aft, proportional, hazards, kaplan, meier, whittle, wavelet, fourier, autoregression, conditional, heteroskedasticity, arch, arima, jenkins, arma, xcf, pacf, autocorrelation, acf, breusch, godfrey, durbin, ljung, johansen, dickey, fuller, granger, causality, break, cointegration, smoothing, trend, decomposition, classification, discriminant, cochran, mantel, haenszel, mcnemar, cohen, kappa, degrees, freedom, ancova, regressions, homoscedasticity, heteroscedasticity, robust, isotonic, semiparametric, nonlinear, predictors, splines, mars, simultaneous, effects, errors, residuals, confounding, credible, der, waerden, ordered, jonckheere, terpstra, friedman, kruskal, wallis, mann, whitney, hodges, signed, wilcoxon, sign, bic, aic, normality, shapiro, wilk, jarque, bera, lilliefors, anderson, darling, smirnov, goodness, fit, wald, comparisons, randomization, permutation, tails, jackknife, resampling, tolerance, prediction, pivot, confidence, plug, scheffé, rao, blackwellization, estimators, frequentist, sensitivity, robustness, asymptotics, efficiency, loss, completeness, specification, empirical, quasi, sectional, cohort, observational, down, stochastic, scientific, assignment, interaction, factorial, blocking, experiments, error, questionnaire, opinion, poll, stratified, survey, methodology, replication, detrending, differencing, preprocessing, component, dimensionality, truncation, winsorizing, outlier, unit, min, max, standardization, feature, anscombe, stabilizing, yeo, cox, transformations, ecdf, heatmap, violin, stem, leaf, display, run, radar, pie, histogram, forest, fan, correlogram, biplot, bar, graphics, spearman, kendall, grouped, summary, tables, count, skewness, kurtosis, central, percentile, interquartile, variation, average, absolute, mode, lehmer, heinz, heronian, harmonic, cubic, contraharmonic, center, descriptive, outline, wainwright, jordan, 2008, archive, today, deprecated, jmef, java, library, earliest, primer, 1998, sec, 98502, 2006, 93838, fahrmeir, ludwig, tutz, 1994, 349, 94233, 345, modelling, richard, ieee, international, conference, image, 2019, original, 1109, icip, 5652054, 2009arxiv0911, 4863n, reprinted, 2014, 111885750, 1002, 9781118857281, chichester, 238, 0489333, 471, 99545, precise, introducing, lacks, inline, efron, bradley, december, 496, 1614, 22505788, pmid, 3325056, pmc, 0162, 1459, issn, 1198, jasa, tm11181, 1602, 106, bias, strimmer, route, via, abs, duality, garderen, kees, jan, 1997, 790, 122742807, s0266466600006253, 771, econometric, book, new, york, 676700036, 93839, brown, lawrence, 1986, hayward, calif, institute, 15986663, 940600, fundamentals, george, 2002, roger, berger, australia, thomson, learning, 46538638, 534, 24312, blei, david, princeton, abramovich, ritov, chapman, 1439851845, concise, introduction, 2022, www, randomservices, 409, 1501854, 399, transactions, admitting, sur, les, lois, probabilites, exhaustive, 1266, 1265, 200, acad, sci, paris, intrinsic, accuracy, 579, 120708376, s0305004100019307, 1936pcps, 567p, 567, proceedings, cambridge, philosophical, andersen, erling, 1970, spaces, 331, 1255, 0268992, 1248, kupperman, 1958, 575, 2237349, 1214, aoms, 1177706633, 571, annals, probabilities, populations, arising, ones, synonym, gibbs, basis, glm, consequence, clearly, necessarily, sums, disguised, don, usually, parameterization, logarithms, completely, determine, enters, parameterized, show, numerical, combined, success, chooses, computation, lead, flexible, belief, likely, bimodal, humped, combination, hyperprior, controlling, effective, contributes, total, amount, ensure, hyperparameters, produce, belongs, counterexample, relaxed, tersely, increase, whatever, shrink, according, estimated, remains, derivations, imposed, letting, observable, quantities, greatest, subject, though, seldom, pointed, counting, assumes, radon, nikodym, positive, mutually, accordingly, pick, measures, arise, naturally, answer, consistent, swapped, fixing, integral, inner, easier, assuming, opposite, kullback, leibler, require, deriving, network, expand, transposed, back, final, matrices, taking, bit, tricky, respective, illustrates, direct, nearly, impossible, notice, indexed, infty, calculations, done, requires, significantly, 5ex, referring, introductory, take, sides, rely, illustrate, analogous, raw, recovered, technique, complicated, subclass, proving, earlier, justifies, calling, analogy, start, encoding, made, proper, embedded, results, correspond, suggests, whereas, should, emerge, correctly, identifiable, pivots, around, access, demonstrates, translates, constrained, bracketed, enclosed, false, zero, notations, wavey, brackets, assumed, laid, inserted, vectorized, trace, parameterizations, refers, iterated, choose, reason, calculated, involve, parameterizing, instead, outside, former, representations, rewrite, refer, flashcards, next, normally, detailed, densities, typical, infinitely, mixing, precision, heavy, tailed, unlike, affect, allowing, appears, rule, remain, across, settings, affects, changing, reasons, held, stopping, alaam, considering, remember, keep, mind, determining, whether, minimal, nonempty, interior, complete, dots, iid, samples, field, best, context, itself, predictive, summarize, amounts, them, own, derived, mathcal, without, integrable, provides, regard, individual, encapsulates, describe, desired, estimate, discussed, were, interpretations, typically, reduce, paragraphs, steps, integrator, generated, cdf, encompass, compactly, dimensions, trivially, expanded, cover, occurrence, replaced, formulations, merely, requiring, yields, algorithms, designed, implicitly, explicitly, forall, writes, compact, consisting, fashion, except, occurring, qualifies, sort, allows, citation, restrictions, occur, expressions, change, characterizes, separated, within, operation, constituting, exponentiation, factorize, integrate, furthermore, map, inverted, still, vectors, described, transformed, unique, nonzero, offset, measurements, numbers, included, treated, excluded, successes, lower, dependent, geq, greater, depend, requirement, exclude, those, possibly, others, subsections, sequence, increasingly, casual, reader, wish, restrict, attention, simplest, meaning, concept, credited, provide, framework, selecting, parameterisation, specified, convenience, including, enabling, user, covariances, algebraic, generality, sense, older, distinct, possess, variety, desirable, existence, redirects, disambiguation, confused, material, challenged, scholar, books, newspapers, news, removed, reliable, needs, redirected, encyclopedia, item, projects, printable, version, download, print, export, switch, legacy, parser, get, shortened, url, permanent, actions, english, talk, nederlands, 한국어, 日本語, עברית, français, فارسی, español, deutsch, català, top, personal, pages, recent, community, current, events, navigation, jump, content,

Text of the page (random words):
bf x a boldsymbol theta right mu d mathbf x for some reference measure μ displaystyle mu interpretation edit in the definitions above the functions t x η θ and a η were arbitrary however these functions have important interpretations in the resulting probability distribution t x is a sufficient statistic of the distribution for exponential families the sufficient statistic is a function of the data that holds all information the data x provides with regard to the unknown parameter values this means that for any data sets x displaystyle x and y displaystyle y the likelihood ratio is the same that is f x θ 1 f x θ 2 f y θ 1 f y θ 2 displaystyle frac f x theta _ 1 f x theta _ 2 frac f y theta _ 1 f y theta _ 2 if t x t y this is true even if x and y are not equal to each other the dimension of t x equals the number of parameters of θ and encompasses all of the information regarding the data related to the parameter θ the sufficient statistic of a set of independent identically distributed data observations is simply the sum of individual sufficient statistics and encapsulates all the information needed to describe the posterior distribution of the parameters given the data and hence to derive any desired estimate of the parameters this important property is discussed further below η is called the natural parameter the set of values of η for which the function f x x η displaystyle f_ x x eta is integrable is called the natural parameter space it can be shown that the natural parameter space is always convex a η is called the log partition function b because it is the logarithm of a normalization factor without which f x x θ displaystyle f_ x x theta would not be a probability distribution a η log x h x exp η θ t x d x displaystyle a eta log left int _ x h x exp left eta theta cdot t x right dx right the function a is important in its own right because the mean variance and other moments of the sufficient statistic t x can be derived simply by differentiating a η for example because log x is one of the components of the sufficient statistic of the gamma distribution e log x displaystyle operatorname mathcal e log x can be easily determined for this distribution using a η technically this is true because k u η a η u a η displaystyle k left u mid eta right a eta u a eta is the cumulant generating function of the sufficient statistic properties edit exponential families have a large number of properties that make them extremely useful for statistical analysis in many cases it can be shown that only exponential families have these properties examples exponential families are the only families with sufficient statistics that can summarize arbitrary amounts of independent identically distributed data using a fixed number of values pitman koopman darmois theorem exponential families have conjugate priors an important property in bayesian statistics the posterior predictive distribution of an exponential family random variable with a conjugate prior can always be written in closed form provided that the normalizing factor of the exponential family distribution can itself be written in closed form c in the mean field approximation in variational bayes used for approximating the posterior distribution in large bayesian networks the best approximating posterior distribution of an exponential family node a node is a random variable in the context of bayesian networks with a conjugate prior is in the same family as the node 8 given an exponential family defined by f x x θ h x exp θ t x a θ displaystyle f_ x x mid theta h x exp left theta cdot t x a theta right where θ displaystyle theta is the parameter space such that θ θ r k displaystyle theta in theta subset mathbb r k then if θ displaystyle theta has nonempty interior in r k displaystyle mathbb r k then given any iid samples x 1 x n f x displaystyle x_ 1 x_ n sim f_ x the statistic t x 1 x n i 1 n t x i textstyle t x_ 1 dots x_ n sum _ i 1 n t x_ i is a complete statistic for θ displaystyle theta 9 10 t displaystyle t is a minimal statistic for θ displaystyle theta if and only if for all θ 1 θ 2 θ displaystyle theta _ 1 theta _ 2 in theta and x 1 x 2 displaystyle x_ 1 x_ 2 in the support of x displaystyle x if θ 1 θ 2 t x 1 t x 2 0 displaystyle theta _ 1 theta _ 2 cdot t x_ 1 t x_ 2 0 then θ 1 θ 2 displaystyle theta _ 1 theta _ 2 or x 1 x 2 displaystyle x_ 1 x_ 2 11 examples edit it is critical when considering the examples in this section to remember the discussion above about what it means to say that a distribution is an exponential family and in particular to keep in mind that the set of parameters that are allowed to vary is critical in determining whether a distribution is or is not an exponential family the normal exponential log normal gamma chi squared beta dirichlet bernoulli categorical poisson geometric inverse gaussian alaam von mises and von mises fisher distributions are all exponential families some distributions are exponential families only if some of their parameters are held fixed the family of pareto distributions with a fixed minimum bound x m form an exponential family the families of binomial and multinomial distributions with fixed number of trials n but unknown probability parameter s are exponential families the family of negative binomial distributions with fixed number of failures a k a stopping time parameter r is an exponential family however when any of the above mentioned fixed parameters are allowed to vary the resulting family is not an exponential family as mentioned above as a general rule the support of an exponential family must remain the same across all parameter settings in the family this is why the above cases e g binomial with varying number of trials pareto with varying minimum bound are not exponential families in all of the cases the parameter in question affects the support particularly changing the minimum or maximum possible value for similar reasons neither the discrete uniform distribution nor continuous uniform distribution are exponential families as one or both bounds vary the weibull distribution with fixed shape parameter k is an exponential family unlike in the previous examples the shape parameter does not affect the support the fact that allowing it to vary makes the weibull non exponential is due rather to the particular form of the weibull s probability density function k appears in the exponent of an exponent in general distributions that result from a finite or infinite mixture of other distributions e g mixture model densities and compound probability distributions are not exponential families examples are typical gaussian mixture models as well as many heavy tailed distributions that result from compounding i e infinitely mixing a distribution with a prior distribution over one of its parameters e g the student s t distribution compounding a normal distribution over a gamma distributed precision prior and the beta binomial and dirichlet multinomial distributions other examples of distributions that are not exponential families are the f distribution cauchy distribution hypergeometric distribution and logistic distribution following are some detailed examples of the representation of some useful distribution as exponential families normal distribution unknown mean known variance edit as a first example consider a random variable distributed normally with unknown mean μ and known variance σ 2 the probability density function is then f σ x μ 1 2 π σ 2 e x μ 2 2 σ 2 displaystyle f_ sigma x mu frac 1 sqrt 2 pi sigma 2 e x mu 2 2 sigma 2 this is a single parameter exponential family as can be seen by setting t σ x x σ h σ x 1 2 π σ 2 e x 2 2 σ 2 a σ μ μ 2 2 σ 2 η σ μ μ σ displaystyle begin aligned t_ sigma x frac x sigma h_ sigma x frac 1 sqrt 2 pi sigma 2 e x 2 2 sigma 2 4pt a_ sigma mu frac mu 2 2 sigma 2 eta _ sigma mu frac mu sigma end aligned if σ 1 this is in canonical form as then η μ μ normal distribution unknown mean and unknown variance edit next consider the case of a normal distribution with unknown mean and unknown variance the probability density function is then f y μ σ 2 1 2 π σ 2 e y μ 2 2 σ 2 displaystyle f y mu sigma 2 frac 1 sqrt 2 pi sigma 2 e y mu 2 2 sigma 2 this is an exponential family which can be written in canonical form by defining h y 1 2 π η μ σ 2 1 2 σ 2 t y y y 2 t a η μ 2 2 σ 2 log σ η 1 2 4 η 2 1 2 log 1 2 η 2 displaystyle begin aligned h y frac 1 sqrt 2 pi boldsymbol eta left frac mu sigma 2 frac 1 2 sigma 2 right t y left y y 2 right mathsf t a boldsymbol eta frac mu 2 2 sigma 2 log sigma frac eta _ 1 2 4 eta _ 2 frac 1 2 log left frac 1 2 eta _ 2 right end aligned binomial distribution edit as an example of a discrete exponential family consider the binomial distribution with known number of trials n the probability mass function for this distribution is f x n x p x 1 p n x x 0 1 2 n displaystyle f x binom n x p x left 1 p right n x quad x in 0 1 2 ldots n this can equivalently be written as f x n x exp x log p 1 p n log 1 p displaystyle f x binom n x exp left x log left frac p 1 p right n log 1 p right which shows that the binomial distribution is an exponential family whose natural parameter is η log p 1 p displaystyle eta log frac p 1 p this function of p is known as logit table of distributions edit the following table shows how to rewrite a number of common distributions as exponential family distributions with natural parameters refer to the flashcards 12 for main exponential families for a scalar variable and scalar parameter the form is as follows f x x θ h x exp η θ t x a η displaystyle f_ x x mid theta h x exp left eta theta t x a eta right for a scalar variable and vector parameter f x x θ h x exp η θ t x a η f x x θ h x g θ exp η θ t x displaystyle begin aligned f_ x x mid boldsymbol theta h x exp left boldsymbol eta boldsymbol theta cdot mathbf t x a boldsymbol eta right 4pt f_ x x mid boldsymbol theta h x g boldsymbol theta exp left boldsymbol eta boldsymbol theta cdot mathbf t x right end aligned for a vector variable and vector parameter f x x θ h x exp η θ t x a η displaystyle f_ x mathbf x mid boldsymbol theta h mathbf x exp left boldsymbol eta boldsymbol theta cdot mathbf t mathbf x a boldsymbol eta right the above formulas choose the functional form of the exponential family with a log partition function a η displaystyle a boldsymbol eta the reason for this is so that the moments of the sufficient statistics can be calculated easily simply by differentiating this function alternative forms involve either parameterizing this function in terms of the normal parameter θ displaystyle boldsymbol theta instead of the natural parameter and or using a factor g η displaystyle g boldsymbol eta outside of the exponential the relation between the latter and the former is a η log g η g η e a η displaystyle begin aligned a boldsymbol eta log g boldsymbol eta 2pt g boldsymbol eta e a boldsymbol eta end aligned to convert between the representations involving the two types of parameter use the formulas below for writing one type of parameter in terms of the other distribution parameter s θ natural parameter s η inverse parameter mapping base measure h x sufficient statistic t x log partition a η log partition a θ bernoulli distribution p displaystyle p log p 1 p displaystyle log frac p 1 p this is the logit function 1 1 e η e η 1 e η displaystyle frac 1 1 e eta frac e eta 1 e eta this is the logistic function 1 displaystyle 1 x displaystyle x log 1 e η displaystyle log 1 e eta log 1 p displaystyle log 1 p binomial distribution with known number of trials n displaystyle n p displaystyle p log p 1 p displaystyle log frac p 1 p 1 1 e η e η 1 e η displaystyle frac 1 1 e eta frac e eta 1 e eta n x displaystyle binom n x x displaystyle x n log 1 e η displaystyle n log 1 e eta n log 1 p displaystyle n log 1 p poisson distribution λ displaystyle lambda log λ displaystyle log lambda e η displaystyle e eta 1 x displaystyle frac 1 x x displaystyle x e η displaystyle e eta λ displaystyle lambda negative binomial distribution with known number of failures r displaystyle r p displaystyle p log 1 p displaystyle log 1 p 1 e η displaystyle 1 e eta x r 1 x displaystyle binom x r 1 x x displaystyle x r log 1 e η displaystyle r log 1 e eta r log p displaystyle r log p exponential distribution λ displaystyle lambda λ displaystyle lambda η displaystyle eta 1 displaystyle 1 x displaystyle x log η displaystyle log eta log λ displaystyle log lambda pareto distribution with known minimum value x m displaystyle x_ m α displaystyle alpha α 1 displaystyle alpha 1 1 η displaystyle 1 eta 1 displaystyle 1 log x displaystyle log x log 1 η 1 η log x m displaystyle begin aligned log 1 eta 1 eta log x_ mathrm m end aligned log α x m α displaystyle log left alpha x_ mathrm m alpha right weibull distribution with known shape k λ displaystyle lambda 1 λ k displaystyle frac 1 lambda k η 1 k displaystyle eta 1 k x k 1 displaystyle x k 1 x k displaystyle x k log 1 η k displaystyle log left frac 1 eta k right log λ k k displaystyle log frac lambda k k laplace distribution with known mean μ displaystyle mu b displaystyle b 1 b displaystyle frac 1 b 1 η displaystyle frac 1 eta 1 displaystyle 1 x μ displaystyle x mu log 2 η displaystyle log left frac 2 eta right log 2 b displaystyle log 2b chi squared distribution ν displaystyle nu ν 2 1 displaystyle frac nu 2 1 2 η 1 displaystyle 2 eta 1 e x 2 displaystyle e x 2 log x displaystyle log x log γ η 1 η 1 log 2 displaystyle begin aligned log gamma eta 1 eta 1 log 2 end aligned log γ ν 2 ν 2 log 2 displaystyle begin aligned log gamma left tfrac nu 2 right tfrac nu 2 log 2 end aligned normal distribution known variance μ displaystyle mu μ σ displaystyle frac mu sigma σ η displaystyle sigma eta e x 2 2 σ 2 2 π σ displaystyle frac e x 2 2 sigma 2 sqrt 2 pi sigma x σ displaystyle frac x sigma η 2 2 displaystyle frac eta 2 2 μ 2 2 σ 2 displaystyle frac mu 2 2 sigma 2 continuous bernoulli distribution λ displaystyle lambda log λ 1 λ displaystyle log frac lambda 1 lambda e η 1 e η displaystyle frac e eta 1 e eta 1 displaystyle 1 x displaystyle x log e η 1 η displaystyle log frac e eta 1 eta log 1 2 λ 1 λ log 2 1 λ 1 displaystyle begin aligned log left tfrac 1 2 lambda 1 lambda right 1ex log 2 left tfrac 1 lambda 1 right end aligned where log 2 refers to the iterated logarithm normal distribution μ σ 2 displaystyle mu sigma 2 μ σ 2 1 2 σ 2 displaystyle begin bmatrix dfrac mu sigma 2 1ex dfrac 1 2 sigma 2 end bmatrix η 1 2 η 2 1 2 η 2 displaystyle begin bmatrix dfrac eta _ 1 2 eta _ 2 1ex dfrac 1 2 eta _ 2 end bmatrix 1 2 π displaystyle frac 1 sqrt 2 pi x x 2 displaystyle begin bmatrix x x 2 end bmatrix η 1 2 4 η 2 1 2 log 2 η 2 displaystyle frac eta _ 1 2 4 eta _ 2 frac 1 2 log 2 eta _ 2 μ 2 2 σ 2 log σ displaystyle frac mu 2 2 sigma 2 log sigma log normal distribution μ σ 2 displaystyle mu sigma 2 μ σ 2 1 2 σ 2 displaystyle begin bmatrix dfrac mu sigma 2 1ex dfrac 1 2 sigma 2 end ...