Meta tags:

Headings (most frequently used words):

generative, models, model, contents, definition, applications, families, and, types, see, also, notes, references, sources, external, links, relationships, between, contrast, with, discriminative, classifiers, examples, deep, simple, example, text, generation,

Text of the page (most frequently used words):
the (114), and (71), model (68), displaystyle (62), generative (55), models (36), data (33), #discriminative (33), distribution (30), probability (27), can (26), are (25), mid (23), learning (19), conditional (19), edit (19), for (19), that (17), given (17), statistics (15), analysis (15), regression (15), from (14), target (14), this (13), classifiers (13), statistical (12), classification (12), bayes (12), one (12), with (11), test (11), joint (11), used (11), not (10), classifier (10), which (10), wikipedia (9), machine (9), deep (9), observation (9), page (8), correlation (8), plot (8), learn (8), these (8), label (8), using (7), function (7), likelihood (7), variable (7), transformation (7), chart (7), 2002 (7), directly (7), inputs (7), between (7), generate (7), variables (7), table (6), estimator (6), time (6), estimation (6), distributions (6), linear (6), variance (6), bayesian (6), interval (6), scale (6), jordan (6), labels (6), estimate (6), algorithms (6), then (6), generation (6), also (6), based (6), other (6), types (6), but (6), two (6), definition (6), toggle (5), view (5), about (5), may (5), use (5), focus (5), different (5), probabilistic (5), log (5), rank (5), density (5), logistic (5), most (5), random (5), jebara (5), neural (5), pdf (5), naive (5), 978 (5), isbn (5), class (5), value (5), rule (5), how (5), example (5), networks (5), more (5), than (5), classes (5), approaches (5), article (5), contents (4), search (4), terms (4), was (4), applications (4), box (4), statistic (4), general (4), series (4), multivariate (4), families (4), simple (4), adaptive (4), power (4), family (4), experiment (4), missing (4), normalization (4), scaling (4), links (4), 2015 (4), mitchell (4), 2016 (4), large (4), training (4), 2004 (4), because (4), new (4), examples (4), instances (4), see (4), parameters (4), adversarial (4), observed (4), application (4), such (4), will (4), particular (4), english (4), while (4), without (4), observable (4), outputs (4), main (4), hide (4), move (4), sidebar (4), text (3), non (3), articles (3), retrieved (3), index (3), portal (3), information (3), system (3), population (3), control (3), process (3), design (3), engineering (3), product (3), survival (3), least (3), squares (3), domain (3), autoregressive (3), cross (3), partial (3), specific (3), tests (3), exponential (3), factor (3), standard (3), posterior (3), inference (3), way (3), median (3), parametric (3), unbiased (3), moments (3), sampling (3), theory (3), study (3), approximation (3), scatter (3), continuous (3), tom (3), 1948 (3), sources (3), 2020 (3), openai (3), generating (3), arxiv (3), better (3), language (3), their (3), both (3), press (3), algorithm (3), contrast (3), often (3), referred (3), some (3), called (3), springer (3), three (3), there (3), has (3), variational (3), hand (3), approach (3), network (3), free (3), mixture (3), word (3), increasingly (3), following (3), set (3), they (3), relationships (3), signal (3), try (3), classify (3), frequently (3), computed (3), hence (3), over (3), symbolically (3), distinguished (3), term (3), discriminate (3), compute (3), tools (3), subsection (3), topic (2), languages (2), contact (2), privacy (2), policy (2), you (2), commons (2), last (2), 2026 (2), hidden (2), categories (2), lacking (2), 2025 (2), short (2), description (2), wikidata (2), mathematics (2), category (2), science (2), methods (2), studies (2), clinical (2), kaplan (2), limit (2), spectral (2), frequency (2), vector (2), structural (2), seasonal (2), adjustment (2), stationarity (2), trend (2), normal (2), cluster (2), discriminant (2), principal (2), manova (2), graphical (2), contingency (2), categorical (2), anova (2), generalized (2), nonparametric (2), equations (2), validation (2), coefficient (2), determination (2), pearson (2), moment (2), maximum (2), prior (2), van (2), alternative (2), sum (2), lehmann (2), sample (2), score (2), multiple (2), bootstrap (2), theorem (2), mean (2), minimum (2), estimators (2), distance (2), method (2), estimating (2), optimal (2), location (2), shape (2), empirical (2), order (2), natural (2), designs (2), trial (2), randomized (2), controlled (2), size (2), reduction (2), cleaning (2), transform (2), processing (2), dependence (2), dispersion (2), deviation (2), range (2), geometric (2), arithmetic (2), above (2), mirror (2), tony (2), hdl (2), massachusetts (2), external (2), andrew (2), shannon (2), april (2), jukebox (2), 2019 (2), high (2), fidelity (2), vae (2), image (2), june (2), bishop (2), international (2), 2006 (2), distinction (2), well (2), uses (2), any (2), estimated (2), colon (2), likely (2), mit (2), what (2), synthetic (2), references (2), give (2), definitions (2), notes (2), dgms (2), include (2), auto (2), regressive (2), been (2), very (2), billions (2), gpt (2), autoencoders (2), increase (2), typically (2), good (2), performance (2), since (2), only (2), values (2), others (2), assumptions (2), problem (2), boltzmann (2), gaussian (2), frequencies (2), pairs (2), input (2), simulation (2), generally (2), tasks (2), generated (2), computing (2), considering (2), discrete (2), either (2), finite (2), related (2), each (2), equivalent (2), together (2), potential (2), samples (2), terminology (2), distinguish (2), consistently (2), when (2), outcomes (2), base (2), refers (2), help (2), talk (2), disambiguation (2), appearance (2), upload (2), file (2), changes (2), history (2), read (2), create (2), account (2), donate (2), menu (2), add, mobile, cookie, statement, developers, code, conduct, legal, safety, contacts, disclaimers, available, under, additional, apply, site, agree, registered, trademark, profit, organization, wikimedia, foundation, inc, creative, attribution, sharealike, license, rendered, parsoid, edited, august, utc, all, https, org, php, title, generative_model, oldid, 1371270811, wikiproject, kriging, geostatistics, geographic, environmental, cartography, spatial, psychometrics, official, national, accounts, jurimetrics, econometrics, demography, crime, census, actuarial, social, identification, reliability, quality, chemometrics, medical, epidemiology, trials, bioinformatics, biostatistics, nelson, aalen, hazard, first, hitting, accelerated, failure, aft, proportional, hazards, meier, whittle, wavelet, fourier, autoregression, var, heteroskedasticity, arch, arima, jenkins, arma, xcf, pacf, autocorrelation, acf, breusch, godfrey, durbin, watson, ljung, johansen, dickey, fuller, granger, causality, break, cointegration, smoothing, decomposition, elliptical, equation, canonical, components, cochran, mantel, haenszel, mcnemar, cohen, kappa, degrees, freedom, ancova, partition, poisson, regressions, binomial, bernoulli, homoscedasticity, heteroscedasticity, robust, isotonic, semiparametric, nonlinear, predictors, ordinary, template, splines, mars, simultaneous, mixed, effects, errors, residuals, confounding, credible, der, waerden, ordered, jonckheere, terpstra, friedman, kruskal, wallis, mann, whitney, hodges, signed, wilcoxon, sign, bic, aic, selection, normality, shapiro, wilk, jarque, bera, lilliefors, anderson, darling, kolmogorov, smirnov, chi, squared, goodness, fit, student, wald, lagrange, multiplier, ratio, comparisons, randomization, permutation, uniformly, powerful, tails, testing, hypotheses, jackknife, resampling, tolerance, prediction, pivot, confidence, plug, scheffé, rao, blackwellization, point, frequentist, sensitivity, robustness, asymptotics, divergence, efficiency, loss, decision, functional, sufficiency, completeness, monotone, parameter, space, specification, quasi, sectional, cohort, observational, down, stochastic, scientific, assignment, interaction, factorial, blocking, experiments, error, questionnaire, opinion, poll, stratified, survey, methodology, replication, effect, collection, detrending, differencing, preprocessing, component, dimensionality, truncation, winsorizing, outlier, unit, min, max, standardization, feature, fisher, anscombe, stabilizing, yeo, johnson, cox, transformations, line, ecdf, matrix, heatmap, violin, stem, leaf, display, run, radar, pie, histogram, forest, fan, correlogram, biplot, bar, graphics, spearman, kendall, grouped, summary, tables, count, skewness, kurtosis, central, percentile, interquartile, variation, average, absolute, mode, lehmer, heinz, heronian, harmonic, cubic, contraharmonic, center, descriptive, outline, published, book, phd, 1721, 8323, institute, technology, imitative, advances, systems, comparison, michael, july, october, 423, 656, archived, original, 10338, dmlcz, 101429, 1002, 1538, 7305, tb01338, doi, 1948bstj, 379s, bibcode, 623, 379, bell, technical, journal, mathematical, communication, razavi, ali, den, oord, aaron, vinyals, oriol, diverse, images, 1906, 00446, brock, donahue, jeff, simonyan, karen, 2018, gan, synthesis, 1809, 11096, february, implications, jared, mccandlish, sam, henighan, brown, chess, benjamin, child, rewon, gray, scott, radford, alec, jeffrey, amodei, dario, laws, stat, 2001, 08361, microsoft, researchers, advance, lasserre, september, 2007, getting, best, worlds, bernardo, oxford, university, 921465, proceedings, eighth, valencia, meeting, direct, map, currently, established, convention, field, sense, discriminating, instance, basis, designing, approximators, follows, wish, equivalently, estimates, classified, plus, type, describing, conditioned, attribute, make, predictions, rules, calculate, picking, computer, kluwer, academic, 4020, 7647, christopher, 0387310732, pattern, recognition, murphy, kevin, 2012, 0262018029, perspective, k2view, goodfellow, ian, bengio, yoshua, computation, cambridge, 262, 03561, leading, divisions, popular, vaes, gans, recently, build, its, precursor, contain, biggan, have, hundreds, millions, musical, audio, contains, rise, formed, through, combination, accompanied, required, truly, sampled, fitting, common, however, approximations, infer, subset, known, argued, makes, necessary, solve, cases, accurate, functions, below, although, details, ultimately, dictate, suitable, case, true, maximize, diffusion, energy, flow, autoencoder, belief, restricted, latent, dirichlet, allocation, averaged, context, grammar, markov, gives, sentence, beginning, representing, speedily, proper, approximate, moved, triplets, etc, measure, suppose, points, semi, supervised, anomaly, detection, imputation, despite, fact, need, cannot, express, complex, don, necessarily, perform, seen, complementary, views, same, procedure, categorize, asks, question, does, care, simply, categorizes, transformed, into, later, advantages, similar, existing, proved, note, conflated, denoted, opposite, individual, integrating, summing, int, marginal, closely, observations, thus, informative, relative, relatively, small, step, always, consisting, interpreted, deterministic, describe, output, clear, relationship, judged, primarily, similarity, regardless, precise, constitutional, difference, subtle, becomes, discrimination, pleonasm, division, defines, symmetrically, wishes, indirect, allowing, applied, practice, depending, hybrids, combine, strengths, knowledge, made, calling, them, distinguishing, latter, analogously, though, loosely, differing, degree, inconsistent, major, modelling, instead, distributed, within, describes, full, draw, resemble, partially, labeled, predict, combining, applying, contrasted, predicting, computational, covers, should, former, already, dedicated, remove, message, please, possibly, creating, relevant, discussion, found, splitting, improve, lack, encyclopedia, item, projects, printable, version, download, print, export, switch, legacy, parser, get, shortened, url, cite, permanent, link, here, actions, tiếng, việt, українська, ไทย, русский, 한국어, 日本語, bahasa, indonesia, עברית, français, فارسی, euskara, español, deutsch, català, العربية, top, personal, special, pages, recent, community, contribute, current, events, navigation, jump, content,

Text of the page (random words):
he target y is generally a discrete variable consisting of a finite set of labels and the conditional probability p y x displaystyle p y mid x can also be interpreted as a non deterministic target function f x y displaystyle f colon x to y considering x as inputs and y as outputs given a finite set of labels the two definitions of generative model are closely related a model of the conditional distribution p x y y displaystyle p x mid y y is a model of the distribution of each label and a model of the joint distribution is equivalent to a model of the distribution of label values p y displaystyle p y together with the distribution of observations given a label p x y displaystyle p x mid y symbolically p x y p x y p y displaystyle p x y p x mid y p y thus while a model of the joint probability distribution is more informative than a model of the distribution of label but without their relative frequencies it is a relatively small step hence these are not always distinguished given a model of the joint distribution p x y displaystyle p x y the distribution of the individual variables can be computed as the marginal distributions p x y p x y y displaystyle p x sum _ y p x y y and p y x p y x x displaystyle p y int _ x p y x x considering x as continuous hence integrating over it and y as discrete hence summing over it and either conditional distribution can be computed from the definition of conditional probability p x y p x y p y displaystyle p x mid y p x y p y and p y x p x y p x displaystyle p y mid x p x y p x given a model of one conditional probability and estimated probability distributions for the variables x and y denoted p x displaystyle p x and p y displaystyle p y one can estimate the opposite conditional probability using bayes rule p x y p y p y x p x displaystyle p x mid y p y p y mid x p x for example given a generative model for p x y displaystyle p x mid y one can estimate p y x p x y p y p x displaystyle p y mid x p x mid y p y p x and given a discriminative model for p y x displaystyle p y mid x one can estimate p x y p y x p x p y displaystyle p x mid y p y mid x p x p y note that bayes rule computing one conditional probability in terms of the other and the definition of conditional probability computing conditional probability in terms of the joint distribution are frequently conflated as well contrast with discriminative classifiers edit a generative algorithm models how the data was generated in order to categorize a signal it asks the question based on my generation assumptions which category is most likely to generate this signal a discriminative algorithm does not care about how the data was generated it simply categorizes a given signal so discriminative algorithms try to learn p y x displaystyle p y x directly from the data and then try to classify data on the other hand generative algorithms try to learn p x y displaystyle p x y which can be transformed into p y x displaystyle p y x later to classify the data one of the advantages of generative algorithms is that you can use p x y displaystyle p x y to generate new data similar to existing data on the other hand it has been proved that some discriminative algorithms give better performance than some generative algorithms in classification tasks 11 despite the fact that discriminative models do not need to model the distribution of the observed variables they cannot generally express complex relationships between the observed and target variables but in general they don t necessarily perform better than generative models at classification and regression tasks the two classes are seen as complementary or as different views of the same procedure 12 applications edit sampling simulation classification density estimation and likelihood missing data and imputation anomaly detection semi supervised learning examples edit simple example edit suppose the input data is x 1 2 displaystyle x in 1 2 the set of labels for x displaystyle x is y 0 1 displaystyle y in 0 1 and there are the following 4 data points x y 1 0 1 1 2 0 2 1 displaystyle x y 1 0 1 1 2 0 2 1 for the above data estimating the joint probability distribution p x y displaystyle p x y from the empirical measure will be the following y 0 displaystyle y 0 y 1 displaystyle y 1 x 1 displaystyle x 1 1 4 displaystyle 1 4 1 4 displaystyle 1 4 x 2 displaystyle x 2 1 4 displaystyle 1 4 1 4 displaystyle 1 4 while p y x displaystyle p y x will be following y 0 displaystyle y 0 y 1 displaystyle y 1 x 1 displaystyle x 1 1 2 displaystyle 1 2 1 2 displaystyle 1 2 x 2 displaystyle x 2 1 2 displaystyle 1 2 1 2 displaystyle 1 2 text generation edit shannon 1948 gives an example in which a table of frequencies of english word pairs is used to generate a sentence beginning with representing and speedily is an good which is not proper english but which will increasingly approximate it as the table is moved from word pairs to word triplets etc families and types edit generative models edit types of generative models are gaussian mixture model and other types of mixture model hidden markov model probabilistic context free grammar bayesian network e g naive bayes autoregressive model generative adversarial network generative ai averaged one dependence estimators latent dirichlet allocation boltzmann machine e g restricted boltzmann machine deep belief network variational autoencoder flow based generative model energy based model diffusion model linear discriminant analysis if the observed data are truly sampled from the generative model then fitting the parameters of the generative model to maximize the data likelihood is a common method however since most statistical models are only approximations to the true distribution if the model s application is to infer about a subset of variables conditional on known values of others then it can be argued that the approximation makes more assumptions than are necessary to solve the problem at hand in such cases it can be more accurate to model the conditional density functions directly using a discriminative model see below although application specific details will ultimately dictate which approach is most suitable in any particular case deep generative models edit with the rise of deep learning a new family of methods called deep generative models dgms 13 14 is formed through the combination of generative models and deep neural networks an increase in the scale of the neural networks is typically accompanied by an increase in the scale of the training data both of which are required for good performance 15 popular dgms include variational autoencoders vaes generative adversarial networks gans and auto regressive models recently there has been a trend to build very large deep generative models 13 for example gpt 3 and its precursor gpt 2 16 are auto regressive neural language models that contain billions of parameters biggan 17 and vq vae 18 which are used for image generation that can have hundreds of millions of parameters and jukebox is a very large generative model for musical audio that contains billions of parameters 19 see also edit mathematics portal discriminative model graphical model notes edit three leading sources ng jordan 2002 jebara 2004 and mitchell 2015 give different divisions and definitions references edit goodfellow ian bengio yoshua 2016 deep learning adaptive computation and machine learning cambridge massachusetts the mit press isbn 978 0 262 03561 3 what is synthetic data generation k2view retrieved 2026 03 19 murphy kevin p 2012 machine learning a probabilistic perspective mit press isbn 978 0262018029 bishop christopher m 2006 pattern recognition and machine learning springer isbn 978 0387310732 jebara tony 2004 machine learning discriminative and generative the springer international series in engineering and computer science kluwer academic springer isbn 978 1 4020 7647 3 ng jordan 2002 generative classifiers learn a model of the joint probability p x y displaystyle p x y of the inputs x and the label y and make their predictions by using bayes rules to calculate p y x displaystyle p y mid x and then picking the most likely label y 1 2 3 mitchell 2015 we can use bayes rule as the basis for designing learning algorithms function approximators as follows given that we wish to learn some target function f x y displaystyle f colon x to y or equivalently p y x displaystyle p y mid x we use the training data to learn estimates of p x y displaystyle p x mid y and p y displaystyle p y new x examples can then be classified using these estimated probability distributions plus bayes rule this type of classifier is called a generative classifier because we can view the distribution p x y displaystyle p x mid y as describing how to generate random instances x conditioned on the target attribute y 1 2 3 mitchell 2015 logistic regression is a function approximation algorithm that uses training data to directly estimate p y x displaystyle p y mid x in contrast to naive bayes in this sense logistic regression is often referred to as a discriminative classifier because we can view the distribution p y x displaystyle p y mid x as directly discriminating the value of the target value y for any given instance x jebara 2004 2 4 discriminative learning this distinction between conditional learning and discriminative learning is not currently a well established convention in the field ng jordan 2002 discriminative classifiers model the posterior p y x displaystyle p y x directly or learn a direct map from inputs x to the class labels ng jordan 2002 bishop c m lasserre j 24 september 2007 generative or discriminative getting the best of both worlds in bernardo j m ed bayesian statistics 8 proceedings of the eighth valencia international meeting june 2 6 2006 oxford university press pp 3 23 isbn 978 0 19 921465 5 1 2 scaling up researchers advance large scale deep generative models microsoft april 9 2020 generative models openai june 16 2016 kaplan jared mccandlish sam henighan tom brown tom b chess benjamin child rewon gray scott radford alec wu jeffrey amodei dario 2020 scaling laws for neural language models arxiv 2001 08361 stat ml better language models and their implications openai february 14 2019 brock andrew donahue jeff simonyan karen 2018 large scale gan training for high fidelity natural image synthesis arxiv 1809 11096 cs lg razavi ali van den oord aaron vinyals oriol 2019 generating diverse high fidelity images with vq vae 2 arxiv 1906 00446 cs lg jukebox openai april 30 2020 sources edit shannon c e 1948 a mathematical theory of communication pdf bell system technical journal 27 july october 379 423 623 656 bibcode 1948bstj 27 379s doi 10 1002 j 1538 7305 1948 tb01338 x hdl 10338 dmlcz 101429 archived from the original pdf on 2016 06 06 retrieved 2016 01 09 mitchell tom m 2015 3 generative and discriminative classifiers naive bayes and logistic regression pdf machine learning ng andrew y jordan michael i 2002 on discriminative vs generative classifiers a comparison of logistic regression and naive bayes pdf advances in neural information processing systems external links edit jebara tony 2002 discriminative generative and imitative learning phd massachusetts institute of technology hdl 1721 1 8323 mirror mirror published as book above v t e statistics outline index descriptive statistics continuous data center mean arithmetic arithmetic geometric contraharmonic cubic generalized power geometric harmonic heronian heinz lehmer median mode dispersion average absolute deviation coefficient of variation interquartile range percentile range standard deviation variance shape central limit theorem moments kurtosis l moments skewness count data index of dispersion summary tables contingency table frequency distribution grouped data dependence partial correlation pearson product moment correlation rank correlation kendall s τ spearman s ρ scatter plot graphics bar chart biplot box plot control chart correlogram fan chart forest plot histogram pie chart q q plot radar chart run chart scatter plot stem and leaf display violin plot heatmap scatter plot matrix ecdf plot line chart statistical data processing transformations data transformation log transformation power transform box cox transformation yeo johnson transformation variance stabilizing transformation anscombe transform fisher transformation scaling and normalization feature scaling normalization standardization z score min max normalization unit vector normalization data cleaning data cleaning outlier winsorizing truncation missing data data reduction dimensionality reduction principal component analysis factor analysis time series preprocessing differencing detrending seasonal adjustment stationarity transformation data collection study design effect size missing data optimal design population replication sample size determination statistic statistical power survey methodology sampling cluster stratified opinion poll questionnaire standard error controlled experiments blocking factorial experiment interaction random assignment randomized controlled trial randomized experiment scientific control adaptive designs adaptive clinical trial stochastic approximation up and down designs observational studies cohort study cross sectional study natural experiment quasi experiment statistical inference statistical theory population statistic probability distribution sampling distribution order statistic empirical distribution density estimation statistical model model specification l p space parameter location scale shape parametric family likelihood monotone location scale family exponential family completeness sufficiency statistical functional bootstrap u v optimal decision loss function efficiency statistical distance divergence asymptotics robustness sensitivity analysis frequentist inference point estimation estimating equations maximum likelihood method of moments m estimator minimum distance unbiased estimators mean unbiased minimum variance rao blackwellization lehmann scheffé theorem median unbiased plug in interval estimation confidence interval pivot likelihood interval prediction interval tolerance interval resampling bootstrap jackknife testing hypotheses 1 2 tails power uniformly most powerful test permutation test randomization test multiple comparisons parametric tests likelihood ratio g test score lagrange multiplier wald z test normal specific tests parametric student s t test f test goodness of fit chi squared kolmogorov smirnov anderson darling lilliefors jarque bera normality shapiro wilk model selection cross validation aic bic rank statistics sign sample median signed rank wilcoxon hodges lehmann estimator rank sum mann whitney nonparametric anova 1 way kruskal wallis 2 way friedman ordered alternative jonckheere terpstra van der waerden test bayesian inference bayesian probability prior posterior credible interval bayes factor bayesian estimator maximum posterior estimator correl...