Meta tags:

Headings (most frequently used words):

regularization, tikhonov, to, in, relation, generalized, estimator, ridge, regression, contents, overview, determination, of, the, parameter, history, for, linear, equations, lavrentyev, hilbert, space, singular, value, decomposition, and, wiener, filter, probabilistic, formulation, bayesian, interpretation, see, also, notes, references, further, reading, cross, validation, plug, application, existing, fit, results,

Text of the page (most frequently used words):
the (207), displaystyle (146), mathbf (112), and (77), #regression (51), for (48), #regularization (45), tikhonov (38), mathsf (37), ridge (33), least (32), linear (31), boldsymbol (31), squares (29), matrix (29), left (27), right (27), beta (27), gamma (26), with (25), lambda (25), hat (25), solution (24), problems (23), edit (21), can (20), this (19), parameter (19), isbn (18), that (18), generalized (17), problem (17), estimator (17), model (15), where (15), from (14), doi (14), method (14), analysis (13), inverse (13), are (13), sigma (13), numerical (11), bayesian (11), ordinary (11), data (11), wikipedia (10), one (10), mathcal (10), statistics (9), non (9), estimation (9), ill (9), posed (9), equations (9), varsigma (9), mathematics (8), validation (8), covariance (8), press (8), estimators (8), hoerl (8), known (8), frac (8), singular (8), has (8), which (8), used (8), may (7), page (7), methods (7), theory (7), mean (7), errors (7), new (7), 978 (7), cross (7), its (7), using (6), was (6), curve (6), optimal (6), square (6), fit (6), decomposition (6), york (6), 1998 (6), bibcode (6), some (6), technometrics (6), wiener (6), choice (6), independent (6), also (6), have (6), parameters (6), case (6), filter (6), value (6), term (6), vector (6), 2020 (5), number (5), response (5), theorem (5), selection (5), variance (5), general (5), weighted (5), statistical (5), learning (5), applied (5), arxiv (5), 1970 (5), nonorthogonal (5), application (5), andrey (5), models (5), see (5), then (5), regularized (5), above (5), alpha (5), operator (5), lavrentyev (5), norm (5), omega (5), toggle (4), contents (4), search (4), view (4), articles (4), clarification (4), retrieved (4), applications (4), approximation (4), design (4), minimum (4), multivariate (4), logistic (4), normal (4), nonlinear (4), standard (4), correlation (4), 2019 (4), machine (4), springer (4), crc (4), efficiency (4), econometrics (4), society (4), industrial (4), 2005 (4), journal (4), kennard (4), arthur (4), phillips (4), practice (4), needed (4), assumptions (4), zero (4), main (4), interpretation (4), when (4), uncertainties (4), relation (4), result (4), smaller (4), formula (4), variables (4), likelihood (4), many (4), pmatrix (4), more (4), than (4), history (4), plug (4), mathrm (4), rss (4), determination (4), effects (4), hide (4), move (4), sidebar (4), additional (3), use (3), needing (3), 2022 (3), portal (3), error (3), residual (3), residuals (3), gauss (3), markov (3), robust (3), simple (3), partial (3), tau (3), rank (3), moment (3), business (3), science (3), sons (3), scientific (3), 387 (3), gruber (3), marvin (3), boca (3), raton (3), 8247 (3), 0156 (3), improving (3), shrinkage (3), james (3), stein (3), further (3), philadelphia (3), 89871 (3), huang (3), yunfei (3), pmid (3), 1038 (3), traction (3), force (3), microscopy (3), automated (3), 2018 (3), august (3), discrete (3), inversion (3), regularisation (3), factorization (3), s2cid (3), pdf (3), 1080 (3), 00401706 (3), context (3), biased (3), integral (3), first (3), original (3), wahba (3), notes (3), multiple (3), related (3), note (3), distribution (3), here (3), following (3), components (3), these (3), most (3), article (3), probabilistic (3), formulation (3), special (3), sum (3), values (3), expressed (3), effect (3), condition (3), hilbert (3), space (3), since (3), minimization (3), written (3), explicitly (3), transpose (3), thus (3), squared (3), not (3), function (3), such (3), cases (3), pass (3), mapping (3), driven (3), unknown (3), varepsilon (3), top (3), operatorname (3), via (3), constraint (3), mixed (3), probit (3), tools (3), languages (2), table (2), contact (2), about (2), privacy (2), policy (2), text (2), under (2), terms (2), apply (2), categories (2), short (2), description (2), wikidata (2), title (2), category (2), system (2), smoothing (2), fitting (2), chebyshev (2), polynomials (2), gaussian (2), studentized (2), goodness (2), predicted (2), background (2), poisson (2), binomial (2), isotonic (2), quantile (2), semiparametric (2), nonparametric (2), local (2), segmented (2), polynomial (2), total (2), iteratively (2), reweighted (2), computational (2), john (2), wiley (2), cambridge (2), university (2), section (2), reading (2), advanced (2), 539 (2), 537 (2), 30679578 (2), 6345967 (2), pmc (2), s41598 (2), 018 (2), 36896 (2), 2019natsr (2), 539h (2), 1810 (2), 05848 (2), reports (2), optimized (2), comparing (2), cells (2), literature (2), 107313 (2), computer (2), physics (2), communications (2), 1st (2), siam (2), hansen (2), mathematical (2), 2012 (2), large (2), classification (2), foster (2), kolmogorov (2), 1962 (2), engineering (2), technique (2), yagola (2), anatolij (2), grigorevic (2), london (2), nikolayevich (2), netherlands (2), 1977 (2), задач (2), doklady (2), akademii (2), nauk (2), sssr (2), 1979 (2), choosing (2), series (2), principal (2), robert (2), 1267352 (2), jstor (2), 2307 (2), 1267351 (2), donald (2), references (2), constrained (2), replaced (2), rather (2), arbitrary (2), process (2), order (2), get (2), unique (2), taken (2), means (2), their (2), subject (2), deviation (2), given (2), priori (2), prior (2), point (2), information (2), representing (2), matrices (2), diagonal (2), reduce (2), derived (2), typically (2), conditioned (2), dimensional (2), elements (2), compared (2), nothing (2), but (2), example (2), positive (2), equivalently (2), constant (2), included (2), corresponding (2), reduces (2), minimize (2), expected (2), valid (2), long (2), quadratic (2), unregularised (2), underlying (2), estimated (2), will (2), possible (2), after (2), existing (2), results (2), contexts (2), preference (2), chosen (2), other (2), high (2), usual (2), identity (2), begin (2), end (2), approach (2), equation (2), often (2), real (2), forward (2), noise (2), version (2), find (2), different (2), widely (2), who (2), named (2), maximum (2), fixed (2), mathbb (2), geq (2), shown (2), define (2), lim (2), downarrow (2), diagonals (2), argmin (2), overview (2), highly (2), correlated (2), provides (2), regressions (2), estimating (2), bias (2), random (2), ordered (2), logit (2), multinomial (2), appearance (2), upload (2), file (2), changes (2), links (2), read (2), subsection (2), log (2), create (2), account (2), donate (2), menu (2), add, topic, mobile, cookie, statement, developers, code, conduct, legal, safety, contacts, disclaimers, available, site, you, agree, registered, trademark, profit, organization, wikimedia, foundation, inc, creative, commons, attribution, sharealike, license, rendered, parsoid, last, edited, july, 2026, utc, hidden, november, citations, april, matches, algebra, https, org, index, php, ridge_regression, oldid, 1365109207, tikhonov_regularization, topics, outline, moving, identification, differentiation, calibration, nodes, orthogonal, quadrature, integration, surface, methodology, experiments, frisch, waugh, lovell, specification, bic, aic, mallows, stepwise, exploration, aov, growth, predictor, structure, confounding, variable, kendall, spearman, rho, pearson, product, dependence, taddy, matt, mcgraw, hill, 104, 260, 45277, combining, economics, optimize, automate, accelerate, decisions, saleh, ehsanes, arashi, mohammad, kibria, golam, 118, 64461, teukolsky, vetterling, flannery, 2007, 3rd, 521, 88068, recipes, art, computing, kress, rainer, 98408, 1985, harvard, 674, 00560, amemiya, takeshi, vogel, curtis, 2002, 550, greenberg, edward, webster, charles, 1983, 213, 471, 09077, 207, bridge, gompper, gerhard, sabass, benedikt, denoising, user, friendly, software, package, 1016, cpc, 2020cophc, 25607313h, 01377, 256, tarantola, albert, 792, per, christian, jan, usa, 403, deficient, aspects, lavrentiev, 1967, improperly, koch, lukas, post, hoc, unfolded, measurements, 10021, 1088, 1748, 0221, p10021, 2022jinst, 17p0021k, 2207, 02125, instrumentation, guan, naiyang, tao, dacheng, luo, zhigang, yuan, online, nonnegative, stochastic, 1099, 8755408, 24807135, 1109, tnnls, 2197827, 2012itnnl, 1087g, 1087, ieee, transactions, neural, networks, systems, fan, chang, hsieh, wang, lin, 2008, liblinear, library, 1874, 1871, research, 2004, proc, icml, feature, rotational, invariance, andrew, roger, 425, 0040, 1706, issn, 1742207, 420, historical, 10488634, 1961, 392, 1137, 0109031, chemical, progress, 35368397, 1145, 321105, 321114, acm, certain, kind, nikolaevich, leonov, aleksandr, chapman, hall, 412, 78660, goncharsky, stepanov, june, 1995, 7923, 3583, arsenin, washington, winston, 470, 99124, translated, incorrectly, formulated, 1035, soviet, 1963, решении, некорректно, поставленных, методе, регуляризации, 504, 501, 151, 1943, stability, 198, archived, 195, устойчивости, обратных, treatment, botev, zdravko, kroese, dirk, taimre, thomas, 2025, 2nd, 267, 268, 032, 48868, golub, heath, 223, 10489751, 215, good, 1990, spline, observational, 1990smod, conf, cbms, nsf, regional, conference, van, wieringen, wessel, 2021, lecture, stat, 1509, 09169, khalaf, ghadban, shukur, ghazi, 1182, 122983724, 1081, sta, 200056836, 1177, jolliffe, 2006, media, 178, 22440, component, 2003, fifth, mit, 206, 262, 61183, 205, guide, kennedy, peter, hilt, seegrist, 5962, bhl, 68934, program, calculating, estimates, modifications, discoveries, variously, levenberg, marquardt, algorithm, twomey, miller, weight, decay, elastic, net, another, lasso, assumption, uncorrelatedness, still, assumes, entails, minimal, unbiased, homoscedasticity, normality, although, look, artificial, indeed, seems, justified, must, necessarily, introduce, statistically, sometimes, simplicity, made, same, assumed, according, bayes, probable, probability, mmse, observation, introduces, all, observed, two, isotropic, finally, weights, analyzed, way, elsewhere, demonstrates, similar, representation, vdu, discretization, formulate, infinite, interpret, domain, range, bounded, invertible, self, adjoint, spaces, compact, applicable, advantageous, better, situations, avoid, proposed, symmetric, definite, set, leading, minimizing, mikhail, because, appears, considered, whitening, cholesky, distributions, transformation, seek, stand, compare, mahalanobis, distance, fits, perturbation, small, regularise, any, presented, best, detailed, knowledge, comes, regularised, bv_, simply, adds, objective, optimization, optimisation, place, yields, presence, neq, aside, support, machines, give, particular, desirable, properties, suitably, scalar, giving, solutions, operators, enforce, smoothness, believed, mostly, continuous, improves, conditioning, enabling, direct, treating, varied, scale, unregularized, provided, exists, hermitian, complex, augmented, fourier, difference, norms, however, satisfies, does, said, leads, world, phenomena, direction, maps, therefore, solving, operates, undesirable, tendency, amplifying, largest, reverse, they, were, smallest, addition, implicitly, nullifies, every, element, reconstructed, null, allowing, seeks, compactly, euclidean, cdot, eigenvalues, low, filters, underdetermined, overdetermined, suppose, wish, sizes, even, invented, independently, became, through, works, david, authors, finite, expounded, took, manus, interpreted, refers, path, kriging, alternative, approaches, include, restricted, discrepancy, principle, replacing, gives, called, widehat, qquad, course, plugging, natural, ones, expectations, treat, hence, test, enjoys, sample, risk, precisely, var, assume, consider, times, effective, degrees, freedom, common, minimizer, loss, generalizations, proved, sense, minimizes, grace, usually, needs, accomplished, either, procedure, follows, denoting, pseudoinverse, eventually, becomes, converges, denoted, binding, alleviated, adding, thereby, decreasing, extra, denominator, serves, shifting, lagrangian, shows, fact, there, relationship, between, know, heuristically, strategies, below, lagrange, multiplier, regressand, near, developed, imprecision, multicollinear, creating, precise, estimate, previously, introduced, papers, scenarios, been, fields, including, chemistry, particularly, useful, mitigate, commonly, occurs, numbers, improved, exchange, tolerable, amount, tradeoff, multicollinearity, coefficients, spectral, absolute, deviations, negative, angle, multilevel, binary, part, redirected, free, encyclopedia, item, projects, printable, download, print, export, switch, legacy, parser, shortened, url, cite, permanent, link, what, actions, english, talk, 한국어, 日本語, français, فارسی, español, català, বাংলা, personal, pages, recent, community, learn, help, contribute, current, events, navigation, jump, content,

Text of the page (random words):
ering 2 it is a widely used method of regularization of ill posed problem inverse problems a it is particularly useful to mitigate the problem of multicollinearity in linear regression which commonly occurs in models with large numbers of parameters 3 in general the method provides improved efficiency in parameter estimation problems in exchange for a tolerable amount of bias see bias variance tradeoff 4 the theory was first introduced by hoerl and kennard in 1970 in their technometrics papers ridge regressions biased estimation of nonorthogonal problems and ridge regressions applications in nonorthogonal problems 5 6 1 ridge regression was developed as a possible solution to the imprecision of least square estimators when linear regression models have some multicollinear highly correlated independent variables by creating a ridge regression estimator rr this provides a more precise ridge parameters estimate as its variance and mean square estimator are often smaller than the least square estimators previously derived 7 2 overview edit in the ordinary least squares solution of y x β ε displaystyle mathbf y mathbf x boldsymbol beta boldsymbol varepsilon the problem of a near singular moment matrix x t x displaystyle mathbf x mathsf t mathbf x is alleviated by adding positive elements to the diagonals thereby decreasing its condition number compared to the ordinary least squares estimator the simple ridge estimator has an extra term λ i displaystyle lambda mathbf i in the denominator β λ x t x λ i 1 x t y displaystyle hat boldsymbol beta _ lambda left mathbf x mathsf t mathbf x lambda mathbf i right 1 mathbf x mathsf t mathbf y where y displaystyle mathbf y is the regressand or response vector x displaystyle mathbf x is the design matrix i displaystyle mathbf i is the identity matrix and the ridge or tikhonov regularization parameter λ 0 displaystyle lambda geq 0 serves as the constant shifting the diagonals of the moment matrix 8 it can be shown that this estimator is the solution to the least squares problem subject to the constraint β t β c displaystyle boldsymbol beta mathsf t boldsymbol beta c which can be expressed as a lagrangian minimization argmin β y x β 2 λ β t β c displaystyle text argmin _ boldsymbol beta mathbf y mathbf x boldsymbol beta 2 lambda left boldsymbol beta mathsf t boldsymbol beta c right which shows that λ displaystyle lambda is nothing but the lagrange multiplier of the constraint 9 in fact there is a one to one relationship between c displaystyle c and λ displaystyle lambda and since in practice we do not know c displaystyle c we define λ displaystyle lambda heuristically or find it via additional data fitting strategies see determination of the tikhonov parameter below note that as λ 0 displaystyle lambda downarrow 0 the constraint eventually becomes non binding and the ridge estimator converges to the minimum norm ordinary least squares estimator here denoted as β β 0 displaystyle hat boldsymbol beta hat boldsymbol beta _ 0 lim λ 0 β λ x y β 0 displaystyle lim _ lambda downarrow 0 hat boldsymbol beta _ lambda mathbf x mathbf y hat boldsymbol beta _ 0 with x displaystyle mathbf x denoting the pseudoinverse of x displaystyle mathbf x determination of the tikhonov parameter edit the optimal regularization parameter λ displaystyle lambda is usually unknown and in practice needs to be estimated typically a data driven choice for the tikhonov regularization parameter λ displaystyle lambda is accomplished either via cross validation or via a plug in procedure as follows generalized cross validation estimator edit a common data driven choice for λ displaystyle lambda is the minimizer of the cross validation loss or its generalizations for example grace wahba proved that the optimal parameter in the sense of generalized cross validation minimizes 10 11 g rss τ 2 x β y 2 tr i x x t x λ 2 i 1 x t 2 displaystyle g frac operatorname rss tau 2 frac left mathbf x hat boldsymbol beta mathbf y right 2 left operatorname tr left mathbf i mathbf x left mathbf x mathsf t mathbf x lambda 2 mathbf i right 1 mathbf x mathsf t right right 2 where rss displaystyle operatorname rss is the residual sum of squares and τ displaystyle tau is the effective number of degrees of freedom plug in estimator edit assume that x displaystyle mathbf x is an n p displaystyle n times p matrix and define the matrix ω x x n displaystyle omega mathbf x top mathbf x n then consider the following choice for the tikhonov regularization parameter λ ς 2 t r ω β ω β 3 ς 2 t r ω 2 n displaystyle lambda frac varsigma 2 mathrm tr omega boldsymbol beta top omega boldsymbol beta 3 varsigma 2 mathrm tr omega 2 n where ς 2 displaystyle varsigma 2 is the variance of the noise ε y x β displaystyle boldsymbol varepsilon mathbf y mathbf x boldsymbol beta that is v a r ε ς 2 i displaystyle mathrm var boldsymbol varepsilon varsigma 2 mathbf i it can be shown 12 that the ridge estimator β λ displaystyle hat boldsymbol beta _ lambda enjoys smaller expected in sample risk than the minimum norm least squares estimator β 0 x y displaystyle hat boldsymbol beta _ 0 mathbf x mathbf y more precisely e y x β 0 2 e y x β λ 2 ς 2 n λ t r ω displaystyle mathbb e mathbf y mathbf x hat boldsymbol beta _ 0 2 geq mathbb e mathbf y mathbf x hat boldsymbol beta _ lambda 2 frac varsigma 2 n lambda mathrm tr omega where the expectations treat x displaystyle mathbf x as fixed and y displaystyle mathbf y is test response data independent from y displaystyle mathbf y and hence independent from the estimators β 0 displaystyle hat boldsymbol beta _ 0 and β λ displaystyle hat boldsymbol beta _ lambda of course in practice the formula for λ displaystyle lambda is used by plugging in statistical estimators for the unknown parameters β displaystyle boldsymbol beta and ς 2 displaystyle varsigma 2 when n p displaystyle n p the most natural estimators for these parameters are the usual least squares ones β x y ς 2 y x β 2 n p displaystyle hat boldsymbol beta mathbf x mathbf y qquad hat varsigma 2 frac mathbf y mathbf x hat boldsymbol beta 2 n p replacing the unknown β ς 2 displaystyle boldsymbol beta varsigma 2 in the formula for λ displaystyle lambda with the corresponding β ς 2 displaystyle hat boldsymbol beta hat varsigma 2 thus gives the so called plug in estimator λ displaystyle widehat lambda for the optimal λ displaystyle lambda alternative approaches to the data driven selection of the tikhonov regularization parameter include the discrepancy principle l curve method 13 restricted maximum likelihood history edit tikhonov regularization was invented independently in many different contexts it became widely known through its application to integral equations in the works of andrey tikhonov 14 15 16 17 18 and david l phillips 19 some authors use the term tikhonov phillips regularization the finite dimensional case was expounded by arthur e hoerl who took a statistical approach 20 and by manus foster who interpreted this method as a wiener kolmogorov kriging filter 21 following hoerl it is known in the statistical literature as ridge regression 22 named after ridge analysis ridge refers to the path from the constrained maximum 23 tikhonov regularization for linear equations edit suppose that for a known real matrix a displaystyle a and vector b displaystyle mathbf b we wish to find a vector x displaystyle mathbf x such that a x b displaystyle a mathbf x mathbf b where x displaystyle mathbf x and b displaystyle mathbf b may be of different sizes and a displaystyle a may even be non square the standard approach is ordinary least squares linear regression clarification needed however if no x displaystyle mathbf x satisfies the equation or more than one x displaystyle mathbf x does that is the solution is not unique the problem is said to be ill posed in such cases ordinary least squares estimation leads to an overdetermined or more often an underdetermined system of equations most real world phenomena have the effect of low pass filters clarification needed in the forward direction where a displaystyle a maps x displaystyle mathbf x to b displaystyle mathbf b therefore in solving the inverse problem the inverse mapping operates as a high pass filter that has the undesirable tendency of amplifying noise eigenvalues singular values are largest in the reverse mapping where they were smallest in the forward mapping in addition ordinary least squares implicitly nullifies every element of the reconstructed version of x displaystyle mathbf x that is in the null space of a displaystyle a rather than allowing for a model to be used as a prior for x displaystyle mathbf x ordinary least squares seeks to minimize the sum of squared residuals which can be compactly written as a x b 2 2 displaystyle left a mathbf x mathbf b right _ 2 2 where 2 displaystyle cdot _ 2 is the euclidean norm in order to give preference to a particular solution with desirable properties a regularization term can be included in this minimization a x b 2 2 γ x 2 2 a x b 2 2 displaystyle left a mathbf x mathbf b right _ 2 2 left gamma mathbf x right _ 2 2 left mathcal a mathbf x mathcal b right _ 2 2 where a a γ displaystyle mathcal a begin pmatrix a gamma end pmatrix and b b 0 displaystyle mathcal b begin pmatrix mathbf b boldsymbol 0 end pmatrix for some suitably chosen tikhonov matrix γ displaystyle gamma in many cases this matrix is chosen as a scalar multiple of the identity matrix γ α i displaystyle gamma alpha i giving preference to solutions with smaller norms this is known as l 2 regularization 24 in other cases high pass operators e g a difference operator or a weighted fourier operator may be used to enforce smoothness if the underlying vector is believed to be mostly continuous this regularization improves the conditioning of the problem thus enabling a direct numerical solution treating it as an ordinary least squares problem with augmented matrices a displaystyle mathcal a and b displaystyle mathcal b the solution is x a t a 1 a t b a t a γ t γ 1 a t b displaystyle hat mathbf x mathcal a mathsf t mathcal a 1 mathcal a mathsf t mathbf mathcal b a mathsf t a gamma mathsf t gamma 1 a mathsf t mathbf b the effect of regularization may be varied by the scale of the matrix γ displaystyle gamma for γ 0 displaystyle gamma 0 this reduces to the unregularized least squares solution provided that a t a 1 exists note that in case of a complex matrix a displaystyle a as usual the transpose a t displaystyle a mathsf t has to be replaced by the hermitian transpose a h displaystyle a mathsf h l 2 regularization is used in many contexts aside from linear regression such as classification with logistic regression or support vector machines 25 and matrix factorization 26 application to existing fit results edit since tikhonov regularization simply adds a quadratic term to the objective function in optimization problems it is possible to do so after the unregularised optimisation has taken place e g if the above problem with γ 0 displaystyle gamma 0 yields the solution x 0 displaystyle hat mathbf x _ 0 the solution in the presence of γ 0 displaystyle gamma neq 0 can be expressed as x b x 0 displaystyle hat mathbf x b hat mathbf x _ 0 with the regularisation matrix b a t a γ t γ 1 a t a displaystyle b left a mathsf t a gamma mathsf t gamma right 1 a mathsf t a if the parameter fit comes with a covariance matrix of the estimated parameter uncertainties v 0 displaystyle v_ 0 then the regularisation matrix will be b v 0 1 γ t γ 1 v 0 1 displaystyle b v_ 0 1 gamma mathsf t gamma 1 v_ 0 1 and the regularised result will have a new covariance v b v 0 b t displaystyle v bv_ 0 b mathsf t in the context of arbitrary likelihood fits this is valid as long as the quadratic approximation of the likelihood function is valid this means that as long as the perturbation from the unregularised result is small one can regularise any result that is presented as a best fit point with a covariance matrix no detailed knowledge of the underlying likelihood function is needed 27 generalized tikhonov regularization edit for general multivariate normal distributions for x displaystyle mathbf x and the data error one can apply a transformation of the variables to reduce to the case above equivalently one can seek an x displaystyle mathbf x to minimize a x b p 2 x x 0 q 2 displaystyle left a mathbf x mathbf b right _ p 2 left mathbf x mathbf x _ 0 right _ q 2 where we have used x q 2 displaystyle left mathbf x right _ q 2 to stand for the weighted norm squared x t q x displaystyle mathbf x mathsf t q mathbf x compare with the mahalanobis distance in the bayesian interpretation p displaystyle p is the inverse covariance matrix of b displaystyle mathbf b x 0 displaystyle mathbf x _ 0 is the expected value of x displaystyle mathbf x and q displaystyle q is the inverse covariance matrix of x displaystyle mathbf x the tikhonov matrix is not explicitly included because the corresponding regularization term γ x x 0 q 2 displaystyle left gamma mathbf x mathbf x _ 0 right _ q 2 reduces to above with γ x 0 x 0 displaystyle gamma mathbf x _ 0 mathbf x _ 0 and q γ t q γ displaystyle q gamma t q gamma for normal regularization where q i displaystyle q i the tikhonov matrix then appears in the cholesky factorization q γ t γ displaystyle q gamma mathsf t gamma and is considered a whitening filter this generalized problem has an optimal solution x displaystyle hat mathbf x which can be written explicitly using the formula x a t p a q 1 a t p b q x 0 x 0 a t p a q 1 a t p b a x 0 displaystyle mathbf hat mathbf x left a mathsf t pa q right 1 left a mathsf t p mathbf b q mathbf x _ 0 right mathbf x _ 0 left a mathsf t pa q right 1 left a mathsf t p left mathbf b a mathbf x _ 0 right right lavrentyev regularization edit in some situations one can avoid using the transpose a t displaystyle a mathsf t as proposed by mikhail lavrentyev 28 for example if a displaystyle a is symmetric positive definite i e a a t 0 displaystyle a a mathsf t 0 so is its inverse a 1 displaystyle a 1 which can thus be used to set up the weighted norm squared x p 2 x t a 1 x displaystyle left mathbf x right _ p 2 mathbf x mathsf t a 1 mathbf x in the generalized tikhonov regularization leading to minimizing a x b a 1 2 x x 0 q 2 displaystyle left a mathbf x mathbf b right _ a 1 2 left mathbf x mathbf x _ 0 right _ q 2 or equivalently up to a constant term x t a q x 2 x t b q x 0 displaystyle mathbf x mathsf t left a q right mathbf x 2 mathbf x mathsf t left mathbf b q mathbf x _ 0 right this minimization problem has an optimal solution x displaystyle mathbf x which can be written explicitly using the formula x a q 1 b q x 0 displaystyle mathbf x left a q right 1 left mathbf b q mathbf x _ 0 right which is nothing but the solution of the generalized tikhonov problem where a a t p 1 displaystyle a a mathsf t p 1 the lavrentyev regularization if applicable is advantageous to the original tikhonov regul...