Meta tags:
description=Posts about Statistics & Econometrics written by Lars Syll;

Headings (most frequently used words):

the, how, econometrics, in, to, is, and, are, not, why, make, recent, comments, lars, syll, finding, late, student, stuff, credible, credibility, revolution, randomness, representativeness, same, what, does, randomisation, guarantee, nothing, model, reification, statistics, democracy, outperforms, dictatorship, rethinking, statistical, toolkit, playing, tennis, with, net, down, do, properly, limits, of, econometric, testing, progress, quantitative, analysis, your, dates, attractive, jerks, valid, causal, inferences, smoking, really, good, for, babies, overcontrolling, iii, posts, policy, reading, list, categories, archives,

Text of the page (most frequently used words):
the (368), and (163), that (136), are (93), not (57), for (51), share (46), new (46), opens (45), window (45), they (42), assumptions (42), data (39), econometrics (38), from (38), but (37), may (36), with (35), have (31), #causal (31), bluesky (30), facebook (30), can (30), only (29), economic (29), this (28), one (27), model (26), when (26), statistical (25), jun (23), which (23), statistics (22), testing (22), results (22), mar (21), 2026 (21), randomisation (21), how (21), about (21), models (20), our (20), often (20), sample (20), apr (19), jan (19), what (19), econometric (19), random (19), will (17), has (17), more (17), such (17), target (17), posted (16), its (16), real (16), make (16), test (16), jul (15), aug (15), sep (15), oct (15), nov (15), dec (15), feb (15), than (15), between (15), world (15), effects (15), comments (14), being (14), does (14), smoking (14), inferences (14), systems (14), average (14), 2019 (13), you (13), birth (13), weight (13), study (13), were (13), treatment (13), experiments (13), their (13), evidence (13), 2012 (12), 2013 (12), 2014 (12), 2015 (12), 2016 (12), 2017 (12), 2018 (12), 2020 (12), 2021 (12), 2022 (12), 2023 (12), 2024 (12), 2025 (12), low (12), without (12), these (12), however (12), effect (12), must (12), whether (12), always (12), many (12), analysis (12), lönebildning (11), och (11), arbetslöshet (11), nothing (11), your (11), there (11), also (11), questions (11), simply (11), because (11), population (11), methods (11), important (11), even (11), lars (10), 2011 (10), why (10), out (10), all (10), value (10), use (10), those (10), estimate (10), comment (9), other (9), randomness (9), off (9), made (9), babies (9), non (9), causes (9), never (9), bias (9), them (9), different (9), under (9), variables (9), both (9), people (9), given (9), significant (9), economics (8), representativeness (8), control (8), reality (8), structure (8), cannot (8), underlying (8), independent (8), hypotheses (8), error (8), civilians (8), making (8), mathematical (8), know (8), set (8), experimental (8), science (7), problem (7), genetic (7), group (7), gives (7), two (7), risk (7), assumption (7), valid (7), treated (7), would (7), should (7), theoretical (7), conclusions (7), work (7), any (7), most (7), properties (7), some (7), phenomena (7), econometricians (7), terms (7), knowledge (7), beyond (7), far (7), single (7), experiment (7), diversity (7), representative (7), class (7), theory (6), like (6), guarantee (6), research (6), result (6), researchers (6), since (6), true (6), just (6), processes (6), relevant (6), wrong (6), here (6), selection (6), white (6), who (6), while (6), economists (6), regression (6), still (6), interest (6), estimand (6), empirical (6), social (6), modelling (6), technical (6), actually (6), come (6), rarely (6), mechanisms (6), young (6), over (6), export (6), validity (6), heterogeneity (6), chance (6), she (6), arbetslöshetssiffran (5), som (5), styr (5), remember (5), limited (5), reification (5), credibility (5), revolution (5), confounding (5), collider (5), conclusion (5), controlling (5), into (5), likely (5), cause (5), within (5), itself (5), draw (5), samples (5), every (5), case (5), provide (5), stopped (5), quantity (5), simple (5), outcomes (5), variable (5), dating (5), individual (5), mean (5), someone (5), rather (5), precise (5), purpose (5), been (5), able (5), theories (5), warrant (5), learn (5), actual (5), fictional (5), kind (5), hold (5), ways (5), process (5), according (5), based (5), correct (5), net (5), standard (5), assignment (5), leave (5), prediction (5), ideal (5), cases (5), works (5), system (5), boys (5), gender (5), whole (5), method (5), prefer (5), syll (4), fred (4), especially (4), time (4), credible (4), precision (4), point (4), false (4), researcher (4), example (4), understanding (4), baby (4), mortality (4), conditions (4), first (4), too (4), much (4), really (4), difficult (4), goal (4), inference (4), serious (4), force (4), used (4), approach (4), very (4), outcome (4), personality (4), conversely (4), assume (4), nor (4), common (4), quantitative (4), hard (4), framework (4), question (4), apply (4), show (4), explain (4), strong (4), build (4), supposed (4), causality (4), intervention (4), unfortunately (4), comes (4), properly (4), need (4), dgp (4), almost (4), produced (4), right (4), statistically (4), merely (4), playing (4), tennis (4), down (4), tests (4), observation (4), design (4), per (4), change (4), theorem (4), collective (4), rule (4), stable (4), small (4), randomised (4), rcts (4), tell (4), natural (4), settings (4), populations (4), ice (4), cream (4), selected (4), teacher (4), three (4), girls (4), children (4), view (3), recent (3), follow (3), leaves (3), policy (3), democracy (3), same (3), paradox (3), finding (3), yours (3), identification (3), everything (3), begin (3), good (3), creates (3), demonstrates (3), genuine (3), mother (3), less (3), mothers (3), specific (3), lbw (3), correlation (3), understand (3), completely (3), anomalies (3), want (3), find (3), lead (3), reducing (3), big (3), themselves (3), yield (3), perspective (3), fundamental (3), treat (3), tested (3), unwarranted (3), looking (3), comparing (3), officers (3), detain (3), suppose (3), observed (3), groups (3), controls (3), serve (3), trade (3), might (3), attractiveness (3), nice (3), attractive (3), large (3), shift (3), clearly (3), fit (3), require (3), claim (3), best (3), narrow (3), needed (3), paper (3), progress (3), fact (3), builds (3), reasons (3), economies (3), central (3), ability (3), values (3), turns (3), standardly (3), probability (3), distributions (3), things (3), worse (3), trying (3), rely (3), distributed (3), additive (3), linear (3), constant (3), coefficients (3), invariant (3), structural (3), seldom (3), where (3), generating (3), requirement (3), relationship (3), extent (3), through (3), course (3), well (3), generally (3), constructed (3), usually (3), means (3), answers (3), become (3), don (3), david (3), particular (3), obtain (3), check (3), ask (3), sense (3), institutional (3), measure (3), confidence (3), capture (3), answer (3), procedures (3), sampling (3), another (3), little (3), help (3), great (3), shows (3), leverage (3), robust (3), run (3), cent (3), highly (3), advantage (3), perspectives (3), students (3), estimates (3), each (3), purely (3), studies (3), circumstances (3), trials (3), findings (3), was (3), ideally (3), quasi (3), repeated (3), balance (3), trial (3), setting (3), external (3), across (3), happen (3), perfectly (3), names (3), yet (3), sausages (3), equal (3), deductive (3), required (2), log (2), sign (2), subscribed (2), subscribe (2), already (2), wordpress (2), com (2), now (2), blog (2), list (2), arbetslösheten (2), inte (2), wiklund (2), milch (2), ones (2), irrelevant (2), outperforms (2), dictatorship (2), nancy (2), cartwright (2), unbiasedness (2), late (2), student (2), stuff (2), overcontrolling (2), iii (2), truly (2), frequently (2), way (2), opening (2), back (2), door (2), paths (2), creating (2), newborns (2), illusion (2), importance (2), behind (2), naive (2), performed (2), consequently (2), did (2), higher (2), defects (2), powerful (2), factor (2), furthermore (2), infant (2), crucially (2), his (2), problems (2), identify (2), relationships (2), manageable (2), art (2), otherwise (2), traditional (2), relying (2), phantasmagoric (2), realistic (2), biased (2), conditioning (2), crime (2), apples (2), necessary (2), read (2), relations (2), individuals (2), rates (2), minority (2), observable (2), differences (2), sets (2), encounters (2), estimator (2), long (2), affects (2), civilian (2), majority (2), racial (2), observe (2), nonwhite (2), fundamentally (2), analyst (2), against (2), conclude (2), exists (2), complex (2), better (2), sometimes (2), correlations (2), warning (2), address (2), possible (2), sound (2), leaving (2), depends (2), added (2), caused (2), reason (2), unattractive (2), successful (2), once (2), focus (2), found (2), forcefully (2), estimated (2), judge (2), chosen (2), leap (2), sciences (2), author (2), call (2), sociologists (2), transparent (2), compelling (2), perfect (2), future (2), argument (2), makes (2), exclude (2), seem (2), built (2), inherently (2), assumed (2), essential (2), tractability (2), information (2), harmfully (2), becomes (2), questionable (2), scientific (2), hinges (2), validate (2), unarticulated (2), apparatus (2), blind (2), eye (2), unfulfilled (2), joint (2), pulled (2), urn (2), awkward (2), position (2), device (2), verified (2), reliable (2), telling (2), preferred (2), infer (2), disturbance (2), identically (2), functions (2), parameters (2), exogenous (2), informed (2), referring (2), relation (2), homomorphism (2), entities (2), determined (2), establish (2), met (2), holds (2), represent (2), complete (2), whose (2), methodological (2), rigour (2), taking (2), advanced (2), generated (2), superpopulation (2), nowadays (2), evident (2), unless (2), forget (2), published (2), obtained (2), produces (2), raises (2), own (2), looks (2), least (2), quality (2), treating (2), towards (2), using (2), including (2), historical (2), practice (2), running (2), treats (2), object (2), economy (2), shaped (2), measurement (2), match (2), easy (2), give (2), useful (2), compatible (2), performing (2), therefore (2), interested (2), probabilistic (2), basic (2), see (2), admit (2), identifiable (2), perhaps (2), put (2), deal (2), technique (2), instead (2), uses (2), fragile (2), journals (2), then (2), imagine (2), insignificant (2), originally (2), closer (2), called (2), others (2), provides (2), harness (2), decision (2), helps (2), conventional (2), generates (2), power (2), experiences (2), predictions (2), differ (2), guesses (2), differently (2), measurable (2), strictly (2), gap (2), engineered (2), regularities (2), laws (2), established (2), ever (2), rare (2), limiting (2), closely (2), demonstrate (2), explanations (2), sharply (2), simplifying (2), measured (2), defined (2), aspect (2), thus (2), guarantees (2), something (2), indeed (2), background (2), factors (2), controlled (2), certainty (2), appropriate (2), extrapolations (2), accidental (2), justified (2), rigorous (2), contexts (2), involve (2), emerge (2), infinite (2), substantial (2), wider (2), 100 (2), alone (2), except (2), claims (2), rests (2), end (2), distinct (2), tells (2), feature (2), consider (2), hat (2), split (2), raise (2), hands (2), main (2), preference (2), opinions (2), procedure (2), six (2), randomly (2), reflect (2), ensures (2), describes (2), remain (2), adaptive (2), away (2), feedback (2), restrictive (2), epistemically (2), isomorphic (2), closed (2), gold (2), experimentation (2), finite (2), discontinuity (2), getting (2), website, name, email, write, loading, collapse, bar, manage, subscriptions, site, reader, report, content, privacy, account, join, 606, subscribers, feeds, entries, 112, archives, 707, varia, 542, methodology, 998, 390, politics, society, 301, education, school, 168, categories, reading, fredtorssander, torssander, netiquette, anonymous, pseudo, argumentations, abusive, language, links, please, full, professor, respond, contributions, philosophy, sverige, statsskulden, problemet, ett, välförtjänt, bottenrekord, audioholic, summer, stein, lesson, love, secret, efter, studenten, personal, den, ekonomiska, politiken, posts, next, page, bears, repeating, mistaking, requires, solution, inevitably, hit, wasn, last, week, leads, bizarre, appears, harmful, mechanically, choosing, powerfully, underscores, adjustments, map, transform, harm, perceived, benefit, smoked, easily, attributed, severe, anomaly, smoke, explained, implying, likelihood, defect, death, rate, trouble, begins, decides, restricting, inadvertently, rise, genetics, appreciate, during, pregnancy, her, separately, direct, smokers, examples, concerns, increases, incorrect, beneficial, judea, pearl, discusses, arise, thoughtlessly, try, quest, book, interesting, analyses, frequent, deeply, problematic, convenience, hypothesis, dubious, risky, manoeuvre, impression, outset, due, post, bank, robbers, black, committed, comparison, seeking, proprietary, administrative, datasets, dean, knox, lowe, jonathan, mummolo, traditionally, analysts, computing, difference, violence, term, naïve, further, interpretation, mediator, race, stop, comparable, exhibited, follows, committing, assault, potentially, warranting, regardless, behavior, amounts, greater, proportion, differential, physical, threat, pose, naïvely, anti, yielding, erroneous, portrait, discrimination, fantasy, source, add, studying, basis, analogy, decades, discipline, developed, tendency, omitted, adding, logic, seems, biases, putting, fix, mechanical, confounder, removes, type, emerges, probably, daters, pool, negative, traits, guessing, reasonable, necessarily, general, subset, currently, choice, date, helpful, intuition, spurious, induced, modern, behaviour, dates, jerks, argued, authors, suffers, oversight, devote, excessive, attention, neglecting, specify, defining, stated, utterly, ambiguous, ian, lundberg, rebecca, johnson, brandon, stewart, sociology, stands, willingness, tackle, burying, obscures, decisions, confuses, link, uncomfortable, ambiguity, intentions, worst, mislead, ambitions, explicit, goals, motivate, projects, believe, develops, relies, recognized, contribution, stage, explore, simpler, obfuscation, capacity, eliminate, invalid, business, users, toolbox, blindness, incomplete, presupposition, argumentation, intended, studied, structures, play, roles, bridge, failing, unrecognised, lost, misleading, largely, anything, societies, compatibility, fictitious, storytelling, admitting, several, note, characterizing, pieces, intra, activity, reveals, matrix, numbers, judged, rules, mathematics, contrast, exist, concerned, satisfy, obviate, contrary, characterization, piece, validly, interpreted, providing, epistemic, access, plausible, aspects, ostensibly, aware, literature, couched, perceive, entirety, essentially, precipitates, limits, verisimilitude, encountered, part, degree, deceptive, appearance, technically, sophisticated, discussed, swept, carpet, assuming, coin, flips, imaginary, get, asking, self, truism, among, expect, take, arguments, seriously, backed, legions, theorems, proved, heaps, fiction, masquerading, reaching, supported, dave, giles, peer, reviewed, practices, followed, keep, mind, fraction, publishing, computer, code, application, hendry, pre, issues, else, analyse, doesn, sure, exactly, relating, numerically, economically, yourself, plot, moving, tool, learning, raised, behave, specificities, contingencies, available, reporting, fits, goodness, final, place, context, raw, unfiltered, facts, definitions, imputations, classification, choices, embody, gdp, figures, unemployment, conventions, verifying, success, artefact, confrontation, targeted, ultimate, assess, evaluate, haavelmo, gave, having, colander, situations, instrument, debating, shortcomings, gets, response, maybe, isn, thing, driven, sensitive, concentrated, recommends, scott, cunningham, 148, 174, treatments, switching, trustworthy, avoid, misled, influential, outliers, mixtape, examining, fifty, top, highlights, reproducibility, crisis, flip, original, matters, level, remove, leveraged, turned, survive, look, alwyn, article, channeling, fisher, insignificance, seemingly, 134, offers, focuses, concept, points, bigger, impact, qje, rethinking, toolkit, democratic, dictatorships, improving, prove, superior, inclusive, participatory, political, outperform, leader, elite, challenges, rationality, efficiency, matter, fairness, representation, tangible, resilient, innovative, wide, range, thinking, accuracy, members, judgements, sufficiently, diverse, effectively, cancels, producing, truth, individually, relatively, inaccurate, 997, remarkably, close, number, explains, fill, jar, 1000, jelly, beans, inside, amanda, 980, bob, 1050, clark, 1020, 940, bringing, together, think, performance, belong, features, aim, metaphor, implies, messy, forgets, inherent, pure, governed, capacities, kinds, presuppose, atomistic, operate, mostly, changing, unstable, outside, man, nomological, machines, existent, aspires, needs, examined, isolate, ceteris, paribus, sander, greenland, thought, toy, restricted, motivated, translated, via, deductions, logical, covariate, levels, surveys, entered, logically, imperfect, applications, widely, applied, misinterpreted, fall, short, establishing, securing, observational, vulnerable, various, forms, sufficient, neither, exportable, generalisable, somewhere, reach, describing, occurred, past, public, held, implemented, closure, straightforward, directly, identified, shown, possess, severely, faced, randomistas, underestimate, interaction, attempting, transfer, periods, internal, affecting, millions, year, sequence, randomisations, reassurance, conducted, occur, confident, drawn, amount, offset, gain, importantly, exhibits, considerably, extrapolating, carried, manner, difficulty, conceal, zero, experience, anyone, considering, undergo, informative, subjects, certainly, extremely, proper, compensate, derived, generalise, ensure, arrive, certain, alternative, simplistic, behalf, exaggerated, speaking, conducting, said, allows, stronger, idea, reduces, systematic, thereby, influence, valuable, enables, calculate, margins, intervals, known, combines, sausage, ideals, unrepresentative, confusing, remains, nonetheless, mechanism, composition, fail, characteristics, carefully, quota, accurately, opposite, adopts, drawing, deliberately, selects, chooses, mirrors, influencing, food, track, words, applies, conscious, quickly, distracted, shy, pupils, demonstrating, let, classroom, asks, wants, select, stand, writes, slips, places, draws, yields, five, girl, strongly, overwhelmingly, heavy, male, ends, favouring, margin, though, child, had, fails, crucial, shapes, distribution, probabilities, produce, proportional, apart, everyday, reasoning, interchangeably, automatically, misunderstanding, enthusiasm, optimism, machinery, fanfare, surrounding, strikes, disproportionate, achievements, traded, correctly, specified, solves, evade, grounding, substantive, waiting, until, arrives, suspect, trivial, shall, sceptic, empiricism, uncritical, religious, devotion, techniques, laden, meant, replace, ontology, conceptualisation, tune, move, paradigm, analytical, abstracted, complexity, agents, interact, strategically, expectations, adjust, policies, propagate, networks, markets, path, dependent, vary, interventions, binary, growth, human, capital, formation, inequality, development, conform, multi, dimensional, interdependent, evolve, sutva, effective, evaluating, isolated, clear, separable, premise, either, role, closures, deduced, licences, despairingly, high, precisely, achieved, worth, rough, sophistication, former, axiomatic, matches, neat, mathematically, tractable, worlds, construct, software, argue, reached, convenient, unobserved, interactions, alter, exported, loops, distort, live, open, dynamic, partly, modifiers, identical, commensurable, pathways, considered, funding, bodies, demand, policymakers, trust, basically, manipulability, transitivity, separability, additivity, linearity, forth, deliver, although, contribute, presupposes, executed, protocol, hypothetical, repetitions, imbalances, says, homogeneity, defeats, allowing, angus, deaton, parametric, nature, seen, incredible, doubtful, approaches, designs, designed, carry, conviction, domain, ate, familiar, accurate, versus, special, requirements, instrumental, local, magnitude, fine, dictated, prioritizing, substance, ranking, hierarchy, targets, estimating, epidemiology, turn, alchemy, reverse, gretl, downloadable, publications, lps,

Text of the page (random words):
mogeneity is added to the list of assumptions which defeats the whole point of allowing for heterogeneity real target systems are seldom epistemically isomorphic to our axiomatic deductive models or systems that is the structure of reality rarely matches the neat closed mathematically tractable worlds we construct on paper or in our statistical software and even if they were isomorphic we would still have to argue for the external validity of the conclusions reached from within these epistemically convenient models causal evidence generated by randomisation procedures may be valid in closed models models in which all relevant variables are measured no unobserved confounding exists no interactions alter the effect when exported and no feedback loops distort the system but what we are usually interested in is causal evidence in the real target system in which we happen to live an open dynamic adaptive and only partly measurable world when does a conclusion established in population x hold for target population y only under very restrictive conditions when the distributions of all effect modifiers are the same when the causal structure is identical when the measurement procedures are commensurable and when no new causal pathways emerge these conditions are rarely if ever met in social science ideally controlled experiments tell us with certainty what causes what effects but only given the right closures that is a big only making appropriate extrapolations from ideal accidental natural or quasi experiments to different settings populations or target systems is not easy it works there is no evidence for it will work here causes deduced in an experimental setting still need to demonstrate that they come with an export warrant to the target population or system the causal background assumptions made must be justified without licences to export the value of rigorous and precise methods and of on average knowledge is despairingly small our statistical precision may be high but if we are precisely wrong about the wrong target we have achieved nothing of worth in the end a precise estimate of the wrong quantity is far worse than a rough estimate of the right one and randomisation for all its technical sophistication often gives us the former experiments are most effective when evaluating isolated causal relationships with clear separable effects they rely on a simplifying premise a policy intervention either produces an identifiable effect or it does not and the role of the experiment is to estimate that effect however many of the central questions in economics such as long run growth human capital formation inequality or institutional development do not conform to this structure these are inherently multi dimensional processes shaped by interdependent variables and feedback mechanisms that evolve over time in such settings the assumptions underlying experimental methods such as stable treatment effects sutva become harmfully restrictive what is often abstracted away is the complexity of real economic systems agents interact strategically expectations adjust and policies propagate through networks and markets in non linear ways outcomes are path dependent and causal effects may vary across contexts and over time as a result interventions rarely yield binary stable effects to address these phenomena we need an ontology and conceptualisation more in tune with real world economies we have to move beyond a narrow experimental paradigm towards an analytical framework that treats the economy as the complex adaptive system it really is so no i find it hard to share the enthusiasm and optimism about the value of quasi natural experiments and all the statistical econometric machinery that comes with them the fanfare surrounding the credibility revolution strikes me as disproportionate to the actual achievements we have perhaps traded one set of naive assumptions the assumption that our structural models are correctly specified for another the assumption that randomisation solves everything in both cases we evade the hard work of understanding causal heterogeneity testing for external validity and grounding our inferences in substantive social theory i suppose i am still waiting for the export warrant until it arrives and i suspect it never will except in the most trivial of cases i shall remain a sceptic not of empiricism per se but of the uncritical almost religious devotion to a particular set of empirical techniques that are no less assumption laden than the theories they were meant to replace share on facebook opens in new window facebook share on x opens in new window x share on bluesky opens in new window bluesky randomness and representativeness are not the same 17 jun 2026 at 14 24 posted in statistics econometrics leave a comment in research and everyday reasoning the terms random and representative are often used interchangeably many people assume that if a sample is selected at random it will automatically reflect the wider population and conversely that a representative sample must have been chosen randomly this is however a misunderstanding randomness and representativeness are two fundamentally distinct properties one describes a procedure the other describes an outcome they can and often do come apart let us consider a simple classroom experiment a teacher has a group of 20 children 10 boys and 10 girls the teacher asks the children whether they prefer sausages or ice cream and wants to select a small group of six children whose opinions can stand for the whole class she first uses a purely random method she writes all 20 names on slips of paper places them in a hat and draws six names without looking by chance the draw yields five boys and only one girl suppose that in this class the boys strongly prefer sausages while the girls overwhelmingly prefer ice cream the randomly selected group with its heavy male majority ends up favouring sausages by a large margin even though the class as a whole given its equal gender split might actually prefer ice cream this sample was random in every technical sense each child had an equal chance of being selected and chance alone determined the outcome yet it is clearly not representative of the class as a whole because it fails to reflect the crucial gender balance that shapes the underlying preference distribution randomness provides no guarantee of representativeness it merely ensures equal probability of selection and probabilities do not always produce proportional outcomes in a single draw now consider the opposite case the teacher adopts a different approach instead of drawing names from a hat she deliberately selects three boys and three girls to match the class s 50 50 gender split to do this she simply chooses the first three boys and the first three girls who raise their hands this sample perfectly mirrors the class in terms of gender and since gender is the main factor influencing food preference the sample s opinions will closely track those of the whole class it is in other words highly representative yet the procedure is clearly not random the teacher does not use chance she applies a conscious non probabilistic rule furthermore she only samples children who happen to raise their hands quickly which may exclude distracted or shy pupils thus a representative sample has been produced through a non random method demonstrating that representativeness does not require randomness what these two cases show is that randomness and representativeness answer different questions randomness tells us how a sample is selected it is a feature of the selection mechanism representativeness tells us how the sample looks like it is a feature of the sample s composition a random sample may by chance fail to capture those characteristics conversely a carefully constructed quota sample may capture them accurately without any use of chance in the end both properties are valuable for different reasons randomness is important because it enables statistical inference we can calculate margins of error and confidence intervals because the selection process is known representativeness is important because it gives us confidence that our conclusions apply to the population of interest the ideal study combines both but as my sausage and ice cream example shows in practice these are two distinct ideals a random sample can be unrepresentative and a representative sample can be non random confusing the two is a common error but it remains an error nonetheless share on facebook opens in new window facebook share on x opens in new window x share on bluesky opens in new window bluesky what does randomisation guarantee nothing 15 jun 2026 at 17 15 posted in statistics econometrics leave a comment the point of conducting a randomised experiment is often said to be that it allows researchers to make stronger causal inferences from observed correlations this claim rests on the idea that randomisation reduces systematic differences between treatment and control groups thereby limiting the influence of confounding factors the problem with this rather simplistic view of randomisation is that the claims made on its behalf are both exaggerated and strictly speaking false even if the assignment of subjects to treatment and control groups is perfectly random sample selection almost certainly is not except in extremely rare cases proper random assignment does not compensate for a biased sample if experimental results are derived from a non representative sample there is always a risk that the findings will not generalise beyond that setting what works there may not work here randomisation therefore does not guarantee or ensure that we arrive at the correct causal conclusion while it may help us rule out certain alternative explanations randomisation in itself guarantees nothing even if both sampling and assignment were carried out in an ideally random manner standard randomised experiments provide only average effects the difficulty is that an estimate of the true average causal effect may conceal important heterogeneity the average treatment effect may be zero while some individuals experience effects of 100 and others effects of 100 for anyone considering whether to undergo treatment such heterogeneity is likely to be far more informative than the average effect alone there is almost always a trade off between bias and precision in real world settings a small amount of bias may be more than offset by a substantial gain in precision more importantly when a population exhibits significant heterogeneity the average treatment effect estimated from a sample may differ considerably from the average treatment effect in the wider population if that is the case the value of extrapolating findings from trial samples to other populations becomes highly questionable since most real world experiments and trials involve only a single randomisation the properties that would emerge over an infinite sequence of repeated randomisations provide little reassurance about the validity of the one experiment that is actually conducted it is difficult to see why knowledge of what would happen under circumstances that will never occur should make us more confident about the conclusions drawn from the single study we have performed the problem faced by many randomistas when they underestimate heterogeneity and interaction effects is not merely one of external validity when attempting to transfer regression results across periods contexts or populations it is also an internal problem affecting the millions of regression estimates produced by economists each year ideally controlled experiments can tell us with certainty what causes what effects but only given the appropriate closure assumptions making valid extrapolations from experiments whether ideal accidental natural or quasi experimental to different settings populations or target systems is far from straightforward since trials are rarely repeated properties such as unbiasedness and balance in repeated sampling tell us nothing directly about any particular trial it works there is not evidence that it will work here causes identified in an experimental setting must still be shown to possess an export warrant to the target population or system the underlying causal assumptions must be justified and without such a warrant the value of even the most rigorous and precise methods and of knowledge that holds only on average is severely limited rcts have little reach beyond describing what has occurred in the past from the perspective of future decision making and public policy they are often of limited value because they cannot tell us which background factors were held constant when the intervention was implemented nor do rcts generally provide evidence that their results are exportable to other target systems they cannot simply be assumed to yield generalisable findings the fact that something works somewhere for someone is no guarantee that it will work for us here or indeed that it works more generally randomisation may in the right circumstances help us draw causal conclusions however it is neither necessary for establishing causality nor sufficient for securing scientific validity randomisation guarantees nothing just as observational studies are vulnerable to various forms of bias so too are randomised studies and trials share on facebook opens in new window facebook share on x opens in new window x share on bluesky opens in new window bluesky model reification in statistics and econometrics 15 jun 2026 at 10 57 posted in economics statistics econometrics 2 comments conventional statistical methods assume that within measured covariate levels the data come from surveys with purely random selection from a sharply defined population or from experiments with purely random treatment assignment in which the researcher has complete control and understanding of every important aspect of those who entered the analysis and how they were treated the methods thus provide logically sound inferences only in ideal cases rather than correct answers for our actual imperfect applications they are however widely applied and misinterpreted in studies that fall far short of their assumptions we may view each statistical analysis as a thought experiment in a fictional small world or toy example sharply restricted by its simplifying assumptions the questions that motivated the study must be translated properly into this fictional world statistical methods then answer the questions via mathematical deductions from the assumptions but those answers apply with logical force only within a fictional world in which all the assumptions hold sander greenland since econometrics aspires to explain things in terms of causes and effects it needs many assumptions limiting model assumptions in economic science always have to be closely examined because if we are to show that the mechanisms or causes we isolate in our mod...