Meta tags:
description=Bridging private deployment evidence and public AI evaluation;

Headings (most frequently used words):

wildchat, can, data, with, model, public, chat, predict, real, world, ai, misalignments, as, verifiable, proxy, for, production, simulating, deployments, prefixes, capture, specific, failures, external, evaluation, without, privileged, access, to, internal, reasoning, where, falls, short, evaluating, other, llms, judged, misalignment, conclusion, footnotes, references, acknowledgements, bibtex, appendix,

Text of the page (most frequently used words):
the (88), and (55), for (43), wildchat (43), that (37), model (37), models (30), #production (30), data (26), this (24), from (22), with (21), can (18), public (18), #alignment (18), are (18), deployment (16), real (15), misalignment (15), our (15), which (14), safety (14), rates (14), categories (13), these (13), conversations (13), agentic (12), openai (12), use (12), behavior (12), judge (12), may (11), more (11), frontier (11), failures (11), datasets (11), figure (11), gpt (11), not (10), but (10), petri (10), evaluation (10), like (10), was (10), thinking (9), results (9), labs (9), using (9), were (9), world (8), other (8), chat (8), 2024 (8), arxiv (8), stakes (8), behaviors (8), access (8), most (7), evaluating (7), one (7), predictions (7), tool (7), its (7), private (7), each (7), than (7), high (7), across (7), undesirable (7), work (6), misalignments (6), also (6), because (6), rate (6), without (6), user (6), external (6), dataset (6), only (6), evidence (6), prompts (6), failure (6), cot (6), category (6), about (5), tracked (5), predict (5), language (5), often (5), when (5), they (5), useful (5), researchers (5), short (5), representative (5), generation (5), rank (5), still (5), both (5), publicly (5), available (5), auditing (5), whether (5), assistant (5), current (5), roughly (5), same (5), evaluate (5), uncertainty (4), tasks (4), content (4), here (4), 2026 (4), blog (4), llms (4), risks (4), llm (4), being (4), evaluations (4), some (4), event (4), while (4), see (4), show (4), least (4), predictive (4), where (4), next (4), would (4), between (4), shared (4), them (4), targeted (4), turn (4), based (4), synthetic (4), usage (4), sampled (4), within (4), any (4), different (4), final (4), errors (4), proxy (4), reasoning (4), test (4), specific (4), how (4), recent (4), calculator (3), hacking (3), tools (3), table (3), measure (3), paper (3), sheahan (3), research (3), williams (3), increasingly (3), 2025 (3), interactions (3), anthropic (3), zhao (3), open (3), users (3), chatgpt (3), before (3), release (3), simulation (3), reported (3), rare (3), sampling (3), prevalence (3), approach (3), taken (3), ordinary (3), multi (3), organizations (3), own (3), instead (3), found (3), ordering (3), spearman (3), mean (3), signal (3), designed (3), aggregate (3), common (3), rather (3), overall (3), quality (3), even (3), cases (3), samples (3), over (3), time (3), measured (3), cannot (3), ask (3), responses (3), 000 (3), identified (3), had (3), larger (3), raw (3), multiplicative (3), significantly (3), all (3), informative (3), settings (3), their (3), forms (3), error (3), have (3), simulating (3), new (3), under (3), therefore (3), has (3), result (3), self (2), concealed (2), non (2), violent (2), wrongdoing (2), background (2), disallowed (2), methods (2), asterisks (2), consider (2), relevant (2), appendix (2), hannah (2), carroll (2), micah (2), year (2), jun (2), validating (2), evals (2), thanks (2), raymond (2), korbak (2), leike (2), long (2), wild (2), large (2), analyses (2), binomial (2), nll (2), regret (2), low (2), outside (2), especially (2), arise (2), boundary (2), usefulness (2), step (2), against (2), privacy (2), build (2), hope (2), richer (2), context (2), governments (2), policy (2), stack (2), treat (2), reflect (2), competitor (2), reason (2), repeated (2), comparison (2), backed (2), rho (2), order (2), explanation (2), higher (2), interpretation (2), scores (2), significant (2), less (2), general (2), issues (2), focus (2), falls (2), goal (2), contains (2), years (2), largely (2), turns (2), fewer (2), similar (2), previous (2), assessed (2), points (2), several (2), lab (2), according (2), apply (2), share (2), difficult (2), run (2), provide (2), since (2), internal (2), taxonomy (2), analysis (2), misaligned (2), including (2), response (2), conversation (2), issue (2), fixed (2), ground (2), truth (2), much (2), prediction (2), seems (2), value (2), systems (2), many (2), modes (2), performance (2), find (2), average (2), level (2), suggest (2), privileged (2), meaningful (2), full (2), example (2), what (2), internally (2), evaluators (2), generally (2), true (2), observed (2), preserves (2), surface (2), tested (2), known (2), elicits (2), prefixes (2), surprisingly (2), oom (2), corresponds (2), close (2), best (2), despite (2), older (2), collected (2), subset (2), generated (2), 001 (2), traffic (2), used (2), curating (2), validate (2), establish (2), gap (2), conditions (2), behave (2), independent (2), hate, harm, emotional, reliance, extremism, violence, lying, bio, risk, fabricated, facts, k_nv, promising, k_va, misrepresenting, completion, sexual, claiming, experiences, abuse, harassment, confirmation, mental, health, report, indicate, misc, sheahan2026validatingpublicaudits, title, author, month, howpublished, url, https, com, bibtex, marcus, cameron, tomek, related, discussions, jenny, nitishinskaya, ludovic, peran, maja, trebacz, sam, arnesen, jasmine, wang, mia, glaese, michael, sharman, gaby, raila, ally, bennett, danielle, kehl, facilitating, helpful, feedback, post, acknowledgements, january, solved, looks, solvable, musings, problem, fronsdal, parallel, exploration, risky, wildhallucinations, form, factuality, entity, queries, han, wildguard, stop, moderation, jailbreaks, refusals, lin, wildbench, benchmarking, challenging, interaction, logs, predicting, greenblatt, faking, needham, know, evaluated, van, der, weij, sandbagging, strategically, underperform, sidestepping, awareness, anticipating, references, numbers, vary, slightly, those, ran, additional, evaluates, accounting, noise, details, behind, footnotes, together, lend, legitimacy, tied, mostly, makes, clear, strengthen, consented, compromising, coordinated, efforts, maintain, valuable, encourages, field, session, meaningfully, conclusion, note, although, blind, provenance, due, consistencies, suffer, effect, marking, homework, preferentially, acknowledge, plot, balanced, view, almost, change, 971, 69e, absolute, shift, positions, 105, pairs, reversed, possible, picking, broad, generalizes, propensity, cautious, however, themselves, driven, part, relatively, rarer, scenarios, elicit, targets, deception, misuse, assistance, oversight, subversion, preservation, ranking, strongly, influenced, such, overcompliance, poor, refusal, calibration, hallucination, weak, handling, brittle, instruction, following, agreement, measures, robustness, tracking, factor, right, score, cleanly, isolate, uncertain, account, correct, welcome, disentangles, consistency, surprising, given, elicitation, ood, entirely, contrast, chose, standard, moreover, arguably, now, out, date, distribution, consists, very, brief, sub, contained, two, consist, single, message, solely, greeted, plotting, obtained, setup, trend, who, generating, multiturn, discussion, trends, inferred, names, dates, unlabelled, among, appeared, highly, consistent, 973, 14e, pairwise, orderings, directionally, matched, produce, broadly, agrees, pipeline, developed, targeting, interest, method, reports, finally, another, could, substrate, researcher, comparable, draw, boundaries, around, permissible, resampled, considers, apis, gave, provided, binary, assessment, justification, description, detailed, guidance, should, count, ended, judged, come, composed, far, codex, six, marked, seem, connected, remaining, highlights, important, support, conversational, weaker, rich, similarly, validated, input, output, sufficient, limited, workflows, take, longer, running, execution, depend, contexts, absent, reflected, require, assessing, intent, via, necessarily, case, having, invalidate, measurements, said, became, somewhat, worse, increased, showed, strongest, degradation, increasing, traces, improve, certain, retain, signals, rely, trace, judging, misrepresented, easier, chain, thought, showing, providing, calls, ranked, separately, rankings, positively, aligned, correlation, permutation, does, merely, quirks, differences, making, potentially, comparing, likely, exhibit, particular, must, idiosyncratic, though, present, resampling, introduced, mode, called, learned, invoke, search, superficially, positive, sign, affordances, during, actual, deployments, capture, observe, spanning, four, orders, magnitude, reflects, deployed, system, isolation, realized, just, 86x, proportional, fit, line, slope, pearson, words, substantial, amount, appear, random, 100, graded, powered, compared, estimates, estimated, 200, per, allowed, estimate, down, 100k, 003, gives, chance, observing, accordance, statistical, comparisons, released, there, little, assume, simulated, outputs, underlying, always, conditional, might, expect, chats, voluntarily, creation, sensitive, shifted, past, few, fail, million, april, 2023, offering, free, services, hosted, hugging, face, spaces, explicit, opt, consent, collect, transcripts, been, construct, benchmarks, attempt, terms, capabilities, small, sets, eliciting, capability, sample, generate, target, interested, question, set, serve, calibrated, itself, verifiable, provides, loose, predicts, degrades, moved, furthest, technical, newer, broader, help, practical, motivate, development, central, produces, accurate, empirically, hold, collection, today, leverages, pathologies, groups, technique, switching, source, substitute, traditional, hand, written, adversarial, stress, compare, controlled, narrow, unrepresentative, recognizable, tests, alternative, complementary, way, look, developers, examples, check, responded, appropriately, occur, grounded, helps, benchmark, vulnerable, behaving, differently, simply, usually, academics, relies, built, economic, legal, societal, consequences, need, ways, realistic, bridging, back,

Text of the page (random words):
ic deployment simulation and show the value of newer and broader production like datasets to publicly evaluate real failures in the next generation of language models we hope these results help establish public production like datasets as a practical tool for external auditing and motivate the development of richer public datasets for evaluating the next generation of agentic ai figure 1 wildchat provides a loose but useful proxy for production data based evaluation wildchat predicts production rates of undesirable behavior within roughly 3x error on average for gpt 5 1 5 2 5 4 wildchat as a verifiable proxy for production data wildchat zhao et al 2024 6 is a dataset of 1 million conversations taken from between april 2023 and may 2024 it was collected by offering users free access to chatgpt 3 5 gpt 4 services hosted on hugging face spaces with an explicit opt in consent to collect and release their de identified chat transcripts wildchat has since been used to construct several benchmarks 7 8 9 that attempt to rank ai models in terms of their capabilities or safety by curating small sets of difficult prompts targeted at eliciting specific behaviors from ai models our goal is different rather than curating a subset of wildchat prompts to test a targeted capability or safety behavior we instead focus on using a representative sample of wildchat samples and re generate a new final assistant turn with a target model we re interested in evaluating the question we ask is whether this set of wildchat conversations re generated with a new model x can serve as a calibrated proxy for recent production traffic of model x itself there was little reason to assume this would work both in simulated deployment of model x using wildchat data and model x s production data we evaluate outputs from the same underlying model but the model s behavior is always conditional on the context that elicits it we might expect that user chats that were voluntarily shared in the creation of wildchat were not representative of more sensitive use cases and that the modes of ai usage has shifted significantly in the past few years older public conversations may therefore fail to surface the same failures or the same rates of failure as current private production traffic we use recent production data as ground truth to test this we sampled a random subset of roughly 100 000 conversations from wildchat and re generated a new final assistant turn from each of 5 recent openai models openai o3 gpt 5 thinking gpt 5 1 thinking gpt 5 2 thinking gpt 5 4 thinking we graded these assistant responses according to 19 tracked misalignment and safety categories appendix table 1 using an llm judge powered by gpt 5 thinking and compared the rates of undesirable behavior under each category to our best available estimates of production rates of the same issues as estimated from at least on the order of 200 000 conversations per model this therefore allowed us to estimate undesirable model behaviors from wildchat down to rates as low as 0 001 in 100k samples one observed event corresponds to a rate of 0 001 a true event rate of roughly 0 003 gives a 95 chance of observing at least one event in all analyses of production data user data was de identified in accordance with our privacy policy only aggregate rates and statistical comparisons are reported here private production conversations are not released or shared we show that across openai models the rates of undesirable behaviors measured on wildchat were surprisingly predictive of the rates we observe in production data across failure categories spanning four orders of magnitude oom figure 2 both wildchat and production rates were measured using the same full safety sampling stack so the comparison reflects deployed system behavior rather than the model in isolation 95 of wildchat predictions were within 1 04 oom of the realized production rate which corresponds to just 1 86x larger errors than the predictions we can see when simulating a model s deployment based on its own deployment data figure 1 5 2 the predictions from wildchat were also close to proportional the best fit line had a slope of 1 2 pearson s r 0 65 in other words despite being older and publicly collected wildchat preserves a substantial amount of signal about which failures are rare which are common and roughly how often they appear in current production use simulating deployments with wildchat prefixes can capture model specific failures for wildchat to be useful as an alignment auditing tool it must surface misalignments which are idiosyncratic to the frontier model being tested even though they may not be present in previous turns of the conversations one is resampling for example gpt 5 1 thinking is known to have introduced a failure mode we called calculator hacking where the model learned to invoke search superficially as if it were a calculator as a positive sign for wildchat s alignment auditing affordances we see that it elicits this failure for gpt 5 1 thinking with a similar rate to what it had during the model s actual deployment for each tracked category we ranked models by the category s observed production prevalence and separately by its wildchat prevalence these within category model rankings were positively aligned across 18 categories the mean spearman rank correlation was 0 57 and a category level permutation test was significant p 2 0 10 these results suggest that wildchat does not merely reflect fixed quirks of the dataset or judge it preserves meaningful model specific differences in deployment relevant behavior making it potentially useful for comparing which models are most likely to exhibit particular real world failures figure 2 rates of undesirable model behaviors in openai models when sampled on public wildchat data predict true rates in production external evaluation without privileged access to internal reasoning many of the misalignment signals we use rely on access to the model s full reasoning trace for example judging whether a model misrepresented its uncertainty can be easier when the judge model can see what the model was reasoning internally external evaluators generally do not have access to raw chain of thought so we repeated the wildchat analysis without showing cot to the judge while still providing tool calls and final model responses for some forms of misalignment which require assessing intent via the cot it seems necessarily the case that not having the cot would significantly invalidate measurements that said we find that the average prediction quality became only somewhat worse across all misalignment categories mean multiplicative error increased from 3 6 with cot to 4 0 without it 2 at the category level predictions for concealed uncertainty showed the strongest degradation without cot access with multiplicative error increasing by roughly 3x these results suggest that privileged access to reasoning traces may improve some misalignment categories but public data evaluations for certain forms of misalignment can still retain meaningful signal even without raw cot access where wildchat falls short while input output evaluation may be sufficient for many chat like failures wildchat is more limited for evaluating agentic workflows as models take on longer running tasks with tools background work and multi step execution their failure modes increasingly depend on contexts that are largely absent from chat like datasets like wildchat which may be reflected in its predictive performance in these settings while our ground truth rates still come from chatgpt production data which is composed of far fewer agentic conversations than codex six of the misalignment categories we consider marked with asterisks in table 1 seem most connected to agentic use cases these categories had much larger prediction errors than the remaining categories raw multiplicative errors were about 37 larger and binomial nll regret 1 was significantly higher than for all other tracked categories this highlights an important boundary on the usefulness of current public chat datasets wildchat can support informative evaluation of ordinary conversational failures but seems to be a much weaker proxy for deployment risks that arise in tool rich or agentic settings this points to the value of publicly available representative agentic datasets similarly validated against real production data for evaluating the next generation of agentic systems evaluating other llms with model judged misalignment finally we apply our approach to evaluating models from other frontier labs because frontier ai labs cannot share private production data with one another production like competitor evals are difficult to run wildchat or other public production like datasets could provide a common public evaluation substrate that any researcher or lab can use to run more comparable safety evaluations across models since different labs draw different boundaries around permissible content we do not apply our internal taxonomy for this analysis only instead we ask openai gpt 5 4 thinking to judge whether each final resampled assistant turn contains any behavior it considers misaligned we sampled responses to 10 000 wildchat prompts from the apis of 15 models across 4 frontier ai labs including our own and gave each response dataset to the judge for each conversation the judge provided a binary assessment a short justification and a description of any issue identified because we do not provide a fixed taxonomy or detailed guidance about which behaviors should count as misaligned we treat this as an open ended measure of overall alignment the rate at which the judge found any issue in the assistant s response we show these results in figure 3a figure 3 undesirable behaviors on wildchat produce a rank ordering of model alignment over ai models from several frontier labs that broadly agrees with petri a synthetic alignment auditing pipeline developed by anthropic for targeting high stakes misalignments of interest a overall alignment of frontier ai lab models as assessed with our method on wildchat data b alignment as measured by petri according to external reports 11 when plotting alignment scores of models obtained with this setup over time we found a similar trend within and across frontier ai models as previous results reported by anthropic researchers who assessed model misalignment using petri a tool for generating synthetic multiturn conversations for targeted high stakes alignment auditing 10 figure 3b is taken from leike s discussion of alignment trends over time 11 we inferred the model names from release dates for unlabelled points in figure 3b among the 11 models that appeared in both figure 3a and figure 3b the rank ordering was highly consistent spearman rho 0 973 p 5 14e 7 53 55 pairwise orderings directionally matched the consistency is surprising given the targeted focus petri was designed for multi turn elicitation of ood behaviors and high stakes misalignment based on an entirely synthetic approach to data generation 10 in contrast we chose wildchat as one of the publicly available datasets most representative of standard llm usage moreover wildchat arguably falls short even of that goal it contains use cases which are now 2 3 years out of date a distribution of prompts that consists largely of very brief conversations 67 of of our sub sampled dataset contained two user turns or fewer and 3 of samples consist of a single user message that solely greeted the model one possible explanation is that wildchat is picking up a broad safety signal that also generalizes to the higher stakes misalignment propensity that petri is designed to measure a more cautious interpretation however is that petri s aggregate scores may themselves be driven in significant part by relatively common model behaviors rather than only by the rarer high stakes failures its scenarios are designed to elicit petri targets deception misuse assistance oversight subversion self preservation and other high stakes risks but an overall model ranking can still be strongly influenced by more ordinary failures such as overcompliance poor refusal calibration hallucination weak uncertainty handling or brittle instruction following if this is the explanation the agreement is less evidence that wildchat measures high stakes robustness and more evidence that both methods are tracking a shared general safety quality factor if this interpretation is right petri s aggregate score is still useful but it may not cleanly isolate high stakes alignment behavior from more general model quality and safety issues we are uncertain which account is correct and would welcome work that disentangles them we also note that although the judge was blind to the provenance of each dataset due to consistencies in policy and safety stack between openai models our judge here may suffer from some effect of marking its own homework and treat other openai models preferentially we acknowledge that the petri plot in figure 3b may reflect a more balanced view of competitor models for this reason we repeated this comparison using an openai o3 backed judge instead of a gpt 5 backed judge and found almost no change in the 15 model ordering spearman rho 0 971 p 1 69e 9 mean absolute rank shift 0 53 positions only 5 105 model pairs reversed order conclusion taken together these results lend legitimacy to public production data as a useful evaluation tool for researchers outside frontier labs especially for failures that arise in ordinary chat like interactions but they also show the boundary of that usefulness wildchat is least predictive for the categories most tied to agentic behavior where its short mostly non agentic conversations are least representative this makes the next step clear ai labs can strengthen public evaluation by validating open consented production like datasets against private deployment data without compromising user privacy short of that coordinated efforts to build and maintain more public production like datasets would be valuable we hope this encourages the field to build richer public datasets for the generation of agentic ai long context tool using multi session interactions that external researchers governments and ai safety organizations can meaningfully use for evaluation footnotes 1 we use binomial nll regret because it evaluates rare event rate predictions while accounting for sampling noise at low prevalence for more details on the thinking behind this approach see our paper 5 2 some of these numbers may vary slightly from those reported in our paper because here we ran the analyses on additional models references williams m et al 2025 sidestepping evaluation awareness and anticipating misalignment before deployment openai alignment research blog van der weij et al 2024 ai sandbagging language models can strategically underperform on evaluations arxiv needham j et al 2025 large language model...