Meta tags:
description= Using large digital libraries to advance literary history;
Headings (most frequently used words):
the, of, ai, to, time, can, and, in, what, we, fiction, estimate, culture, posts, stone, shell, more, upside, make, us, intelligence, be, next, does, this, gpt, measure, why, how, well, do, mapping, is, cultural, technology, editing, marionette, theater, interesting, will, overconfident, should, artificial, person, shaped, language, models, predict, twist, story, have, any, relation, human, uncertainty, reason, not, trust, so, could, learn, from, predicting, three, pages, liberally, educated, students, need, than, consumers, save, matters, about, writing, at, price, using, passage, bag, words, methods, train, llms, no, model, latent, spaces, science, hasn, prepared, imagine, machine, learning, pagination, some, better, recent, categories, blogroll, archives, super, for, as, doesn, anyone, less, depressed, wait, good, idea, references,
Text of the page (most frequently used words):
the (769), and (390), that (326), this (166), but (160), for (147), not (132), #models (125), are (120), can (113), about (112), with (107), will (97), have (90), #language (85), more (81), they (80), human (79), from (78), what (69), model (67), like (62), because (60), don (58), than (56), you (56), learning (54), has (49), one (49), our (48), new (48), intelligence (47), could (46), how (45), think (45), when (43), was (43), other (42), may (42), there (42), people (41), time (41), need (39), also (39), their (39), all (39), post (39), just (38), some (36), would (35), good (34), culture (33), writing (32), know (32), them (32), social (31), even (31), fiction (30), make (30), only (30), who (30), been (30), https (29), machine (28), students (28), use (28), which (28), now (27), cultural (27), might (27), see (27), gpt (27), 2021 (26), things (26), space (26), find (26), way (26), where (25), these (25), different (25), reason (25), out (25), those (24), say (24), own (24), problem (24), why (24), should (23), want (23), readers (23), questions (23), still (23), using (23), artificial (22), between (22), data (22), meaning (22), story (21), its (21), here (21), org (21), passage (21), 2023 (20), work (20), next (20), instance (20), hard (20), had (20), question (19), into (19), measure (19), books (18), ways (18), let (18), words (18), something (18), much (18), many (18), get (18), technology (18), were (18), then (18), already (17), july (17), text (17), effects (17), plot (17), world (17), possible (17), same (17), author (17), point (16), research (16), understand (16), itself (16), does (16), researchers (16), each (16), parrots (16), often (16), minutes (16), blog (15), change (15), any (15), thinking (15), process (15), mean (15), years (15), actually (15), map (15), little (15), large (15), self (15), word (15), history (14), person (14), upside (14), take (14), range (14), science (14), yet (14), approach (14), neural (14), task (14), since (14), first (14), answer (14), june (13), 2017 (13), 2024 (13), project (13), after (13), right (13), agents (13), very (13), give (13), over (13), step (13), clip (13), end (13), stochastic (13), paper (13), feel (13), 2012 (12), scale (12), deep (12), predict (12), recent (12), better (12), beings (12), lot (12), whether (12), sense (12), look (12), making (12), really (12), date (12), across (12), specific (12), assignments (12), personality (12), comments (11), january (11), 2011 (11), analysis (11), topic (11), modeling (11), categories (11), literary (11), thing (11), clear (11), maybe (11), create (11), going (11), course (11), two (11), kind (11), less (11), doing (11), most (11), image (11), past (11), bad (11), least (11), enough (11), suspense (11), thought (11), challenge (11), his (11), above (11), apple (11), february (10), march (10), 2016 (10), 2018 (10), 2025 (10), interesting (10), expect (10), future (10), explore (10), latent (10), eventually (10), patterns (10), training (10), toward (10), keep (10), tedunderwood (10), force (10), positive (10), generative (10), used (10), debate (10), openai (10), several (10), prediction (10), help (10), style (10), code (10), references (10), both (10), back (10), working (10), ability (10), september (9), 2015 (9), historical (9), theater (9), reading (9), too (9), open (9), idea (9), through (9), perhaps (9), imagine (9), read (9), produce (9), general (9), run (9), goal (9), doi (9), knowledge (9), technologies (9), likely (9), single (9), while (9), effect (9), doesn (9), week (9), passages (9), started (9), learn (9), ask (9), chapter (9), report (8), com (8), october (8), 2013 (8), humanities (8), character (8), posts (8), tags (8), probably (8), well (8), instead (8), vast (8), stories (8), possibility (8), art (8), wrong (8), play (8), another (8), summary (8), collective (8), makes (8), century (8), pages (8), however (8), side (8), argument (8), believe (8), prompt (8), fully (8), underwood (8), critical (8), long (8), conversation (8), example (8), represent (8), recognize (8), short (8), part (8), describe (8), experiment (8), predictions (8), found (8), page (8), reference (8), real (8), balance (8), professors (8), happen (8), bluesky (8), consciousness (8), content (7), stone (7), december (7), august (7), 2014 (7), 2020 (7), search (7), methods (7), trend (7), transformer (7), shaped (7), marionette (7), seven (7), put (7), boring (7), few (7), number (7), ever (7), either (7), kinds (7), form (7), others (7), novel (7), system (7), being (7), important (7), behavior (7), yes (7), hand (7), sentence (7), humans (7), images (7), old (7), cost (7), your (7), scene (7), arxiv (7), aspects (7), picture (7), big (7), potential (7), agree (7), train (7), public (7), disciplines (7), fact (7), pretend (7), matters (7), asked (7), three (7), chatgpt (7), elapsed (7), narrative (7), press (7), construct (7), require (7), start (7), getting (7), sometimes (7), her (7), days (7), confidence (7), try (7), estimate (7), teach (7), save (7), trained (7), robin (7), education (7), vision (7), surprise (7), anthropic (7), depressed (7), write (6), shell (6), april (6), attention (6), problems (6), twist (6), quickly (6), always (6), worlds (6), version (6), ends (6), sort (6), library (6), door (6), whole (6), seem (6), early (6), suspect (6), add (6), dall (6), writers (6), seems (6), place (6), field (6), especially (6), great (6), similar (6), nothing (6), means (6), prefer (6), made (6), hasn (6), did (6), conference (6), value (6), ted (6), abs (6), rather (6), book (6), anyone (6), down (6), usually (6), decide (6), tools (6), mapping (6), produced (6), surprising (6), role (6), around (6), understanding (6), mostly (6), llms (6), simple (6), api (6), method (6), provide (6), before (6), assistant (6), reasons (6), easier (6), experience (6), described (6), every (6), themselves (6), github (6), best (6), voice (6), term (6), university (6), uncertainty (6), bum (6), someone (6), novels (6), breaks (6), moral (6), super (6), media (6), aesthetic (6), persona (6), required (5), reader (5), account (5), victorian (5), article (5), digital (5), 19c (5), teaching (5), analytics (5), saying (5), become (5), useful (5), reality (5), storytelling (5), matter (5), surprised (5), pattern (5), never (5), game (5), exploring (5), support (5), characters (5), styles (5), users (5), starting (5), trying (5), without (5), individual (5), created (5), larger (5), mental (5), resemble (5), title (5), gets (5), claim (5), interested (5), doubt (5), test (5), isn (5), bert (5), certain (5), job (5), haven (5), carbon (5), feature (5), dreams (5), artists (5), white (5), material (5), risks (5), dangers (5), dangerous (5), consequences (5), biases (5), pictures (5), position (5), behind (5), explicitly (5), historians (5), conflict (5), offer (5), texts (5), response (5), given (5), forms (5), far (5), earlier (5), advance (5), second (5), stable (5), jane (5), statistical (5), currently (5), remember (5), measuring (5), called (5), compare (5), core (5), challenges (5), day (5), final (5), address (5), edit (5), reply (5), characterize (5), set (5), average (5), axis (5), until (5), student (5), principles (5), home (5), valid (5), sound (5), suggest (5), distance (5), continue (5), chang (5), 900 (5), necessarily (5), installments (5), hound (5), difficult (5), askell (5), personhood (5), claude (5), power (5), invented (5), void (5), bots (5), stateful (5), name (4), log (4), november (4), creative (4), era (4), theory (4), genre (4), strategies (4), phenomenon (4), humanists (4), libraries (4), decade (4), direction (4), cyberpunk (4), hidden (4), wider (4), moment (4), analogous (4), news (4), helpful (4), allow (4), everything (4), works (4), familiar (4), interactive (4), fictional (4), tend (4), thousands (4), friends (4), understood (4), having (4), plans (4), sure (4), thoughts (4), reach (4), assume (4), presumably (4), living (4), impressive (4), order (4), below (4), reasoning (4), evidence (4), limited (4), access (4), trust (4), knows (4), four (4), turn (4), bit (4), guess (4), hope (4), prepared (4), low (4), against (4), mark (4), journal (4), perspective (4), handle (4), notes (4), trans (4), bender (4), proceedings (4), spaces (4), imagined (4), intent (4), merely (4), suggested (4), easy (4), essay (4), closely (4), air (4), failed (4), respond (4), internet (4), looks (4), unreal (4), engine (4), consider (4), didn (4), discourse (4), critics (4), scholars (4), universities (4), tech (4), seemed (4), move (4), nsf (4), begin (4), perspectives (4), cover (4), develop (4), multiple (4), fine (4), structure (4), corpora (4), reflect (4), web (4), wide (4), exactly (4), genres (4), wouldn (4), published (4), mind (4), pose (4), tends (4), center (4), tasks (4), yauney (4), david (4), running (4), spite (4), agreement (4), she (4), memory (4), hours (4), hour (4), gave (4), results (4), slightly (4), autonomy (4), tune (4), estimates (4), bag (4), experiments (4), valuable (4), focused (4), cases (4), compared (4), provided (4), tell (4), assumptions (4), honest (4), instruction (4), year (4), thirty (4), explain (4), asks (4), won (4), anything (4), sciences (4), mla (4), cccc (4), thesis (4), gentle (4), effectively (4), break (4), assistance (4), essays (4), asking (4), ourselves (4), insight (4), feelings (4), initial (4), advice (4), guide (4), comes (4), ago (4), expected (4), jobs (4), automated (4), implies (4), exciting (4), talk (4), review (4), encouraging (4), serial (4), accuracy (4), roughly (4), baskervilles (4), holmes (4), avoid (4), names (4), scaling (4), birds (4), sky (4), theatrical (4), mask (4), signals (4), last (4), altman (4), persons (4), ease (4), retrieved (4), golden (4), gate (4), sentimental (4), marionettes (4), agent (4), email (3), view (3), site (3), sign (3), wordpress (3), 2026 (3), 2019 (3), planet (3), dan (3), correlation (3), mining (3), diction (3), interpretive (3), criticism (3), discovery (3), building (3), liberally (3), educated (3), consumers (3), overconfident (3), distant (3), emergence (3), babel (3), places (3), invent (3), almost (3), fear (3), worry (3), doors (3), piece (3), thread (3), corporations (3), autonomous (3), loosely (3), robots (3), alien (3), taught (3), within (3), examples (3), genuinely (3), original (3), authors (3), universe (3), bigger (3), call (3), borrowed (3), maps (3), interactively (3), build (3), worse (3), reproduce (3), solving (3), primates (3), life (3), predicting (3), nineteenth (3), twitter (3), boxes (3), mistake (3), leave (3), off (3), smart (3), chimpanzee (3), individuals (3), quite (3), complex (3), association (3), certainly (3), taking (3), supposed (3), fill (3), released (3), clearly (3), verbal (3), obvious (3), energy (3), importantly (3), weigh (3), footprint (3), argue (3), successful (3), spatial (3), charlie (3), edu (3), generation (3), house (3), national (3), black (3), net (3), katherine (3), foundation (3), transparency (3), systems (3), enjoy (3), shared (3), concerns (3), note (3), gives (3), allowed (3), analogy (3), political (3), everyone (3), expertise (3), negative (3), absolutely (3), existing (3), impossible (3), attempt (3), practice (3), together (3), features (3), particular (3), visual (3), although (3), graduate (3), tuning (3), purpose (3), companies (3), differences (3), publication (3), service (3), somehow (3), free (3), bias (3), typically (3), immediate (3), templates (3), creatively (3), necessary (3), study (3), consistent (3), definition (3), captioned (3), phrase (3), traces (3), communication (3), hear (3), state (3), simply (3), computer (3), discuss (3), event (3), roundtable (3), consensus (3), stanford (3), implications (3), simon (3), willison (3), greg (3), computational (3), measured (3), 1980 (3), accurate (3), interpretable (3), defined (3), relatively (3), concepts (3), thousand (3), seen (3), show (3), needs (3), assistants (3), bamman (3), explains (3), nature (3), man (3), night (3), again (3), close (3), whatever (3), narrator (3), childhood (3), replies (3), amount (3), turbo (3), plus (3), ambiguity (3), providing (3), high (3), learned (3), front (3), count (3), yesterday (3), ideal (3), notice (3), cautious (3), needed (3), explicit (3), events (3), spoken (3), passed (3), weeks (3), user (3), interaction (3), durations (3), counting (3), precisely (3), feels (3), length (3), represents (3), entity (3), specify (3), documents (3), based (3), corey (3), changed (3), fall (3), willing (3), once (3), got (3), reflection (3), preserve (3), solve (3), ideas (3), further (3), wheel (3), along (3), fake (3), spin (3), cheat (3), metacognition (3), suggesting (3), midjourney (3), harder (3), angle (3), suggestions (3), beginning (3), wanted (3), putting (3), technical (3), lives (3), html (3), concept (3), allows (3), framed (3), available (3), talking (3), case (3), poses (3), plato (3), novelty (3), family (3), serialization (3), ryan (3), anders (3), group (3), arguing (3), narratives (3), turns (3), expectation (3), mystery (3), installment (3), encourage (3), divergence (3), summaries (3), tempting (3), written (3), interfaces (3), laugh (3), cringe (3), telling (3), products (3), provides (3), caddies (3), interacting (3), argues (3), abstract (3), demo (3), pygmalion (3), chatbot (3), evans (3), obsessed (3), flight (3), spectrum (3), alignment (3), passive (3), accessible (3), passively (3), textbook (3), points (3), wait (3), kapoor (3), farrell (3), gopnik (3), monosemanticity (3), singularity (3), editing (3), manipulate (3), printing (3), modes (3), durable (3), protean (3), legal (3), freedom (3), journey (3), kleist (3), constructed (3), distasteful (3), advantage (3), protagonist (3), inner (3), penny (3), network (3), morality (3), moltbook (3), animals (3), familiars (3), subscribed (2), subscribe (2), 2010 (2), google (2), cohen (2), american (2), under (2), politics (2), uncategorized (2), statistics (2), literature (2), reproducibility (2), replication (2), poetic (2), pace (2), disciplinary (2), collection (2), 20c (2), gender (2), broad (2), older (2), extrapolating (2), deepfakes (2), fears (2), borges (2), optimistically (2), money (2), portal (2), fantasy (2), 2020s (2), subgenre (2), sinister (2), etc (2), led (2), considering (2), decades (2), inscrutable (2), transforming (2), containing (2), robot (2), collaborative (2), script (2), protein (2), folding (2), intelligent (2), music (2), activity (2), expectations (2), increasing (2), fan (2), developed (2), variety (2), arthur (2), devoted (2), safe (2), children (2), symbolic (2), coordinates (2), desires (2), structures (2), bananas (2), advances (2), shrug (2), none (2), envision (2), includes (2), jerome (2), lived (2), true (2), generates (2), expressing (2), intention (2), inference (2), implied (2), contest (2), pursuing (2), unfamiliar (2), manner (2), broadly (2), learns (2), photo (2), television (2), detail (2), plausible (2), recently (2), unlikely (2), threat (2), ready (2), struggling (2), lately (2), said (2), contemporary (2), efforts (2), cause (2), reduction (2), actively (2), agenda (2), renewable (2), sham (2), deceptive (2), campaign (2), mashable (2), skeptical (2), lies (2), institute (2), skeptically (2), limits (2), interest (2), berkeley (2), gray (2), chen (2), ilya (2), sutskever (2), zero (2), resource (2), gov (2), wang (2), autoregressive (2), martin (2), daniel (2), jacob (2), eisenstein (2), william (2), corr (2), size (2), improve (2), discussed (2), bommasani (2), 100 (2), soni (2), lauren (2), abolitionist (2), networks (2), activist (2), newspapers (2), culturalanalytics (2), list (2), production (2), bode (2), non (2), oct (2), recap (2), gebru (2), mcmillan (2), major (2), 610 (2), 623 (2), 1145 (2), 3442188 (2), 3445922 (2), acm (2), fairness (2), accountability (2), information (2), reframe (2), imply (2), illuminate (2), fascinating (2), transform (2), deeply (2), contrast (2), connects (2), plan (2), intelligences (2), tropes (2), conscious (2), various (2), care (2), ethical (2), dilemma (2), controlled (2), movable (2), type (2), war (2), meant (2), shape (2), common (2), observation (2), machines (2), angel (2), diffusion (2), region (2), fun (2), themes (2), differently (2), embeddings (2), dizzying (2), devoid (2), rhetorical (2), broader (2), outside (2), alternatives (2), forward (2), danger (2), computing (2), resources (2), huggingface (2), helping (2), languages (2), distinguish (2), flexibility (2), associate (2), statement (2), challenging (2), requires (2), correctly (2), contrary (2), acceptable (2), audiences (2), delicate (2), share (2), token (2), practices (2), disappointing (2), starts (2), instructions (2), select (2), dictionary (2), named (2), subjects (2), tendency (2), grounded (2), communicative (2), 616 (2), intentional (2), sequences (2), defines (2), trajectory (2), parts (2), anyway (2), natural (2), changing (2), academics (2), credit (2), discussion (2), applications (2), princeton (2), prepare (2), follows (2), mimno (2), away (2), loud (2), automatically (2), document (2), fast (2), modest (2), measurement (2), error (2), confident (2), separate (2), systematically (2), coming (2), showing (2), addressed (2), signs (2), six (2), heard (2), dreaming (2), child (2), incident (2), couch (2), hands (2), water (2), remembers (2), 10080 (2), moderate (2), passing (2), interpretability (2), myself (2), prompts (2), cheaper (2), correlated (2), 121 (2), inter (2), closer (2), took (2), months (2), llm (2), main (2), pointed (2), complete (2), added (2), allowing (2), extremely (2), answers (2), capacity (2), assess (2), quinn (2), dombrowski (2), query (2), morning (2), window (2), coffee (2), dressed (2), seeing (2), car (2), ego (2), responding (2), slowly (2), twice (2), pretty (2), encouraged (2), responses (2), five (2), summarize (2), include (2), taken (2), description (2), dialogue (2), units (2), times (2), understands (2), higher (2), transformed (2), lower (2), definitely (2), estimating (2), labeled (2), predicted (2), hundred (2), showed (2), manually (2), literally (2), says (2), conclude (2), 250 (2), calculated (2), passes (2), minute (2), automate (2), addresses (2), hundreds (2), watkins (2), marc (2), 1984 (2), rudy (2), scarborough (2), joint (2), overview (2), annoying (2), patching (2), pedagogy (2), sufficient (2), capable (2), framing (2), came (2), fresh (2), puts (2), emphasis (2), turning (2), critique (2), normative (2), venture (2), meta (2), adapting (2), intellectual (2), advanced (2), gather (2), interpret (2), rehearse (2), processes (2), originally (2), simulate (2), assignment (2), simulating (2), replicate (2), attitude (2), wisdom (2), optimistic (2), elegiac (2), opportunity (2), spins (2), weird (2), cynical (2), foreground (2), externalize (2), focus (2), imitate (2), trace (2), adjusting (2), practical (2), college (2), avoids (2), class (2), exams (2), solution (2), daughter (2), team (2), price (2), imperfect (2), department (2), critically (2), admittedly (2), illustration (2), variance (2), basic (2), level (2), 2030s (2), courses (2), recommend (2), consumer (2), looking (2), collar (2), choose (2), emerge (2), solved (2), discover (2), today (2), coherent (2), warhol (2), tobin (2), harvard (2), smuts (2), 2009 (2), paradox (2), sinykin (2), sap (2), choi (2), acl (2), meeting (2), piper (2), revelation (2), mcgrath (2), modernist (2), haugtvedt (2), magazines (2), periodicals (2), garrett (2), dames (2), twenty (2), speak (2), beekman (2), woman (2), previously (2), presented (2), multi (2), chunks (2), appropriate (2), related (2), crude (2), trusting (2), cliffhanger (2), standard (2), ensure (2), epistemic (2), rhythms (2), doctor (2), footprints (2), answered (2), predictable (2), steel (2), inferences (2), draw (2), diss (2), cosine (2), check (2), supply (2), masked (2), moreover (2), flying (2), blind (2), shouldn (2), studying (2), reveals (2), tended (2), actual (2), drawn (2), johnson (2), claiming (2), correlate (2), sees (2), ernie (2), happens (2), sesame (2), street (2), beyond (2), impulse (2), warner (2), presents (2), spike (2), jonze (2), mallarmé (2), stéphane (2), neuromancer (2), amanda (2), richie (2), stuart (2), creeped (2), reminds (2), full (2), truth (2), anthropomorphize (2), pure (2), sounding (2), interface (2), impersonal (2), lie (2), reminders (2), owners (2), sounded (2), scarlett (2), johansson (2), eyes (2), opinions (2), determine (2), react (2), grumpy (2), anthropocentric (2), gendered (2), laughter (2), creature (2), emotion (2), caddy (2), quietly (2), extension (2), personal (2), siri (2), coalesce (2), woo (2), sam (2), insist (2), fair (2), risk (2), ambitious (2), downsides (2), lean (2), league (2), boots (2), frame (2), monolith (2), stack (2), overflow (2), threads (2), repos (2), entry (2), daunting (2), prove (2), 1990s (2), program (2), disagree (2), overconfidence (2), bugs (2), evolve (2), strong (2), adding (2), couldn (2), wilder (2), bryanwilder (2), ebooks (2), 2022 (2), jowett (2), phaedrus (2), narayanan (2), normal (2), mollick (2), manning (2), shalizi (2), circuits (2), index (2), extracting (2), sonnet (2), samaltman (2), dark (2), depressing (2), sounds (2), costs (2), benefits (2), purely (2), sharpen (2), swing (2), forth (2), grok (2), heat (2), eliminate (2), load (2), bearing (2), ideological (2), weapons (2), chance (2), habits (2), produces (2), outweigh (2), sustained (2), similarly (2), adjectives (2), directions (2), neurons (2), vertigo (2), bridge (2), able (2), among (2), sref (2), stylistic (2), dimensional (2), vector (2), james (2), fail (2), survey (2), skeptics (2), potentially (2), mature (2), generally (2), stage (2), orders (2), magnitude (2), greater (2), ill (2), substitute (2), false (2), agi (2), generalizable (2), entities (2), path (2), capability (2), independent (2), command (2), external (2), else (2), decisions (2), agency (2), nerds (2), recognized (2), naive (2), tree (2), grace (2), personas (2), shtick (2), struggle (2), sentimentality (2), wince (2), voyage (2), puppet (2), sitting (2), lap (2), pinocchio (2), schiller (2), staged (2), humanity (2), flaw (2), actors (2), alas (2), weakness (2), inhabit (2), accounts (2), riff (2), dancer (2), gesture (2), truly (2), aesthetically (2), discomfort (2), cookie (2), puppets (2), cheesy (2), morally (2), maltese (2), audience (2), patron (2), memento (2), menu (2), website, comment, loading, collapse, bar, manage, subscriptions, privacy, join, 596, subscribers, top, archives, scottbot, irregular, seasr, presentations, sapping, parezco, digo, monk, meandre, installation, ngram, viewer, geoffrey, nunberg, digitally, assisted, collocates, english, 1810, artfl, 18thconnect, blogroll, licensed, commons, attribution, sharealike, unported, license, visualization, unsupervised, undigitized, sociology, romantic, representativeness, reception, poetry, ocr, correction, nonconsumptive, ngrams, methodology, metadata, math, lsa, linguistics, impressionistic, historicism, comparison, bayesian, 18c, rss, dataset, 1700, 1922, structuralist, theories, computers, instability, standards, outlines, jdh, newer, pagination, movements, qanon, eager, parallel, armored, endlessly, impenetrable, servant, refuses, pod, bay, heroines, chalk, chosen, leading, bookshelf, personally, mashup, wardrobes, encylopedias, tlön, futuristic, eric, desmazieres, emotions, servile, excite, terrify, labyrinth, infinite, eye, alternate, shelves, god, extrapolation, weirder, journalism, invariably, invite, servants, antagonists, terminator, hal, paternal, condescension, dread, reactions, domains, creators, collaboratively, emerges, exploration, solitary, undertaken, geniuses, scratch, origin, folk, tales, rearranging, moves, riffing, established, involved, creating, replace, interconnected, proposed, grown, ups, settings, illustrations, rackham, graphic, welcome, collaborators, happening, sites, tens, competitors, enter, startup, motto, subreddit, dungeon, endeavors, including, needn, attempts, imagining, collaborator, amino, acids, infancy, improved, foolish, ignore, clues, bizarrely, parochial, writer, stacking, supposedly, overrated, buzz, relax, realist, goals, achieve, paragraph, wander, understandably, ultimately, wolfgang, köhler, 1925, mentality, apes, driven, mario, klingemann, byline, extrapolate, expressed, symbols, opinion, probability, sentences, shivery, crossed, significant, gary, marcus, ernest, davies, correlations, fusing, abstractions, lemur, hold, umbrella, surprisingly, 1910s, output, baby, daikon, radish, tutu, walking, dog, involving, underway, developments, imaginary, translate, descriptions, opera, zombie, apocalypse, receding, fight, rescue, replicants
Text of the page (random words):
emic signals of bias and limitation suppressing those signals is not necessarily more honest it may be relevant here that the impersonal transparency of the caddies in all the birds in the sky turns out to be a lie no spoilers but the caddies actually have an agenda and are using those neutral notifications and reminders to steer their human owners it wouldn t be shocking if corporate interfaces did the same thing so should we anthropomorphize ai i think it s a much harder question than is commonly assumed and maybe not a question that can be answered at all apple and anthropic are selling different products to different audiences there s no reason one of them has to be wrong on bluesky dave palfrey reminds me that the etymology of person leads back through fictional character to mask more fundamentally this is a hard question because it s not clear that we re telling the full truth when we anthropomorphize people writers and critics have been arguing for a long time that the personality of the author is a mask as stéphane mallarmé puts it the pure work implies the disappearance of the poet speaking who yields the initiative to words 208 there s a sense in which all of us are language models how do i know what i think until i see what i say this shoggoth could also be captioned language and the mask could be captioned personality authorship of the image not 100 clear see the full history of this meme so if we feel creeped out by all the interfaces for artificial intelligence both those that pretend to be neutrally helpful and those that pretend to laugh at our jokes the reason may be that this dilemma reminds us of something slightly cringe and theatrical about personality itself our selves are not bedrock atomic realities they re shaped by collective culture and the autonomy we like to project is mostly a fiction but it s also a necessary fiction projecting and perceiving personality is how we reason about questions of character and perspective and we may end up trusting models more if they can play the same game even if we flinch a little every time they laugh references anders charlie jane all the birds in the sky tor 2016 askell amanda and richie stuart what should an ai s personality be anthropic blog june 8 2024 gibson william neuromancer ace 1984 mallarmé stéphane the crisis of verse in divagations trans barbara johnson harvard university press 2007 warner bros picture presents an annapurna pictures production produced by megan ellison spike jonze vincent landay written and directed by spike jonze her burbank ca distributed by warner home video 2014 tags ai anthropic apple artificial intelligence chatgpt openai personality technology categories 19c 20c cultural analytics deep learning fiction plot plot can language models predict the next twist in a story post author by tedunderwood post date january 5 2024 2 comments on can language models predict the next twist in a story while distant reading has taught us a lot about the history of fiction it hasn t done much yet to explain why we keep turning pages suspense is the word we use to explain that impulse but what is suspense does it require actual anxiety or just uncertainty about what happens next if suspense depends on not knowing what will happen how can we enjoy re reading familiar books smuts 2009 and why do we enjoy being surprised tobin 2018 beyond these big theoretical puzzles there are historical questions scholars might like to ask about the way authors use chapter breaks to structure narrative revelation dames 2023 219 38 right now distant reading can t fully answer any of these questions when we want to measure surprise or novelty for instance we typically measure change in the verbal texture of a story from beginning to end i made a coarse attempt of that kind in a blog post a few years ago other articles use better methods and give us new ways to think about form mcgrath et al 2018 piper et al 2023 but how closely does the pace of verbal change correlate with readers experience of uncertainty or surprise we don t know autoregressive language models offer a tempting new angle on this problem because they re trained specifically to predict how a given text will continue intuitively it feels like we could measure the predictability of a plot by first asking a model to continue the story and then measuring the divergence between predicted continuation and real text even if this isn t exactly how readers form expectations and experience surprise it might begin to give us some leverage on the question researchers have run a loosely similar experiment on very short stories contributed by experimental subjects sap et al 2020 but scaling that up to published novels poses a challenge for one thing language models may not be equally good at imitating every style a contemporary model s failure to predict the next sentence by jane austen might just mean that it s bad at channeling the regency so to factor style out of the question let s ask a model to predict what will happen in say the next three pages of a story and then compare those predictions to its own summaries of the pages when it sees them readers of a certain age will recognize this as a game ernie invites bert to play on sesame street ernie asks bert what happens next in this picture bert anticipates that the man will step in the pail and disaster will ensue to spell the method out more precisely we move through a novel roughly 900 words at a time on each pass we give a language model both a recap of earlier events and a new 900 word passage we ask the model to summarize the new passage and also ask it to predict what will happen next then we compare its prediction to the summary it generates when it actually sees the next passage and measure cosine distance between the two sentence embeddings a large distance means the model did a poor job of predicting what would happen next does this have any relation to human uncertainty i m not claiming that this is a good model of the way readers experience plot we don t have a good model of that yet the more appropriate question to ask is does this correlate at all with anything human readers do we can check by asking a reader to do the same thing read roughly 900 word passages and make predictions about the future then we can compare the human reader s predictions to automatically generated summaries passages were drawn from now in november by josephine johnson and murder is dangerous by saul levinson n 51 passages pearson s r 41 p 01 human predictions are more variable in quality than the model s when i did this for two novels that were complete blanks to me my predictions tended to diverge from the actual course of the story in roughly the same places where the model found prediction difficult so there does seem to be some relationship between a language model s in ability to see what s coming and a human reader s the image above also reveals that there are broad consistent differences between books for both people and models some stories are easier to predict than others a reason not to trust this readers of this story may already anticipate the next twist which is that of course we shouldn t use llms to study uncertainty because these models have already read many of the books we re studying and will presumably already know the plot this is a particularly nasty problem because we don t have a list of the books commercial models were trained on we re flying blind but before we give up let s test how much of a problem this really poses researchers at berkeley have defined a convenient test of the extent to which a model has memorized a book chang et al 2023 in essence they ask the model to fill in missing names running this test chang et al find that gpt 4 remembers many books in detail moreover its ability to fill in masked names correlates with its accuracy on certain other tasks like its ability to estimate date of publication this could be a problem for questions about plot to avoid this problem and also save money i ve been using gpt 3 5 which chang et al find is less prone to memorize books but is that enough to address the problem let s check below i ve plotted the average divergence between prediction and summary for 25 novels on the y axis and gpt 3 5 s ability to supply masked names in those texts on the x axis if memorization was making prediction more accurate we would expect to see a negative correlation predictions divergence from summaries should go down as name_cloze accuracy goes up the y axis is average cosine distance between prediction and summary x axis is gpt 3 5 s accuracy on the name cloze test defined in chang et al 25 books is not enough for a conclusive answer but so far i cannot measure any pattern of that kind if anything there is a faint trend in the opposite direction in an ideal world researchers would use language models trained on open data sets that they know and control but until we get to an ideal world it looks like it may be possible to run proof of concept experiments with things like gpt 3 5 at least if we avoid extremely famous books scrutinizing the image above readers will probably notice that the most predictable book in this sample was zoya by danielle steel although steel has a reputation that may encourage disparaging inferences see dan sinykin big fiction for why i don t think we re in a position to draw those inferences yet the local rhythms that make prediction possible across three pages are not necessarily what critics mean when they use predictable to diss a book so what could we learn from predicting the next three pages to consider one possible payoff it might give us a handle on the way chapter breaks and other divides structure the epistemic rhythms of fiction for instance many readers have noticed that the installments of novels originally published in magazines tend to end with an explicit mystery to ensure that you keep reading haugtvedt 2016 and beekman 2017 in the first installment of arthur conan doyle s hound of the baskervilles which covers two chapters watson and holmes learn about a legendary curse that connects the family of the baskervilles to a fiendish hound in the final lines of the first installment holmes asks the family doctor about footprints found near the body of sir charles baskerville a man s or a woman s holmes asks the doctor s voice sank almost to a whisper as he answered mr holmes they were the footprints of a gigantic hound end installment novels don t have soundtracks but unexplained suggestive new information is as good as a sting bum bum bum the hound of the baskervilles by sidney paget 1902 it appears that we can measure this cliffhanger effect the serial installments of the hound of the baskervilles often end with a moment of heightened mystery at least if inability to predict the next three pages is any measure of mystery when we measure predictive accuracy throughout the story making four different passes to ensure we have roughly 900 word chunks aligned with all the chapter breaks we find that predictions are farther from reality at the ends of serial installments there is no similar effect at other chapter breaks the mean for breaks at serial installments is more than one standard deviation above the mean for other chapter breaks in spite of tiny n this is actually p 05 now this is admittedly a cherry picked example so far i have only looked at seven novels where we can distinguish the ends of serial installments from other kinds of chapter break using data from warhol et al and i don t see this pattern in all of them so i m not yet making any historical argument about serialization and the rise of the cliffhanger i m just suggesting that it s the kind of question someone could eventually address using this method a doctoral student could do it for instance with a locally hosted model i don t recommend doing it with gpt 3 5 because i dropped 150 or so on this post and that might add up across a dissertation some initial tests suggest to me that this approach will produce results significantly different than we re getting with lexical methods since i m explicitly encouraging people to run with this let me also say that someone actually writing a paper using this method might want to tinker with several things before trusting it measuring the distance between the embedding of one prediction sentence and one summary sentence is a crude way to measure expectation and surprise readers don t necessarily form a single expectation about plot maybe it would be better to model expectation as a range of possibility related to this models may need to be nudged to speculate and not just predict that current actions will continue 900 word chunks may not be the only appropriate scale of analysis when readers talk about narrative surprise they re often thinking about larger arcs like who will he marry or who turns out to be the murderer we need a way to handle braided narratives where each chapter is devoted to a different group of characters garrett 1980 in a multi plot story the b or c plot will often not continue across a chapter break but we re in a multi plot narrative ourselves so those problems may be solved by a different group of characters this was just a blog post to share an idea and get people arguing about it tune in next time for our thrilling conclusion bum bum bum code and data used for this post are available on github the ideas discussed here were previously presented in paris at a workshop on ai for the analysis of literary corpora and in copenhagen in a conference on generative methods in the social sciences and humanities i d like to thank the organizers of those events esp thierry poibeau anders munk and rolf lund for stimulating conversation and also many people in attendance especially david bamman lynn cherny and meredith martin in writing code to query the openai api i borrowed snippets from quinn dombrowski and also of course from gpt 4 itself oh brave new world c my thinking about 19c serialization was advanced by conversation with david bishop and eleanor courtemanche and by suggestions from ryan cordell and elizabeth foxwell on bluesky references beekman g 2017 emotional density suspense and the serialization of the woman in white in all theyear round victorian periodicals review 50 1 chang k cramer m soni s bamman d 2023 speak memory an archaeology of books known to chatgpt gpt 4 https arxiv org abs 2305 00118 dames n 2023 the chapter a segmented history from antiquity to the twenty first century princeton university press garrett p 1980 the victorian multiplot novel studies in dialogical form yale university press haugtvedt e 2016 the sympathy of suspense gaskell and braddon s slow and fast sensation fiction in family magazines victorian periodicals review 49 1 mcgrath l higgins d hintze a 2018 measuring modernist novelty journal of cultural analytics https culturalanalytics org article 11030 measuring modernist novelty piper a xu h kolac...
|
