Meta tags:
Headings (most frequently used words):
an, attempt, at, quantifying, changes, to, genre, medium, structuralist, in, post, posts, menu, distant, reading, of, my, autosomes, relinquishing, control, readability, formulas, cosine, similarity, parameters, tf, idf, or, boolean, cont, research, soundtrack, methods, humanities, some, questions, about, centrality, measurements, text, networks, fives, navigation, top, pages, recent, archives, categories, econ, politics, linguistics, rhetoric, writing, sci, tech,
Text of the page (most frequently used words):
the (593), and (244), that (133), are (78), for (72), with (66), but (58), written (55), this (54), not (54), from (47), text (44), all (41), texts (40), similarity (39), have (38), more (37), these (37), which (34), what (33), words (33), between (33), union (33), they (32), when (32), oral (32), most (31), word (31), #medium (30), centrality (30), one (30), cosine (29), can (29), data (28), about (27), their (27), you (26), other (25), has (25), results (25), states (25), point (24), scores (24), two (23), each (23), some (22), there (22), methods (22), also (22), post (21), however (21), addresses (21), use (21), out (20), would (20), measurements (19), than (19), into (19), them (19), corpus (19), then (18), network (18), both (18), could (18), sotu (18), same (18), humanities (17), readability (17), length (17), contraction (17), been (16), different (16), century (16), address (16), audience (16), classifier (16), like (15), first (15), way (15), questions (15), were (15), structuralist (15), rhetoric (14), 2015 (14), idf (14), example (14), should (14), thus (14), may (13), above (13), long (13), think (13), degree (13), used (13), does (13), might (13), method (13), state (13), flesch (13), gender (13), writing (12), digital (12), analysis (12), only (12), person (12), many (12), any (12), critical (12), say (12), spoken (12), boolean (11), just (11), such (11), others (11), see (11), here (11), work (11), matrices (11), president (11), pronouns (11), now (10), research (10), language (10), 2013 (10), control (10), posted (10), seth (10), betweenness (10), metric (10), sense (10), though (10), likely (10), how (10), communication (10), much (10), another (10), its (10), using (10), was (10), obvious (10), change (10), delivered (10), document (10), political (10), features (10), computational (10), ancestry (10), comment (9), formulas (9), leave (9), across (9), position (9), information (9), problem (9), number (9), general (9), well (9), differences (9), frequency (9), argument (9), historical (9), makes (9), take (9), rule (9), trend (9), speeches (9), space (9), second (9), score (9), networks (8), 2016 (8), changes (8), central (8), influence (8), nodes (8), less (8), even (8), graph (8), points (8), terms (8), similar (8), article (8), important (8), returned (8), formula (8), fact (8), either (8), congress (8), public (8), identification (8), presidents (8), non (8), level (8), white (8), scholarship (8), european (8), uncategorized (7), september (7), 2014 (7), genre (7), reading (7), posts (7), without (7), need (7), ultimately (7), because (7), pairs (7), interesting (7), direct (7), our (7), textual (7), hand (7), seen (7), his (7), social (7), distance (7), literary (7), theories (7), too (7), far (7), average (7), study (7), vector (7), appears (7), relative (7), set (7), people (7), feature (7), style (7), function (7), explain (7), english (7), authors (7), difficult (7), kincaid (7), grade (7), explanation (7), spahr (7), irish (7), wordpress (6), com (6), blog (6), found (6), march (6), parameters (6), who (6), said (6), mathematical (6), had (6), isn (6), node (6), case (6), straightforward (6), lot (6), measurement (6), again (6), high (6), come (6), following (6), enough (6), appear (6), right (6), humanistic (6), comes (6), move (6), almost (6), always (6), tools (6), early (6), effect (6), vectors (6), look (6), models (6), lower (6), american (6), negative (6), very (6), find (6), comparison (6), given (6), according (6), good (6), whether (6), progressive (6), mexican (6), ancestrydna (6), view (5), content (5), world (5), technology (5), 2012 (5), distant (5), don (5), james (5), metrics (5), over (5), simple (5), potential (5), pair (5), becomes (5), description (5), therefore (5), usage (5), talking (5), rather (5), himself (5), those (5), being (5), things (5), previous (5), center (5), star (5), possible (5), since (5), close (5), although (5), seems (5), through (5), assumed (5), ways (5), variation (5), new (5), calculated (5), step (5), researcher (5), around (5), probably (5), baker (5), own (5), sometimes (5), history (5), part (5), formal (5), documents (5), twentieth (5), messages (5), why (5), mary (5), list (5), represented (5), representing (5), nixon (5), particularly (5), python (5), few (5), plural (5), reference (5), ceremonial (5), third (5), particular (5), indeed (5), nineteenth (5), bayes (5), training (5), accuracy (5), note (5), characters (5), sentences (5), applied (5), underwood (5), representation (5), zacatecas (5), write (4), bar (4), log (4), already (4), project (4), linguistics (4), mining (4), british (4), semiotics (4), science (4), maps (4), literature (4), categories (4), january (4), april (4), attempt (4), recent (4), shows (4), put (4), moretti (4), after (4), perhaps (4), perspective (4), simply (4), informative (4), learn (4), theory (4), calculating (4), something (4), context (4), figure (4), geodesics (4), limited (4), major (4), model (4), whose (4), next (4), off (4), process (4), notions (4), structural (4), directly (4), never (4), run (4), natural (4), course (4), form (4), goal (4), debate (4), rhetorical (4), matrix (4), higher (4), tension (4), multiple (4), structuralism (4), end (4), working (4), incommensurable (4), quantitative (4), possibility (4), readings (4), certainly (4), mean (4), evidence (4), topic (4), bound (4), remains (4), correspond (4), count (4), algorithms (4), obviously (4), euclidean (4), calculate (4), contains (4), represent (4), want (4), compare (4), suggest (4), graphed (4), chronologically (4), 1973 (4), 1790 (4), choose (4), construct (4), nation (4), informality (4), lexical (4), contrast (4), majority (4), term (4), diction (4), middle (4), your (4), difference (4), themselves (4), naïve (4), frequencies (4), test (4), time (4), imagine (4), presidential (4), sure (4), understand (4), make (4), aren (4), useful (4), 100 (4), predict (4), thing (4), doing (4), field (4), activist (4), fiction (4), politically (4), stories (4), trace (4), mexico (4), maternal (4), italian (4), region (4), southern (4), scandinavian (4), ancestral (4), get (3), site (3), comments (3), reader (3), create (3), genetic (3), detail (3), library (3), attention (3), october (3), february (3), quantifying (3), blue (3), men (3), response (3), gif (3), clearly (3), relatively (3), concept (3), doubt (3), cases (3), seemingly (3), will (3), once (3), read (3), order (3), valuable (3), intellectual (3), via (3), several (3), believe (3), freeman (3), reality (3), toward (3), contact (3), discussion (3), goes (3), least (3), respect (3), addressing (3), based (3), explicitly (3), intuitive (3), thinking (3), answer (3), researchers (3), utilize (3), needed (3), question (3), exactly (3), left (3), visualize (3), best (3), require (3), anyone (3), larger (3), discover (3), precisely (3), principles (3), going (3), trying (3), job (3), years (3), latter (3), start (3), back (3), follow (3), place (3), complete (3), build (3), conference (3), humanists (3), date (3), impact (3), interpretation (3), europe (3), input (3), issue (3), seem (3), overlooked (3), strong (3), shift (3), immediate (3), tradition (3), wilson (3), last (3), 000 (3), result (3), shared (3), great (3), every (3), numbers (3), range (3), copied (3), master (3), files (3), eisenhower (3), 1956 (3), making (3), algorithm (3), claim (3), scholars (3), delivery (3), full (3), call (3), primary (3), house (3), secondary (3), speech (3), goals (3), standards (3), contractions (3), during (3), decrease (3), rates (3), beginning (3), rarely (3), cultural (3), extent (3), including (3), authority (3), analyzing (3), runs (3), must (3), ten (3), rate (3), automotive (3), label (3), taken (3), provides (3), categorize (3), unknown (3), individual (3), humanist (3), novels (3), cannot (3), common (3), tend (3), studies (3), comparing (3), uses (3), proxies (3), forward (3), often (3), easily (3), understood (3), down (3), current (3), difficulty (3), harder (3), doesn (3), single (3), large (3), explanations (3), demographic (3), programs (3), mfa (3), essay (3), becoming (3), young (3), rise (3), program (3), fluid (3), become (3), gendered (3), male (3), female (3), allington (3), lineage (3), mexicans (3), amount (3), mother (3), euro (3), ancestors (3), aguascalientes (3), family (3), split (3), paternal (3), father (3), amerind (3), website (2), report (2), sign (2), subscribed (2), subscribe (2), technaverbascripta (2), account (2), collin (2), brooke (2), zero (2), economy (2), steven (2), bias (2), culture (2), politics (2), edm (2), november (2), december (2), july (2), august (2), 2017 (2), cont (2), relinquishing (2), autosomes (2), visualizations (2), axis (2), top (2), planet (2), human (2), medieval (2), select (2), franco (2), lists (2), done (2), debates (2), better (2), gives (2), sensitive (2), scale (2), local (2), phrase (2), robust (2), disposable (2), lead (2), conclusions (2), underlying (2), math (2), still (2), measure (2), necessarily (2), idea (2), ostensibly (2), determining (2), convert (2), probability (2), neither (2), connecting (2), complicated (2), falls (2), basic (2), equally (2), aspects (2), ideal (2), activity (2), permits (2), channel (2), focal (2), mainstream (2), flow (2), semantic (2), adopt (2), essentially (2), pathway (2), meaning (2), potentially (2), him (2), writers (2), responding (2), attempts (2), three (2), distinct (2), properties (2), located (2), structurally (2), size (2), face (2), overall (2), determine (2), remarkably (2), discussing (2), yet (2), answers (2), correlation (2), exist (2), remain (2), matter (2), ask (2), setting (2), proxy (2), core (2), proximity (2), anything (2), normal (2), dialectical (2), calls (2), her (2), consciously (2), exists (2), enact (2), positivist (2), recursive (2), j_w_baker (2), critique (2), helps (2), melvinwevers (2), replied (2), where (2), sorts (2), building (2), challenging (2), meaningful (2), ground (2), gathering (2), strict (2), categorization (2), reject (2), empirical (2), 21st (2), worldview (2), par (2), excellence (2), collections (2), advantage (2), raised (2), brilliant (2), interest (2), specifically (2), million (2), show (2), tomorrow (2), granular (2), despite (2), shorter (2), while (2), produce (2), carter (2), initial (2), 1801 (2), fails (2), began (2), increase (2), 1913 (2), corresponding (2), nevertheless (2), near (2), year (2), norm (2), raw (2), below (2), compared (2), intuitively (2), modeling (2), hates (2), dogs (2), cats (2), loves (2), birds (2), cows (2), removed (2), comprised (2), operation (2), treat (2), dot (2), product (2), returns (2), check (2), series (2), analyses (2), quantify (2), turn (2), row (2), column (2), typically (2), constructing (2), stemmed (2), called (2), summarizing (2), among (2), interpreted (2), tell (2), expect (2), measured (2), adapting (2), occurred (2), radio (2), 1980 (2), provide (2), opportunity (2), mediums (2), appropriate (2), script (2), dennis (2), until (2), nearer (2), prior (2), interpret (2), bit (2), coming (2), days (2), overly (2), paragraphs (2), historically (2), participants (2), role (2), shaping (2), markers (2), depends (2), effects (2), pronoun (2), takes (2), indicate (2), citizens (2), marker (2), excessively (2), proper (2), against (2), affect (2), correct (2), avoid (2), details (2), true (2), motivates (2), subjective (2), conclusion (2), drawn (2), pretty (2), break (2), wall (2), broken (2), apostrophes (2), little (2), changing (2), attested (2), accepted (2), traced (2), ignore (2), discovered (2), turns (2), terror (2), twenty (2), centuries (2), media (2), nothing (2), influences (2), frequent (2), type (2), dark (2), indicator (2), murder (2), mysteries (2), sports (2), made (2), classification (2), within (2), labeled (2), hidden (2), return (2), mix (2), pointed (2), benchmarks (2), cline (2), pride (2), prejudice (2), interpreting (2), finding (2), under (2), exceedingly (2), small (2), readers (2), 19th (2), wildly (2), standpoint (2), exact (2), copies (2), moment (2), really (2), smart (2), pursuing (2), noise (2), inclined (2), capture (2), entries (2), slightly (2), decreasing (2), ended (2), line (2), syllables (2), semi (2), colons (2), originally (2), levels (2), military (2), industry (2), ensure (2), college (2), studying (2), popular (2), canonical (2), ones (2), easier (2), thesis (2), lim (2), turned (2), argues (2), physical (2), technical (2), sciences (2), culturally (2), relevant (2), ted (2), examples (2), wikipedia (2), scholar (2), deeply (2), ideology (2), won (2), maybe (2), sociology (2), apolitical (2), phd (2), rightly (2), ideological (2), america (2), recognize (2), present (2), open (2), agree (2), arguments (2), piece (2), sharply (2), delineated (2), along (2), previously (2), else (2), signifiers (2), women (2), eyes (2), peak (2), hair (2), body (2), certain (2), counter (2), predicting (2), side (2), volatile (2), toolbox (2), final (2), deep (2), looking (2), eastern (2), per (2), analabha (2), basu (2), exhibit (2), african (2), matches (2), san (2), luis (2), potosi (2), fourth (2), home (2), trivial (2), ish (2), revolution (2), native (2), aztec (2), regions (2), surprised (2), iberian (2), celtic (2), amounts (2), cluster (2), damn (2), paper (2), western (2), panel (2), northern (2), assuming (2), knew (2), started, design, name, email, loading, collapse, manage, subscriptions, privacy, join, subscribers, free, phylogenetic, orbiting, frog, ian, bogost, literacy, gene, expression, dienekes, contagions, ars, technica, sci, tech, silvae, rhetoricae, planned, obsolescence, page, tectonics, journal, jeff, rice, derek, mueller, gifford, clay, spinuzzi, ancient, scripts, allison, hitt, alex, reid, replicated, typo, omniglot, linguischtick, geoffrey, sampson, translation, hedge, super, landsburg, spengler, ross, wolfe, overcoming, graphic, dissent, magazine, code, econ, stephen, ramsay, scott, weingart, sapping, nodus, labs, flowing, futurism, june, archives, lying, misleading, truncate, grammatical, anaphors, command, robot, pages, search, older, navigation, wild, india, trust, apartment, mad, netflix, instant, que, fireball, brandy, triple, sec, jose, cuervo, silver, bud, lite, lime, alcoholic, beverages, kitchen, whipped, cream, cookie, dough, chocolate, chips, peanut, butter, cups, toppings, frozen, yogurt, built, city, starship, tickets, paradise, eddie, money, memory, tiesto, beyond, night, lionel, richie, played, songs, itunes, invaders, pat, shipman, bibliography, murphy, warriors, cloisters, asian, origins, christopher, beckwith, footsteps, genghis, khan, john, defrancis, bourgeois, books, desk, asked, fives, concepts, comprehend, assume, demonstrates, worthless, skewed, exert, inform, influential, whole, underestimate, temporal, correlates, determinative, accessing, writer, phenomenon, needs, consider, wrong, expunge, functional, demonstrated, surprising, messy, spaghetti, monster, meeting, requires, recourse, probabilities, probabilistic, give, complexities, involved, muscle, exerted, linking, nor, strictly, geodesic, connects, completely, situation, messier, preceding, formation, orient, mental, construction, interpersonal, attempted, productive, creator, bigrams, significations, slip, opposite, extreme, low, occupant, peripheral, isolates, involvement, cuts, active, participation, ongoing, begin, whom, develop, somehow, thick, speculate, defined, visibility, says, grapple, uniquely, possessed, maximum, largest, minimum, maximally, compete, defining, property, measures, stated, theme, earlier, hub, wheel, shown, universally, intuition, sort, special, structure, unique, transfers, semantics, notion, linton, merely, linear, algebra, generates, positive, theoretically, knowing, defense, application, gained, posing, discovers, string, applying, edges, dozens, employed, suited, dependent, edge, weight, relevance, critics, alternates, smaller, possess, signified, significant, meaningfully, tackled, enacting, discuss, regarding, topics, fair, game, skeptical, peer, reviewers, balk, subtly, remove, nuance, searching, nuances, refining, coordinated, effort, knee, jerk, critiques, requests, author, unhelpful, chosen, real, former, theorist, posed, comfortable, living, enjoy, subjecting, positivism, sustained, somewhere, beyondmining, suggested, tries, development, reaction, quant, cliometrics, etc, sethlargo, throws, light, fundamentally, got, interested, condemned, forever, gather, ever, unnatural, vivisection, insert, twitter, brings, contradiction, knowledge, existence, overhaul, melvin, wevers, techniques, moving, forces, requiring, inflexible, ontologies, orientation, foucault, derrida, tendency, rely, uncover, socially, constructed, experimental, realize, incompatibility, statement, gets, heart, purpose, evoke, papers, citing, actual, generate, generated, geographical, sole, ownership, implications, usefulness, contexts, personal, hero, geography, workshop, seeks, explore, validity, digitally, mined, focus, digitizing, developing, lacking, provided, projects, developers, solicited, historians, utrecht, university, raises, ists, pooling, traditional, focuses, lse, soundtrack, later, needless, correlate, broad, stroke, approach, caveats, demands, tends, lengthier, longest, 1981, message, original, orally, shortest, thomas, jefferson, took, decades, woodrow, broke, drop, coolidge, hoover, 1924, 1932, 1919, 1920, sudden, rebound, none, terribly, lengthy, counts, 057, 818, diachronically, longer, resulting, thousands, impossible, tool, stopword, four, info, handy, calculator, keeping, hate, love, dog, cat, bird, cow, stop, remaining, let, simplified, wondering, business, quick, stab, reflects, stability, challenged, undermined, parallel, initiated, medial, alteration, ambiguity, invite, deeper, substantial, turning, color, shaded, txt, independently, total, fdr, 1945, 1972, 1974, 1978, televised, rare, analyze, annual, objects, cleaned, stopwords, porter, stemming, muhlestein, 1791, 1792, 1793, 1917, afterward, premodern, modern, eras, discourse, transition, heat, bold, occurring, clusters, diachronic, finished, muddying, dump, muster, energy, corpora, alongside, intimately, linked, motivate, choices, reached, choice, avoidance, drops, fifty, percent, feel, alone, exigent, citizenry, signaled, salutations, television, addressed, fellow, denoting, salutation, members, senate, shorten, eschews, subject, verb, sound, sounds, stilted, amusing, witnessed, film, class, individuals, shortening, noted, ronald, reagan, conversational, tone, maintaining, tactic, informal, instead, professional, manner, grit, favored, devices, importance, deliberate, disruptions, disruption, observed, plurals, intimate, professed, value, encoded, grammatically, referenced, unit, existing, facilitating, invitation, share, experiences, plurality, willingness, unwillingness, frequently, mark, display, functionally, instances, stark, sixteenth, generally, today, prescriptive, dictate, avoided, swift, addison, 1700s, motivated, arbitrary, standard, soon, speaking, before, sairio, 2010, displays, disparate, lesser, surface, noticeable, locus, figures, tallies, contain, pronomial, inflections, reflexives, emerged, fitting, requirement, optimal, twice, occur, root, exclusively, reasons, categorizing, synonymous, coding, vast, risk, uncovering, preferences, events, gone, speechmaking, foreign, affairs, neglect, reduced, influenced, exigency, shifting, unusable, terrorism, approximately, decided, ran, nltk, times, randomly, shuffling, utilized, considerations, starts, closer, considers, weak, football, contribution, checks, closest, assigns, explained, containing, abstract, illustration, procedure, loper, processing, occurs, steps, defines, finally, labels, types, tokens, symbols, punctuation, helping, naive, described, dozen, places, throw, posterity, sake, highly, ambiguous, suggests, margins, mode, understanding, hesitant, apparent, melville, austen, dissimilarity, benchmark, compute, sensibility, ago, subsequently, lost, quora, thread, recommended, commonly, asking, group, judgments, absolutely, specimens, stylistic, priori, moby, dick, nature, itself, leaves, hell, habit, parameter, adjustment, fun, relying, default, whatever, kind, controlling, producing, signal, suggesting, divergent, warrant, accurately, reflect, attached, discrepancies, ending, dissimilar, scikit, pairwise, module, tfidfvectorizer, controlled, compliments, muhlstein, wasn, wanted, induces, modification, inverse, clear, numeric, representations, brief, recap, sparse, entered, entry, problems, reason, curious, know, nlp, inputs, vagaries, steinbeck, hemingway, easy, reads, short, monosyllabic, dialogue, indicating, 6th, grades, periods, deserve, equate, dealt, marking, sentence, dividers, designed, gauge, adapted, meant, educations, school, diplomas, complex, begeny, limitations, complexity, economics, sales, genres, uncovered, contra, supports, matters, simplify, dumb, elevated, delivering, discussed, seconds, googling, nice, offers, textstat, quality, notably, elvin, applies, inaugurals, discovering, marked, 18th, parcel, intellectualism, broadly, anti, presidency, lookout, codes, adopting, statistical, operations, matthew, jocker, fourier, transform, retrieval, multiplied, produced, navy, education, territory, upper, graduate, chapter, link, ease, counterintuitively, highest, easiest, tops, 120, 500, nearly, combination, perform, calculation, entire, section, lexile, developed, assist, educators, choosing, ages, picked, documentation, soldiers, schooling, skepticism, conservatism, tentative, suspicious, purporting, support, platform, piss, everyone, spectrum, held, conform, confirming, orthodoxy, orthodoxies, confident, gregory, clark, opens, stupid, usually, immediately, piraha, possesses, syntax, dunno, sociologists, psychologists, recognized, quietly, means, pinker, jon, haidt, pushing, lately, heterodox, academy, big, demography, imply, statistic, problematic, partially, ameliorates, gap, foreclose, facing, window, recognizing, keeps, inquiry, echo, chamber, wonderful
Text of the page (random words):
cation although the method has been described for humanists in a dozen places at this point i ll throw my own description into the mix for posterity s sake naïve bayes classification occurs in three steps first the researcher defines a number of features found in all texts within the corpus typically a list of the most frequent words second the researcher shows the classifier a limited number of texts from the corpus that are labeled according to text type the training set finally the researcher runs the classifier algorithm on a larger number of texts whose labels are hidden the test set using feature information discovered in the training set including information about the number of different text types the classifier attempts to categorize the unknown texts another algorithm can then check the classifier s accuracy rate and return a list of tokens words symbols punctuation that were most informative in helping the classifier categorize the unknown texts more intuitively the method can be explained with the following example taken from natural language processing with python imagine we have a corpus containing sports texts automotive texts and murder mysteries figure 2 provides an abstract illustration of the procedure used by the naïve bayes classifier to categorize the texts according to their features loper et al explain in the training corpus most documents are automotive so the classifier starts out at a point closer to the automotive label but it then considers the effect of each feature in this example the input document contains the word dark which is a weak indicator for murder mysteries but it also contains the word football which is a strong indicator for sports documents after every feature has made its contribution the classifier checks which label it is closest to and assigns that label to the input each feature influences the classifier therefore the number and type of features utilized are important considerations when training a classifier given the sotu corpus s word count approximately 2 million words i decided to use as features the 2 000 most frequent words in the corpus the top 10 i ran nltk s classifier ten times randomly shuffling the corpus each time so the classifier could utilize a new training and test set on each run the classifier s average accuracy rate for the ten runs was 86 9 after each test run the classifier returned a list of most informative features the majority of which were content words such as authority or terrorism however a problem a direct comparison of these words is not optimal given my goals i could point out for example that authority is twice as likely to occur in written than in oral states of the union i could also point out that the root terror is found almost exclusively in the oral corpus nevertheless these results are unusable for analyzing the effects of media on content for historical reasons categorizing the sotu into oral and written addresses is synonymous with coding the texts by century the vast majority of written addresses were delivered in the nineteenth century the majority of oral speeches were delivered in the twentieth and twenty first centuries analyzing lexical differences thus runs the risk of uncovering not variation between oral and written states of the union a function of media but variation between nineteenth and twentieth century usage a function of changing style preferences or differences between political events in each century a function of history the word authority has likely just gone out of style in political speechmaking terror is a function of twenty first century foreign affairs there is nothing about medium that influences the use or neglect of these terms a lexical comparison of written and oral states of the union must therefore be reduced to features least likely to have been influenced by historical exigency or shifting usage in the lists of informative features returned by the naïve bayes classifier pronouns and contraction emerged as two features fitting that requirement relative frequencies of first second and third person pronouns relative frequencies of select first and second person pronouns relative frequencies of apostrophes and negative contraction it turns out that pronoun usage is a noticeable locus of difference between written and oral states of the union the figures above show relative frequencies of first second and third person pronouns in the two text categories the tallies in the first graph contain all pronomial inflections including reflexives as discovered by the naïve bayes classifier first and second person pronouns are much more likely to be found in oral speeches than in written addresses the second graph above displays particularly disparate pronouns we us you and to a lesser extent your third person pronouns however surface equally in both delivery mediums the third graph shows relative frequency rates of apostrophes in general and negative contractions in particular in the two sotu categories contraction is another mark of the oral medium in contrast written states of the union display very little contraction indeed the relative frequency of negative contraction in the written sotu corpus is functionally zero only 3 instances this stark contrast is not a function of changing usage negative contraction is attested as far back as the sixteenth century and was well accepted during the nineteenth century contraction generally is also well attested in nineteenth century texts see this post at language log however both today and in the nineteenth century prescriptive standards dictate that contractions are to be avoided in formal writing a norm which sairio 2010 has traced to swift and addison in the early 1700s thus if not the written medium directly then the cultural standards for the written medium have motivated presidents to avoid contraction when working in that medium presidents ignore this arbitrary standard as soon as they find themselves speaking before the public the conclusion to be drawn from these results should have been obvious from the beginning the differences between oral and written states of the union are pretty clearly a function of a president s willingness or unwillingness to break the wall between himself and his audience that wall is frequently broken in oral speeches to the public but rarely broken in written addresses to congress as seen above plural reference we us and direct audience address you your are favored rhetorical devices in oral states of the union but less used in the written documents the importance underlying this difference is that both features plural reference and direct audience address are deliberate disruptions of the ceremonial distance that exists between president and audience during a formal address this disruption in my view can be observed most explicitly in the use of the pronouns we and us the oral medium motivates presidents to construct with the use of these first person plurals an intimate identification between themselves and their audience plurality a professed american value is encoded grammatically with the use of plural pronouns president and audience are different and many but are referenced in oral speeches as one unit existing in the same subjective space also facilitating a decrease in ceremonial distance as seen above is the use of second person you at much higher rates in oral than in written states of the union i would suggest that the oral medium motivates presidents to call direct attention to the audience and its role in shaping the state of the nation in other cases second person pronouns may represent an invitation to the audience to share in the president s experiences contraction is a secondary feature of the oral medium s attempt at audience identification if a president s goal is to build identification with american citizens and to shorten the ceremonial distance between himself and them then clearly no president will adopt a formal diction that eschews contraction contraction either negative or subject verb is the informality marker par excellence non contraction on the other hand though it may sound normal in writing sounds stilted and excessively proper in speech the amusing effect of this style of diction can be witnessed in the film true grit in a nation comprised of working and middle class individuals this excessively proper diction would work against the goals of shortening ceremonial distance and constructing identification many scholars have noted ronald reagan s use of contraction to affect a conversational tone in his states of the union but contraction appears as an informality marker across multiple oral speeches in the sotu corpus in contrast when a president s address takes the form of a written document maintaining ceremonial distance seems to be the general tactic as presidents follow correct written standards and avoid contractions the president does not go out of his way to construct identification with his audience congress through informal diction instead the goal of the written medium is to report the details of the state of the nation in a professional distant manner what i think these results indicate is that the state of the union s primary audience changes from medium to medium this fact is signaled even by the salutations in the sotu corpus the majority of oral addresses delivered via radio or television are explicitly addressed to fellow citizens or some other term denoting the american public in written addresses to congress however the salutation is almost always limited to members of the house and the senate two lexical effects of this shift in audience are pronoun choice and the use or avoidance of contraction we us you the frequency of these pronouns drops by fifty percent or more when presidents move from the oral to the written medium from an address to the public to an address to congress the same can be said for contraction presidents it seems feel less need to construct identification through these informality markers through plural and second person reference when their audience is congress alone in contrast audience identification becomes an exigent goal when the citizenry takes part in the state of the union address to put the argument another way the sotu s change in medium has historically occurred alongside a change in genre participants these intimately linked changes motivate different rhetorical choices does a president choose or not choose to construct a plural identification between himself and his audience we us or to call attention to the audience s role you in shaping the state of the nation does a president choose or not choose to use obvious informality markers i e contraction the answer depends on medium and on the participants reached via that medium congress or the american people tomorrow i ll post results from two 30 run topic models of the written oral sotu corpora posted in rhetoric technology writing and semiotics leave a comment january 19 2016 by seth long an attempt at quantifying changes to genre medium rule et al s 2015 article on the state of the union makes the rather bold claim for literary and rhetorical scholars that changes to the sotu s medium of delivery has had no effect on the form of the address measured as co occurring word clusters as well as cosine similarity across diachronic document pairs i ve just finished an article muddying their results a bit so here s the initial data dump i ll do it in a series of posts full argument to follow if i can muster enough energy in the coming days to convert an overly complicated argument into a few paragraphs first cosine similarity essentially rule et al calculate the cosine similarity between each set of two sotu addresses chronologically 1790 and 1791 1790 and 1792 1790 and 1793 and so on until each address has been compared to all other addresses they discover high similarity measurements nearer to 1 across most of the document space prior to 1917 and lower similarity measurements nearer to 0 afterward which they interpret as a shift between premodern and modern eras of political discourse they visualize these measurements in the transition matrices which look like heat maps in figure 2 of their article adapting a python script written by dennis muhlestein i calculated the cosine similarity of states of the union delivered in both oral and written form in the same year this occurred in 8 years a total of 16 texts fdr in 1945 eisenhower in 1956 and nixon in 1973 delivered written messages to congress as well as public radio addresses summarizing the written messages nixon in 1972 and 1974 and carter in 1978 1980 delivered both written messages and televised speeches these 8 textual pairs provide a rare opportunity to analyze the same annual address delivered in two mediums making them particularly appropriate objects of analysis the texts were cleaned of stopwords and stemmed using the porter stemming algorithm cosine similarity of oral written sotu pairs the results are graphed above not a lot of numbers so there s no point turning them into a color shaded matrix as rule et al do the cosine similarity measurements range from 0 67 a higher similarity to 0 40 a lower similarity the cosine similarity measurement of all written and all oral sotu texts copied chronologically into two master txt files is 0 55 remarkably close to the average of the 8 pairs measured independently there is much ambiguity in these measurements on one hand they can be interpreted to suggest that rule et al overlooked differences between oral and written states of the union the measurements invite a deeper analysis of the corpus on the other hand the measurements also tell us not to expect substantial variation in the article to take a quick stab at summarizing my argument i suggest that this metric among others reflects a genre whose stability is challenged but not undermined by changes to medium as well as parallel changes initiated by the medial alteration but you re probably wondering what this cosine similarity business is all about without going into too much detail vector space models that s what this method is called can be simplified with the following intuitive example let s say we want to compare the following texts text 1 mary hates dogs and cats text 2 mary loves birds and cows one way to quantify the similarity between the texts is to turn their words into matrices with each row representing one of the texts and each column representing every word that appears in either of the texts typically when constructing a vector space model stop words are removed and remaining words are stemmed so the complete word list representing texts 1 and 2 would look like this 1 mary 2 hate 3 love 4 dog 5 cat 6 bird 7 cow each text however contains only some of these words we represent this fact in each text s matrix each word from the complete word list that appears in a text is represented as a 1 in the matrix each word th...
|
