Meta tags:
description= Genteelly Observing the Enemy since 2011;
Headings (most frequently used words):
the, python, what, exactly, does, for, do, step, in, of, historical, recent, skulking, holes, and, corners, world, siege, sale, cleaning, text, with, sabbatical, rear, view, mirror, you, might, be, millner, from, source, to, data, where, historians, are, 2017, have, mentioned, future, is, digital, early, modern, spain, on, budget, part, reminder, smh, 2019, computer, program, scholars, this, historian, posts, comments, categories, bibliography, archives, blogroll, parsing, unparseable, converting, semi, structured, document, into, files,
Text of the page (most frequently used words):
the (548), and (263), you (251), that (149), with (119), for (114), python (74), but (74), can (70), have (68), from (59), this (58), your (53), all (51), code (50), into (45), text (44), more (44), are (44), data (43), what (39), like (38), some (38), just (35), those (35), will (31), want (31), which (30), one (29), list (28), other (27), use (26), not (26), then (26), there (25), out (25), they (25), create (24), about (24), each (24), history (23), information (23), #digital (22), could (22), document (22), does (21), their (21), few (21), them (21), how (21), over (20), any (20), was (20), well (19), also (19), word (19), already (18), world (18), most (18), has (18), now (17), early (17), modern (17), may (17), spanish (17), been (17), programming (17), make (17), words (17), computer (17), need (16), would (16), lots (16), even (16), own (16), 2018 (15), war (15), siege (15), research (15), when (15), catalan (15), many (15), its (15), historians (15), specific (15), analysis (15), documents (15), maps (14), least (14), bit (14), had (14), entries (14), get (13), 2014 (13), might (13), see (13), through (13), because (13), time (13), lot (13), several (13), libraries (13), historical (12), 2012 (12), august (12), 2013 (12), 2015 (12), 2019 (12), two (12), using (12), another (12), people (12), much (12), things (12), way (12), learning (12), folio (12), online (11), book (11), exactly (11), next (11), our (11), find (11), these (11), long (11), after (11), here (11), should (11), analyze (11), same (11), letters (11), only (11), results (11), different (11), excel (11), december (10), january (10), 2016 (10), together (10), year (10), who (10), barcelona (10), were (10), where (10), every (10), used (10), done (10), his (10), know (10), above (10), probably (10), language (10), clean (10), source (10), don (10), work (10), etc (10), maybe (10), automate (10), web (10), september (9), 2017 (9), zotero (9), day (9), process (9), think (9), back (9), various (9), case (9), separate (9), than (9), since (9), years (9), map (9), important (9), set (9), first (9), between (9), mentioned (9), big (9), available (9), dictionary (9), files (9), item (9), convert (9), format (9), errors (9), july (8), spain (8), search (8), take (8), number (8), off (8), count (8), making (8), authors (8), databases (8), read (8), sure (8), allows (8), example (8), file (8), easily (8), string (8), means (8), date (8), require (8), page (8), website (7), comments (7), skulking (7), holes (7), corners (7), blog (7), march (7), june (7), october (7), sources (7), methodology (7), cleaning (7), new (7), fortress (7), once (7), figure (7), did (7), better (7), access (7), small (7), museum (7), before (7), say (7), course (7), yak (7), full (7), really (7), start (7), future (7), run (7), still (7), structured (7), look (7), journal (7), yet (7), common (7), person (7), numbers (7), comment (6), content (6), wars (6), november (6), february (6), note (6), library (6), great (6), sale (6), recent (6), someone (6), including (6), let (6), sense (6), following (6), very (6), spent (6), particular (6), particularly (6), thing (6), shaving (6), never (6), steps (6), keep (6), check (6), wikipedia (6), whatever (6), order (6), allow (6), change (6), easy (6), value (6), nlp (6), perform (6), changes (6), jupyter (6), notebook (6), notes (6), tools (6), created (6), numerous (6), type (6), project (6), without (6), low (6), ocr (6), program (6), nouns (6), versions (6), features (6), texts (6), based (6), haven (6), enough (6), programs (6), functions (6), started (5), site (5), com (5), name (5), view (5), free (5), bibliography (5), april (5), sieges (5), military (5), google (5), devonthink (5), sabbatical (5), posts (5), via (5), leave (5), around (5), possibly (5), learn (5), doesn (5), ways (5), madrid (5), imagine (5), three (5), end (5), include (5), getting (5), real (5), fact (5), across (5), hand (5), line (5), combine (5), details (5), graph (5), past (5), step (5), yourself (5), campaign (5), little (5), add (5), dates (5), structure (5), pretty (5), entry (5), dozen (5), good (5), parse (5), record (5), copy (5), result (5), automatically (5), bunch (5), place (5), individual (5), thousands (5), historian (5), hanging (5), hard (5), projects (5), places (5), output (5), question (5), machine (5), working (5), letter (5), manipulate (5), types (5), tcp (5), wordpress (4), write (4), 2011 (4), france (4), conference (4), millner (4), finding (4), held (4), part (4), worth (4), visit (4), especially (4), wouldn (4), otherwise (4), fortunately (4), beginning (4), events (4), citadel (4), tell (4), fort (4), 1640 (4), while (4), why (4), give (4), main (4), right (4), said (4), heritage (4), catalonia (4), similar (4), therefore (4), guess (4), instead (4), english (4), city (4), succession (4), whole (4), during (4), four (4), semi (4), names (4), something (4), actually (4), almost (4), come (4), simple (4), scraping (4), open (4), deal (4), basic (4), series (4), doing (4), months (4), key (4), sorts (4), resulting (4), loop (4), delimiter (4), multiple (4), cleaned (4), camp (4), info (4), transcription (4), others (4), looks (4), regular (4), expressions (4), parsing (4), precise (4), problem (4), terms (4), phrases (4), likely (4), put (4), github (4), fruit (4), generally (4), ocred (4), dirty (4), kind (4), form (4), proper (4), entities (4), interactive (4), workflow (4), along (4), mean (4), makes (4), science (4), rename (4), download (4), network (4), made (4), eebo (4), pages (4), index (4), processing (4), books (4), coding (4), scholars (4), 000 (4), email (3), subscribe (3), account (3), rohan (3), british (3), army (3), primary (3), officers (3), taking (3), marlborough (3), art (3), resources (3), professional (3), miscellaneous (3), interested (3), help (3), smh (3), panels (3), coming (3), topic (3), definitely (3), wife (3), less (3), force (3), decided (3), build (3), starting (3), dozens (3), 1705 (3), thought (3), always (3), correct (3), forward (3), difference (3), central (3), recently (3), referendum (3), final (3), performing (3), interesting (3), somebody (3), rest (3), languages (3), ended (3), duke (3), service (3), allied (3), point (3), fall (3), king (3), memory (3), him (3), close (3), aren (3), unlike (3), today (3), currently (3), salses (3), left (3), medieval (3), pain (3), scratch (3), later (3), century (3), again (3), mostly (3), flags (3), far (3), having (3), week (3), last (3), whether (3), sparql (3), battles (3), location (3), post (3), hopefully (3), play (3), try (3), knowledge (3), until (3), qgis (3), reading (3), often (3), eventually (3), reason (3), images (3), journals (3), lists (3), hit (3), bits (3), statistical (3), function (3), returns (3), split (3), methods (3), notice (3), delete (3), standard (3), won (3), editor (3), standardizing (3), regex (3), sample (3), preprocessing (3), nor (3), separated (3), adding (3), top (3), level (3), relational (3), assuming (3), eye (3), sophisticated (3), database (3), searching (3), compared (3), nice (3), days (3), paragraph (3), collection (3), publication (3), compare (3), quickly (3), task (3), textual (3), ones (3), examples (3), image (3), digitized (3), institutional (3), provides (3), corpus (3), variable (3), works (3), syntax (3), repeat (3), folder (3), questions (3), software (3), answer (3), curve (3), rules (3), semester (3), department (3), stats (3), manually (3), entire (3), given (3), both (3), area (3), pdf (3), table (3), websites (3), quantitative (3), edit (3), pre (3), complicated (3), feature (3), natural (3), published (3), humanities (3), behind (3), commands (3), print (3), worry (3), microsoft (3), version (3), packaged (3), vep (3), required (2), log (2), sign (2), subscribed (2), louis (2), smhblog (2), shot (2), lineages (2), archives (2), group (2), society (2), strategy (2), operations (2), graphics (2), fortifications (2), dead (2), conferences (2), battle (2), 18c (2), ememh (2), teaching (2), admin (2), categories (2), jostwald (2), rear (2), mirror (2), older (2), panelists (2), proposals (2), european (2), love (2), considered (2), tramping (2), olympics (2), ditch (2), taken (2), commemorating (2), random (2), overlooking (2), submission (2), reapers (2), revolt (2), centuries (2), human (2), rights (2), torture (2), surprise (2), castle (2), exhibition (2), montjuïc (2), interactivity (2), says (2), opening (2), maintained (2), fortresses (2), below (2), hill (2), park (2), either (2), named (2), fast (2), built (2), decades (2), chapter (2), provide (2), community (2), however (2), government (2), support (2), population (2), against (2), symbol (2), resistance (2), relationship (2), national (2), tourism (2), creating (2), overall (2), saw (2), southern (2), philippe (2), berwick (2), fighting (2), everywhere (2), else (2), besiege (2), rather (2), austrian (2), karl (2), mind (2), clearly (2), earlier (2), familiar (2), minute (2), went (2), tolédo (2), experience (2), display (2), castilian (2), franco (2), thirty (2), keeping (2), track (2), gateway (2), model (2), nationalism (2), away (2), golden (2), video (2), flag (2), prehistory (2), undoubtedly (2), life (2), sticks (2), moderns (2), section (2), john (2), mention (2), guessing (2), speculation (2), historiographical (2), experienced (2), virtual (2), gives (2), moving (2), down (2), impression (2), bread (2), infinitely (2), league (2), looking (2), culture (2), budget (2), europe (2), meantime (2), query (2), listed (2), wikidata (2), curious (2), skills (2), scientist (2), gotta (2), granular (2), https (2), sheets (2), speed (2), times (2), faithful (2), usable (2), dataset (2), spreadsheets (2), warfare (2), downloaded (2), period (2), consider (2), bonus (2), internet (2), goes (2), overview (2), continue (2), yarn (2), surprising (2), allowing (2), oyster (2), extracting (2), depending (2), deep (2), aha (2), keywording (2), cycle (2), parsed (2), hundreds (2), metadata (2), visualization (2), modules (2), become (2), nested (2), items (2), strip (2), strings (2), associated (2), call (2), object (2), single (2), empty (2), seem (2), error (2), programmer (2), focus (2), original (2), easier (2), bbedit (2), formatting (2), seems (2), stage (2), lucky (2), scanned (2), missing (2), changing (2), spelling (2), contractions (2), spaces (2), needed (2), specifically (2), extracted (2), layout (2), towards (2), tabs (2), provided (2), argue (2), subject (2), piece (2), discrete (2), keyword (2), diary (2), drawn (2), being (2), converting (2), explain (2), lack (2), got (2), extract (2), limited (2), focusing (2), offer (2), possible (2), secondary (2), frequency (2), coordinates (2), according (2), criteria (2), visualizations (2), duration (2), similarly (2), groups (2), itself (2), further (2), talk (2), class (2), students (2), explore (2), method (2), pattern (2), parameter (2), input (2), practically (2), package (2), desired (2), field (2), topics (2), looked (2), algorithms (2), schedule (2), collecting (2), colleague (2), student (2), requests (2), interfaces (2), played (2), though (2), classification (2), sound (2), discuss (2), identify (2), context (2), calculate (2), 1700 (2), insert (2), update (2), records (2), mass (2), fields (2), relationships (2), nodes (2), topological (2), properties (2), international (2), visual (2), bokeh (2), spreadsheet (2), pandas (2), plot (2), charts (2), readable (2), requires (2), hyphenated (2), mistake (2), capitalize (2), uses (2), entity (2), corrections (2), hundred (2), ago (2), high (2), originals (2), amount (2), punctuation (2), dealing (2), ideas (2), previous (2), request (2), api (2), brute (2), apis (2), application (2), downloading (2), linked (2), copying (2), paste (2), pay (2), indexed (2), adobe (2), indexing (2), import (2), importance (2), literally (2), giant (2), kinds (2), digitally (2), fantasy (2), custom (2), usually (2), scenes (2), beyond (2), general (2), specialized (2), domain (2), tutorials (2), plenty (2), thanks (2), certain (2), pass (2), block (2), principles (2), attention (2), matter (2), quote (2), perfect (2), algorithm (2), expertise (2), emphasize (2), purchase (2), environments (2), automated (2), control (2), larger (2), upon (2), increase (2), inputs (2), manipulations (2), wish (2), summary (2), nothing (2), lets (2), layer (2), browser (2), variety (2), asking (2), matrix (2), 18th (2), share (2), myself (2), learned (2), geographical (2), summer (2), discoveries (2), installed (2), transcribed (2), england (2), volume (2), countries (2), genteelly (2), observing (2), enemy (2), design, loading, collapse, bar, manage, subscriptions, reader, report, privacy, join, 144, subscribers, quatorze, pike, oderint, dum, probent, perspectives, turenne, anno, domini, 1672, blogroll, wss, weapons, politics, veterans, travel, tactics, religion, recruitment, publishing, prussia, professionalism, pow, ottomans, netherlands, navy, enlightenment, mercenaries, medical, xiv, logistics, laws, italy, italian, ireland, intelligence, gtd, captains, gis, fwr, firearms, finance, engineers, emconventions, crusades, cavalry, caturday, casualties, britain, bcw, austria, artillery, sizes, armor, architecture, 30yw, 9yw, 7yw, uncategorized, historiography, select, category, wayne, lee, pradana, friends, propose, contact, seeking, forum, deadline, panel, paper, putting, alma, mater, ohio, state, columbus, theme, soldiers, civilians, cauldron, although, papers, germane, annual, reminder, caught, hosted, 1992, cue, archer, holdover, team, butts, dry, montjuic, locations, measurements, establish, meter, 1792, plaque, slightly, depressing, facts, occupying, stone, garrison, intimidate, town, coincidence, castilians, broke, bombard, restless, barcelonans, civil, became, violations, imprisonment, execution, political, prisoners, ends, quotation, universal, declaration, article, freedom, seek, involved, 1697, moat, monsters, guarding, ravelin, nope, guérite, smell, urine, photos, talking, zoom, descriptions, underwent, street, splay, embrasures, outward, funnel, themselves, flower, beds, aerial, bearings, complex, screenshot, surrounded, cable, car, funicular, walk, interest, sites, pronounced, mont, jew, initial, jewish, burial, ground, outcropping, mediterranean, transformed, lighthouse, successive, formidable, ethnographic, autonomous, northeast, known, implicitly, recognised, 1978, constitution, disagreements, tax, situation, hostility, independence, soared, polarising, outcome, remains, uncertain, previously, ruling, party, rebranded, non, binding, symbolic, pressure, politically, sensitive, significance, strained, johannes, venetia, identity, observations, edited, catherine, palmer, jacqueline, tivers, routledge, abstract, descriptive, noticed, witness, hot, press, study, catalonians, ours, explanation, oppressive, neighbor, defense, peeved, ordered, commander, slaughter, rebellion, capitulation, 11th, privileges, revoked, downhill, digress, surrender, meant, french, loaned, exposing, third, decade, fourth, brief, unsuccessful, attack, bent, eluded, large, exhibit, catastrophe, 1714, largely, 1713, crazy, catalans, kept, candidate, carlos, iii, abandoned, imperial, throne, 1711, becoming, holy, roman, emperor, appears, former, digs, remain, refused, acknowledge, felipe, stayed, death, witnessed, viewing, crypt, vienna, tomb, celebrates, 1706, liberation, kapuzinergruft, iberian, western, front, portuguese, border, animated, holdings, iberia, narrative, forces, managed, repulse, occupations, recapture, territory, battlefield, victory, almansa, obligingly, facilitated, reconquest, choosing, abandon, allies, contingent, captured, brihuega, museo, del, ejército, museu, història, catalunya, relevant, irregular, miquelets, rose, taxes, governance, olivares, home, songs, conflict, anthem, els, segadors, reaper, languedoc, visited, excursion, pyrenean, trouble, pirates, fun, came, origin, legend, senyera, 897, grateful, dipped, fingers, mortally, wounded, blood, dragged, shield, thanking, requested, killers, reward, differently, apparently, dating, 13c, prefer, stories, pen, sword, parchment, quill, ink, indeed, austrians, tale, crimson, streaked, half, waterfront, randomly, selected, destination, covers, region, hear, present, mannequin, cavemen, jump, 19th, rooms, kingdoms, principalities, shake, proverbial, stick, catalonian, speaking, struck, predominance, anglo, bookstores, geoffrey, parker, lynch, elliot, keegan, translations, histories, heavy, seeming, silence, senior, native, imperialism, reality, smartphone, headphone, fated, modernist, casa, bottló, photo, vertigo, anyway, attractions, mandatory, tourist, dollars, visiting, gaudí, standing, narrow, tower, waiting, wanting, contemplate, sagrada, familia, cathedral, tourists, must, proximity, bake, bâtards, wasn, rioting, spanked, liverpool, champions, survived, ugly, american, brooklynite, incident, train, arrived, northern, capital, seemed, sort, disagreement, parts, country, clue, proudly, hung, apartment, balconies, outnumbering, spying, vestiges, conquistadors, visits, castile, second, trip, side, conquerors, conquered, decide, eastern, belligerent, locating, efficiency, oriented, mindset, wherein, collects, ancient, explores, durations, attributes, cool, stuff, accurate, www, gokhan, collate, chart, heart, desires, wants, bureaucracy, inject, entered, factual, approximation, refine, beta, complete, defined, preparing, appeal, skulkers, assist, quixotic, quest, robust, envision, posting, aspects, combats, queries, lights, consist, phrase, describe, alludes, backward, move, sweater, shaved, sweaters, colorful, analogy, reilly, minor, tweaks, collections, newhailes, deane, takes, sucker, oysters, beauty, camps, filename, nest, dictionaries, layers, discussed, basics, 8th, precisely, target, hits, straddling, divide, pair, congratulations, ish, return, pickle, binary, passes, massage, values, efficient, fewer, lines, beginner, dict, comprehension, chained, encoding, splitting, hurdles, neophyte, discovered, pieces, fit, nutshell, within, interact, mileage, vary, 61404, muddled, curly, brackets, odd, equal, loss, care, robustly, relying, kindness, beggars, choosers, happen, reproduction, partial, tells, expanding, bother, rid, extra, stripped, standardized, identified, consists, carriage, distinctive, colons, ultimately, organized, per, appropriate, minutes, aside, examining, swearing, unstructured, consistent, scheme, judicious, unique, proof, concept, kindly, lawrence, smith, extent, mimicking, fidelity, 20th, maastricht, london, confusing, seen, explicitly, reinforces, best, passwords, solution, hone, tagging, transcriptions, requiring, wading, dross, likelihood, distracted, appear, filter, tested, physically, components, organize, old, quite, notecards, documented, journey, modified, applescripts, rudimentary, explode, rescue, tag, archival, begin, isolated, silos, timespan, needles, haystack, unparseable, automates, took, assistance, progress, show, preliminary, sweet, juicy, tantalizingly, reach, impediments, playing, pdfs, meaningful, systematic, slowly, catching, thus, pockets, multi, collaborators, position, sights, lower, taste, fruits, acquired, five, writing, diving, suggest, scan, decent, notepad, wrangler, killer, app, reads, statistics, identifies, rare, unusually, extracts, nationality, age, mentions, tables, graphs, locate, theaters, analyzed, runs, includes, comparisons, assigned, screen, chaining, objects, logic, copacetic, classes, voyant, among, heat, settings, parameters, performed, unable, faithfully, grant, agencies, recipients, submit, replicability, intriguing, rerun, exploration, super, anywhere, turn, extensible, related, automating, ask, drudgery, counting, sorting, revising, whom, hire, gets, burgeoning, classifying, superseded, neural, nets, specified, seeing, syllabus, meeting, removing, holidays, chair, reports, enrollment, assessment, surveys, scheduler, faculty, timeslots, university, departmental, scheduling, requirements, meets, administrative, school, batch, flexibility, mac, arcgis, boring, tasks, fledged, scriptable, portraits, facebook, reconstructing, soundscapes, genealogical, royal, genealogy, 16m, subsets, hathitrust, publications, periods, comparing, cited, affiliations, sciences, publish, ties, citation, networks, bibliometrics, biggie, extraction, cluster, segments, tend, collocated, keywords, sentiment, fuzzy, spelled, prose, phrasings, grammatical, structures, overuse, nltk, spacy, textacy, gensim, embeddings, calendars, tuesday, written, quick, calculation, calendar, datetime, dateutil, convertdate, dateparser, arrow, direction, contents, enter, pyzotero, interactions, sqlite, mysql, sqlite3, believe, triplets, verb, diagram, edges, measure, hubs, spokes, networkx, multiples, projection, whichever, aka, geocode, spatial, teams, gazetteers, georeferenced, geopandas, cartopy, mapping, explored, timeline, load, matplotlib, seaborn, humans, computers, detailed, description, ryan, cordell, hands, late, occurrence, outwork, misocred, mariborough, due, endings, changed, audit, difficult, capitalizations, liked, capitalization, clues, identifying, recognition, ideally, needs, special, handling, critical, bottleneck, idiosyncratic, historically, variant, usage, widely, varying, genre, vocabulary, sometimes, derived, ocring, irregularly, accuracy, rates, 100, ideal, quality, scans, average, scholar, acquire, ecco, paid, cheap, foreign, labor, theirs, social, scientists, born, everything, lowercase, stripping, imperfectly, retain, atomize, tokens, suggestions, welcome, useful, venues, 000s, skill, ted, underwood, jtb, raven, seriously, illustrate
Text of the page (random words):
u to for and if loops in case you didn t notice as long as the above two assumptions are met nice to meet you boolean value true you re golden imagine that you were asked what exactly does microsoft excel do instrumentally you d probably reply that it allows you to type in numbers edit the numbers with features like copy and paste format the numbers in a variety of ways perform mathematical calculations on the numbers and print out or email the results or you could answer the question by giving specific examples of projects you could do excel allows me to balance my budget it lets me keep track of student grades or my fantasy league stats good answers all but just the tip of the iceberg you could create an entire excel spreadsheet that doesn t have a single number in it you could use excel to lay out a particularly complicated table that word s table feature chokes on or that word generally makes a pain in the arse to edit you could like i did twenty years ago use excel as a go between when transferring word notes into a note taking access database see that python post does have some utility after all you could even use excel to rename a whole folder full of files based off another list e g rename all those eebo tcp files you downloaded which have names like a10352 txt in a sense that literally has almost nothing to do with spreadsheets you delete the excel file when you re done because it was just a means to an end in other words what exactly excel does is limited most broadly by the features built into the application but more practically it depends on the vision and technical expertise of the person using it same with a computer programming language but the python canvas starts out with nothing at all on it not even a grid of empty cells intimidating but its tabula rasa lets python deal with all sorts of different types of information and allows you to manipulate them in an almost infinite number of ways because you can keep building layer upon layer of functions on top of each other drawing in data and inputs from all sorts of places and sending them wherever you want so if you can think of a specific series of commands you invoke using some packaged software like word or excel or your web browser you can probably recreate that workflow in python but it s not just about replication i see three main advantages to coding over a packaged piece of software coding allows you to totally control the inputs the manipulations and the outputs don t like the way a program formats the results change it wish the summary feature would also include your favorite statistic include it hate how the program always does this one annoying thing fix it performing the task will be automated with python so you could run it thousands of times in a few seconds or more depending on the complexity of the task without getting carpal tunnel syndrome some packaged programs allow you to install or use macros to record these repeated actions and programming is like that but so much more because you have so much more control and a much larger toolbox to draw upon the result is a quantitative increase in speed but that quantitative increase is really a qualitative advance you can do things that you d never do otherwise because the computer does most of it for you you don t need to purchase learn and maintain the dozen different programs that would be needed to to do most of the data analysis you can perform with python nor do you need to worry about your specialized programs disappearing into history as happens shockingly often unless your name is microsoft or apple or adobe nor do you need to worry about radical changes from version to version if you ve experienced several generations of microsoft office or blackboard or heaven help you mobile apps you know what i mean python will definitely require you to keep an eye on changing python library versions and possible incompatibilities that might result but at least with python you can create virtual environments so that you can run different versions of the same libraries on the same machine and there are even web environments that allow you to set all of that up for the user python in other words allows you to create workflows that automate a lot of the processing and analysis steps you d have to do manually possibly in many different programs input data manipulate it in any number of ways and then output it but it definitely comes with a learning curve i won t lie a learning curve that is currently a lot harder for humanists in my humble opinion because we are not the primary users and therefore we don t have many pre made history specific libraries with functions already designed as a result i ve spent most of my time learning the fundamentals the eye candy of maps and charts is much better served by dedicated libraries that require less programming knowledge and more domain expertise my experience over the past several months has reinforced three important programming principles that i m sure programming 101 courses emphasize think through the nature of your data at each point in your code think through each logical step in your algorithm pay constant attention to the numerous syntax rules ignoring any of the above will lead to runtime errors fortunately you can google those error messages and probably figure out the problem since it will likely be a violation of one of the above principles are you trying to use string methods on a list item are you expecting your iterator to be an iterable do you really want to put the print function inside that loop are you blindly copying an online example without considering how your case is different and as i keep telling my students attention to details matter whether it s a missing close quote or a mismatched variable name but the computer will usually tell you if you make a mistake you learn these things over time and build up working code that you can copy or use as functions and practice makes perfect what exactly does python do for scholars for scholars a programming language like python can help manipulate and analyze whatever text numbers image video and sound information we have in digital form fortunately it s not as minimalist as all that thanks to a few thousand free libraries or modules that you import into the basic python package the libraries do most of the hard work behind the scenes i e most of the coding has already been done by the libraries authors so you just plug and play with some standard commands your specific data the specific variables you create and the specific commands combined in the specific order you choose figure out how to get your information into the code i e the computer s memory in a specific format based on its structure is it a string a list a dictionary manipulate the resulting data maybe you convert the string to a list after replacing certain features and adding others pass it on to the next block of code that does something else now that it s a list you can loop through each item and count those that start with the letter q do more things to the list and the count maybe if the count exceeds a certain threshold you send that word to another list then pass any of those results on to the next block of code until you end up with what you want to give a simple real world example maybe you ve started with a long string of text like this paragraph here you tokenize it into a list of individual words deciding how you want to deal with punctuation and contractions then count the words according to the letter they start with then plot a histogram showing the frequency of each letter python is also attractive to scholars because it s free insert joke about poor professor here its costlessness and open source ethos have encouraged hundreds of people to create free general libraries focusing on particular types of data and particular types of analysis along with specialized domain libraries for astronomers for geographers for audiologists for linguists for stock market analysts there is also a massive number of tutorials available online every year there are a dozen py conferences held all over the world and several hundred of the presentations are available on youtube including numerous 3 hour tutorials for beginners you can also check out the programming historian website which has numerous examples in python there are numerous cautions with programming in our case use python 3 not 2 7 but there are lots of resources that discuss those plenty of ways to get started in other words a final benefit of particular importance for humanities types is python s ability to convert words into numbers usually behind the scenes and highlight patterns using various statistical properties of text such powerful text functions allow businesses to data mine tweets and online content business demand seems to have juiced computer science research leading to lots of advanced natural language processing nlp features on top of those driven by the older linguistic and literary interests of academics so if you have a lot of digitized text or images and you want to clean analyze them beyond just reading each document one by one or manually cycling through search results one hit at a time then python is worth a look what exactly does python do for this historian so here s a list of the python projects i ve been working on and those i will be working on in the future a few are completed a few have draft code a few have some ideas sketched out with snippets of code and a couple are still in the fantasy phase many use the standard functions of off the shelf libraries while others require a bit more custom coding but they all should be viable projects time will tell semi automate a book index find all okay maybe most of the proper nouns in a pdf document along with which pdf page each occurred on then combine them together into a back of the book index format if you don t want to pay 1000 to have your book professionally indexed you could use word s or adobe s indexing feature which requires you to go through every sentence and identify which terms will need to be indexed or you can get 85 of that with python s nlp natural language processing libraries or you can import in a list of people places events and it will find those for you as with all programs things will get complicated the more edge and corner cases you try to address do i need to include a he on the next page with the full name on the previous page do i just combine together all consecutive pages into 34 39 or do i need to judge the importance of the headword to each page s discussion tough questions but this code will at the least give you a basis from which to tweak and judging from recent indexes in books published by highly reputable academic presses nobody cares about the art of indexing anymore some are literally headwords with a giant undifferentiated list of several dozen page numbers separated only by commas some don t even provide any kinds of topics only proper nouns of course the index may well die as more works are consumed digitally web scraping automate downloading content from a website either the text on pages entries in a list images of battle paintings from wikipedia or linked files maybe automate the sparql query on historical battles i posted about awhile back or download a bunch of letters from a site that puts each letter on its own separate page maybe automate scraping publication abstracts from a website based off records in zotero with a library like beautifulsoup web form entry i d like to create code that would automate copying bib info from zotero author title date pages etc and then paste it into our library s online ill form in the respective webform fields which of course aren t in the same order as the zotero field order that means a bunch of cutting and pasting for every request look up associated information on an entity person place organization with linked open data e g find the date of birth for person x mentioned in your text document via the web rdflib added by request get it apis numerous institutional websites allow you to access their online data more directly through api rather than using brute force scraping to harvest information from their pages html those websites have apis application program interfaces to allow more sophisticated downloading of information you can search their site for api to see if they offer it requests convert information into data my two previous posts on aha department enrollments and parsing long notes illustrate how this can be done with python code that converts information into data is particularly important low hanging fruit for historians since we lack a lot of already digitized datasets this kind of code allows us to create them with our own data clean dirty ocr text correct ocr errors and generally make a document more readable by humans and computers a good detailed description is ryan cordell s q i jtb the raven taking dirty ocr seriously this requires a lot of hands on work with the code which i ve been doing of late e g find every occurrence of out work and convert it to outwork so we can count them all the same way find every misocred mariborough and convert it to marlborough there are big lists of common errors available to make this search and edit process a bit more precise but since you ll never guess every word that might be hyphenated due to line endings finding every hyphenated word like be siege and converting it back to besiege is easy enough with regular expressions you can even create a list of all the changes your code makes i e create a dictionary with each mistake and what it was changed to if you want to audit the process more difficult are the questions of capitalizations case especially when our 17 18c authors liked to capitalize lots of nouns yet modern nlp uses capitalization as one of its clues for identifying proper nouns named entity recognition ideally you d have this code as a series of functions so that you can run the various corrections across an entire folder of documents based on what that source needs then you could use more code to check for any other errors or documents that require special handling i d argue that this is currently the most critical area for digital history the bottleneck in fact so few historians have their own sources in clean full text yet it s also a very idiosyncratic thing to program based on historically variant word usage and widely varying source genre vocabulary as well as the sometimes random errors derived from ocring irregularly set type from a few hundred years ago it d be great if ocr accuracy rates were 100 but that ideal would seem to require having high quality scans of the originals which your average scholar does not have and will likely never acquire because we don t actually own the originals as a result cleaning historical ocred text is the area with the least amount of pre made code available big projects like eebo and ecco pa...
|
