Meta tags:

Headings (most frequently used words):

cover, story, try, it, out, matisse, is, team, player, some, context, grammar, school,

Text of the page (most frequently used words):
the (26), phrase (19), and (19), noun (10), for (8), this (8), verb (7), was (7), that (6), book (6), can (6), #grammar (5), sentence (4), with (4), each (4), but (4), challenge (4), adjective (3), prepositional (3), from (3), into (3), which (3), use (3), dependency (3), etc (3), would (3), here (3), phrases (3), some (3), its (3), story (3), cover (3), adj (2), end (2), plural (2), type (2), click (2), title (2), try (2), were (2), couple (2), other (2), cases (2), why (2), using (2), not (2), active (2), their (2), plurality (2), used (2), instead (2), constituents (2), word (2), exactly (2), one (2), constituency (2), found (2), more (2), what (2), idea (2), could (2), sentences (2), very (2), language (2), productivity (2), even (2), form (2), many (2), also (2), these (2), behind (2), linguistics (2), books (2), munroe (2), part (2), tour (2), help (2), how (2), advice (2), matisse (2), want (2), judge (2), you (2), read (2), adjps, start, adjpe, coordinating, conjunction, vps, nps, choose, add, new, select, dropdown, insert, your, double, shuffle, right, delete, out, there, just, edge, handled, tags, line, entirely, penn, treebank, tagset, root, tree, parts, speech, include, markers, properness, tense, different, model, grammars, dependent, state, art, parser, fits, nicely, nltk, provided, decent, results, issue, ran, handling, granular, details, such, subject, alignment, those, wanted, know, benepar, main, label, then, compose, may, make, ton, sense, are, least, grammatical, way, construct, expressive, productive, english, means, limited, vocabulary, distinct, thoughts, because, have, chain, together, recursively, fact, linguists, believe, all, human, recursive, see, background, linguist, turf, wars, might, optional, adjectives, determiners, maybe, example, base, key, concept, project, something, learned, introduction, class, last, fall, trees, basic, represented, combination, words, school, list, over, 200, 000, listed, amazon, jupyter, notebooks, python, packages, later, off, races, decided, hack, acquiring, dataset, titles, having, computer, generate, stories, winner, rewarded, visit, though, ann, arbor, already, itinerary, couldn, think, about, knowledge, advantage, write, best, nothing, covers, announcement, introduced, interesting, mid, june, randall, genius, xkcd, author, quite, few, announced, going, promote, his, latest, absurd, scientific, common, real, world, problems, context, two, influential, artists, sure, collaborating, ancient, kabbalists, did, people, leonardo, shadow, team, player, thanks, friends, blablablab, don, linguistic, probably, desktop, mobile, less, cool, version, previously, published, blog,

Text of the page (random words):
cover story cover story don t judge a book by its cover judge it by its linguistic productivity instead you ll probably want to read this on desktop if on mobile you can read a less cool version of this previously published on my blog thanks also to my friends at the blablablab for their advice and help matisse is a team player two very influential artists to be sure but why was matisse collaborating with ancient kabbalists and what exactly did the people want from leonardo s shadow some context in mid june randall munroe the genius behind xkcd and the author of quite a few books announced he was going on a book tour to promote his latest book how to absurd scientific advice for common real world problems as part of the announcement he introduced an interesting challenge write the best story using nothing but book covers the winner of the challenge would be rewarded by a visit from munroe as part of the tour even though ann arbor was already in the itinerary i couldn t help but think about this challenge and how i could use some of my linguistics knowledge to my advantage i decided i would try to hack this challenge by acquiring a dataset of book titles and having a computer generate these stories for me i found a list of over 200 000 books listed on amazon and a couple of jupyter notebooks and python packages later we were off to the races grammar school the key concept behind this project was something i learned in my introduction to linguistics class last fall constituency trees the basic idea is that each sentence can be represented as a combination of words and phrases constituents for example a sentence is in its base form a noun phrase and a verb phrase and a noun phrase might be a noun with optional adjectives and determiners and maybe a prepositional phrase and because prepositional phrases can also have noun phrases we can chain these together recursively in fact many linguists believe that all human language is recursive see here and here for more background and some linguist turf wars and in this way we can construct a very expressive and productive grammar for the english language productivity here means that with even a limited vocabulary we can form many distinct sentences and thoughts so the main idea was we would label each book title as a noun phrase verb phrase adjective phrase etc then we could use this grammar to compose sentences which may not make a ton of sense but are at least grammatical i used benepar which is a state of the art constituency parser that fits nicely into nltk i found that it provided decent results one issue that i ran into was handling more granular details such as subject verb plurality alignment in those cases i wanted to know what the noun was in the noun phrase etc for this i used a different grammar model dependency grammar in dependency grammars instead constituents each word is dependent on exactly one other word in the sentence i use the root of each dependency tree as the active noun or active verb in the phrase and their parts of speech include markers for plurality properness verb tense etc there were just a couple of other edge cases i handled which is why the phrase tags i end up using do not line up entirely with the penn treebank tagset try it out select a phrase type from the dropdown to insert it into your sentence double click to shuffle the title right click to delete add a new choose phrase type np noun phrase nps plural noun phrase vp verb phrase vps plural verb phrase pp prepositional phrase cc coordinating conjunction adjpe adj phrase w adjective at end adjps adj phrase w adjective at start