Meta tags:

Headings (most frequently used words):

cache, language, model, contents, see, also, references, further, reading,

Text of the page (most frequently used words):
the (60), #language (40), cache (25), model (23), for (13), and (12), speech (12), recognition (11), with (9), from (9), statistical (9), word (9), that (9), elephant (9), machine (8), has (8), words (8), wikipedia (7), processing (7), natural (7), edit (7), models (7), which (7), probability (7), this (6), translation (6), modeling (6), pdf (6), will (6), text (5), page (5), 2011 (5), systems (5), gram (5), been (5), one (5), sequences (5), system (5), spoken (5), contents (4), search (4), use (4), was (4), adaptation (4), neural (4), history (4), component (4), assigned (4), are (4), time (4), hide (4), move (4), sidebar (4), view (3), about (3), may (3), computational (3), linguistics (3), retrieved (3), archived (3), original (3), 2018 (3), interspeech (3), doi (3), domain (3), proceedings (3), information (3), 2010 (3), also (3), concept (3), recent (3), certain (3), other (3), very (3), not (3), citation (3), needed (3), first (3), have (3), occurred (3), higher (3), probabilities (3), letter (3), likely (3), occur (3), tools (3), main (3), topic (2), toggle (2), table (2), contact (2), privacy (2), policy (2), terms (2), using (2), non (2), categories (2), articles (2), unsourced (2), statements (2), isbn (2), jelinek (2), further (2), reading (2), s2cid (2), feed (2), forward (2), california (2), association (2), conference (2), workshop (2), context (2), 2001 (2), research (2), version (2), june (2), 1990 (2), pattern (2), intelligence (2), references (2), see (2), recently (2), paradigm (2), contexts (2), perplexity (2), another (2), when (2), discussing (2), small (2), interest (2), general (2), particular (2), there (2), group (2), yielded (2), once (2), few (2), dictated (2), techniques (2), standard (2), caching (2), but (2), contain (2), dictating (2), assign (2), each (2), recognized (2), tell (2), these (2), manually (2), occurrence (2), more (2), correctly (2), english (2), given (2), means (2), appearance (2), upload (2), file (2), changes (2), links (2), read (2), article (2), log (2), create (2), account (2), donate (2), menu (2), add, mobile, cookie, statement, statistics, developers, code, conduct, legal, safety, contacts, disclaimers, available, under, additional, apply, site, you, agree, registered, trademark, profit, organization, wikimedia, foundation, inc, creative, commons, attribution, sharealike, license, rendered, parsoid, last, edited, march, 2024, utc, hidden, september, all, https, org, index, php, title, cache_language_model, oldid, 1214930617, 1997, 262, 10066, mit, press, methods, frederick, karel, benes, santosh, kesiraju, lukas, burget, hyderabad, india, 3387, 52192034, 21437, 1070, 3383, vectors, efficient, way, edouard, grave, moustapha, cisse, armand, joulin, 2017, long, beach, computing, machinery, 6054, 978, 5108, 6096, 6044, nips, 31st, international, unbounded, online, open, vocabulary, tiedemann, jorg, acl, uppsala, sweden, exponentially, decaying, joshua, goodman, redmond, microsoft, technical, report, msr, 2001cs, 8005g, bibcode, 0108005v1, arxiv, bit, progress, extended, held, pacific, grove, usa, february, 1999, merialdo, roukos, strauss, 1991, 295, 2006, 11601499, 3115, 112405, 112464, 293, journal, acoustical, society, america, dynamic, abstract, kuhn, 583, 1939, 3539, issn, 1109, 56193, 570, ieee, transactions, analysis, based, mori, artificial, originally, conceived, adapted, instance, work, continuous, rnn, setting, applied, much, larger, than, before, yielding, significant, reductions, line, involves, incorporating, achieve, rapid, recurrent, network, success, improving, rests, human, tendency, bursty, fashion, frequency, uses, quite, different, their, frequencies, topics, traditional, rely, entirely, number, four, three, two, preceding, adequately, burstiness, prediction, development, generated, considerable, among, those, concerned, applying, field, proposed, paper, published, after, experimented, found, implementation, form, drop, hundred, document, had, detailed, survey, concluded, new, improvements, over, approach, our, results, show, far, most, useful, technique, reduction, medium, sizes, training, data, error, rates, ibm, exist, variants, only, single, multi, previously, san, francisco, near, beginning, subsequent, instances, would, understand, why, good, idea, might, consider, someone, who, elephants, low, because, rare, does, person, annoyed, sequence, according, plan, erroneous, deleted, replaced, still, probably, misrecognized, entered, into, however, point, aware, again, estimated, increased, making, several, times, recognize, every, until, completely, increase, example, consequence, specifically, learning, type, subfield, key, components, many, they, such, possible, output, probable, improbable, characteristic, contains, assigns, relatively, high, elsewhere, primary, sole, distribution, computer, science, free, encyclopedia, wikidata, item, projects, printable, download, print, export, switch, legacy, parser, get, shortened, url, cite, permanent, link, related, what, here, actions, talk, català, top, personal, special, pages, community, portal, learn, help, contribute, random, current, events, navigation, jump, content,

Text of the page (random words):
cache language model wikipedia jump to content main menu main menu move to sidebar hide navigation main page contents current events random article about wikipedia contact us contribute help learn to edit community portal recent changes upload file special pages search search appearance donate create account log in personal tools donate create account log in contents move to sidebar hide top 1 see also 2 references 3 further reading toggle the table of contents cache language model 1 language català edit links article talk english read edit view history tools tools move to sidebar hide actions read edit view history general what links here related changes upload file permanent link page information cite this page get shortened url switch to legacy parser print export download as pdf printable version in other projects wikidata item appearance move to sidebar hide from wikipedia the free encyclopedia a cache language model is a type of statistical language model these occur in the natural language processing subfield of computer science and assign probabilities to given sequences of words by means of a probability distribution statistical language models are key components of speech recognition systems and of many machine translation systems they tell such systems which possible output word sequences are probable and which are improbable the particular characteristic of a cache language model is that it contains a cache component and assigns relatively high probabilities to words or word sequences that occur elsewhere in a given text the primary but by no means sole use of cache language models is in speech recognition systems citation needed to understand why it is a good idea for a statistical language model to contain a cache component one might consider someone who is dictating a letter about elephants to a speech recognition system standard non cache n gram language models will assign a very low probability to the word elephant because it is a very rare word in english if the speech recognition system does not contain a cache component the person dictating the letter may be annoyed each time the word elephant is spoken another sequence of words with a higher probability according to the n gram language model may be recognized e g tell a plan these erroneous sequences will have to be deleted manually and replaced in the text by elephant each time elephant is spoken if the system has a cache language model elephant will still probably be misrecognized the first time it is spoken and will have to be entered into the text manually however from this point on the system is aware that elephant is likely to occur again the estimated probability of occurrence of elephant has been increased making it more likely that if it is spoken it will be recognized correctly once elephant has occurred several times the system is likely to recognize it correctly every time it is spoken until the letter has been completely dictated this increase in the probability assigned to the occurrence of elephant is an example of a consequence of machine learning and more specifically of pattern recognition there exist variants of the cache language model in which not only single words but also multi word sequences that have occurred previously are assigned higher probabilities e g if san francisco occurred near the beginning of the text subsequent instances of it would be assigned a higher probability citation needed the cache language model was first proposed in a paper published in 1990 1 after which the ibm speech recognition group experimented with the concept the group found that implementation of a form of cache language model yielded a 24 drop in word error rates once the first few hundred words of a document had been dictated 2 a detailed survey of language modeling techniques concluded that the cache language model was one of the few new language modeling techniques that yielded improvements over the standard n gram approach our caching results show that caching is by far the most useful technique for perplexity reduction at small and medium training data sizes 3 the development of the cache language model has generated considerable interest among those concerned with computational linguistics in general and statistical natural language processing in particular recently there has been interest in applying the cache language model in the field of statistical machine translation 4 the success of the cache language model in improving word prediction rests on the human tendency to use words in a bursty fashion when one is discussing a certain topic in a certain context the frequency with which one uses certain words will be quite different from their frequencies when one is discussing other topics in other contexts the traditional n gram language models which rely entirely on information from a very small number four three or two of words preceding the word to which a probability is to be assigned do not adequately model this burstiness citation needed recently the cache language model concept originally conceived for the n gram statistical language model paradigm has been adapted for use in the neural paradigm for instance recent work on continuous cache language models in the recurrent neural network rnn setting has applied the cache concept to much larger contexts than before yielding significant reductions in perplexity 5 another recent line of research involves incorporating a cache component in a feed forward neural language model fn lm to achieve rapid domain adaptation 6 see also edit artificial intelligence history of natural language processing history of machine translation speech recognition statistical machine translation references edit kuhn r de mori r june 1990 a cache based natural language model for speech recognition pdf ieee transactions on pattern analysis and machine intelligence 12 6 570 583 doi 10 1109 34 56193 issn 1939 3539 archived from the original pdf on 2011 09 01 retrieved 2011 09 24 abstract f jelinek b merialdo s roukos m strauss 1991 a dynamic language model for speech recognition pdf the journal of the acoustical society of america 98 2 293 295 doi 10 3115 112405 112464 s2cid 11601499 archived from the original pdf on june 14 2006 conference speech and natural language proceedings of a workshop held at pacific grove california usa february 19 22 1999 joshua t goodman 2001 a bit of progress in language modeling extended version redmond wa us microsoft research arxiv cs 0108005v1 bibcode 2001cs 8005g technical report msr tr 2001 72 tiedemann jorg 2010 context adaptation in statistical machine translation using models with exponentially decaying cache pdf proceedings of the 2010 workshop on domain adaptation for natural language processing acl 2010 uppsala sweden association for computational linguistics pp 8 15 edouard grave moustapha cisse armand joulin 2017 unbounded cache model for online language modeling with open vocabulary nips 17 proceedings of the 31st international conference on neural information processing systems long beach california association for computing machinery pp 6044 6054 isbn 978 1 5108 6096 4 karel benes santosh kesiraju lukas burget 2018 i vectors in language modeling an efficient way of domain adaptation for feed forward models interspeech 2018 hyderabad india interspeech pp 3383 3387 doi 10 21437 interspeech 2018 1070 s2cid 52192034 further reading edit jelinek frederick 1997 statistical methods for speech recognition the mit press isbn 0 262 10066 5 archived from the original on 2011 08 05 retrieved 2011 09 24 retrieved from https en wikipedia org w index php title cache_language_model oldid 1214930617 categories language modeling computational linguistics natural language processing speech processing speech recognition machine translation hidden categories all articles with unsourced statements articles with unsourced statements from september 2011 this page was last edited on 22 march 2024 at 02 33 utc page was rendered with parsoid text is available under the creative commons attribution sharealike 4 0 license additional terms may apply by using this site you agree to the terms of use and privacy policy wikipedia is a registered trademark of the wikimedia foundation inc a non profit organization privacy policy about wikipedia disclaimers contact wikipedia legal safety contacts code of conduct developers statistics cookie statement mobile view search search toggle the table of contents cache language model 1 language add topic