Meta tags:
Headings (most frequently used words):
and, of, word, embedding, contents, development, history, the, approach, polysemy, homonymy, for, biological, sequences, biovectors, game, design, sentence, embeddings, software, ethical, implications, see, also, references, examples, application,
Text of the page (most frequently used words):
the (115), and (83), word (80), learning (51), #embeddings (47), for (44), language (37), neural (30), #embedding (28), semantic (24), sense (22), words (20), text (19), model (19), doi (19), processing (18), vector (18), computational (17), network (17), machine (17), models (17), analysis (17), are (17), conference (17), space (17), linguistics (16), that (16), natural (15), multi (15), arxiv (15), data (14), proceedings (14), vectors (14), edit (14), knowledge (13), representation (13), pdf (13), using (12), with (12), based (12), information (12), association (12), artificial (11), systems (11), bert (11), 2019 (11), from (10), computer (10), speech (10), s2cid (10), 2015 (10), been (10), sentence (9), can (9), used (9), wikipedia (8), applications (8), john (8), linguistic (8), 2018 (8), representations (8), sequences (8), research (8), dimensionality (8), which (8), approach (8), isbn (7), bengio (7), self (7), recognition (7), intelligence (7), deep (7), game (7), word2vec (7), distributional (7), methods (7), probabilistic (7), such (7), terms (6), this (6), networks (6), retrieved (6), term (6), vision (6), yoshua (6), supervised (6), method (6), history (6), sentiment (6), extraction (6), also (6), vol (6), biological (6), 2014 (6), meaning (6), 2000 (6), random (6), main (6), have (6), search (5), use (5), page (5), was (5), articles (5), references (5), short (5), james (5), image (5), human (5), general (5), context (5), training (5), latent (5), engineering (5), bias (5), regression (5), software (5), document (5), corpus (5), lexical (5), matrix (5), disambiguation (5), issn (5), empirical (5), reduction (5), 978 (5), international (5), skip (5), distributed (5), advances (5), new (5), feature (5), through (5), science (5), indexing (5), 2013 (5), their (5), these (5), has (5), each (5), club (5), topic (4), contents (4), about (4), policy (4), cs1 (4), long (4), different (4), wikidata (4), modeling (4), generative (4), transformer (4), david (4), andrew (4), cognitive (4), reasoning (4), list (4), large (4), generation (4), diffusion (4), automated (4), gradient (4), datasets (4), clustering (4), related (4), automatic (4), segmentation (4), google (4), english (4), glove (4), fasttext (4), explicit (4), example (4), mining (4), understanding (4), gram (4), society (4), chang (4), kai (4), wei (4), 2020 (4), 2016 (4), man (4), programmer (4), woman (4), homemaker (4), computing (4), october (4), emnlp (4), journal (4), one (4), multiple (4), mikolov (4), 2005 (4), linear (4), sahlgren (4), magnus (4), most (4), some (4), occurrence (4), trained (4), application (4), article (4), contexts (4), into (4), single (4), after (4), hide (4), move (4), sidebar (4), toggle (3), view (3), available (3), may (3), non (3), categories (3), visual (3), architecture (3), explainable (3), autoencoder (3), recurrent (3), daniel (3), alex (3), manning (3), rule (3), expert (3), synthesis (3), agent (3), symbolic (3), reinforcement (3), gensim (3), semantics (3), annotation (3), classification (3), viewer (3), corpora (3), translation (3), detection (3), part (3), parsing (3), named (3), 1007 (3), zhao (3), jieyu (3), 2017 (3), 18653 (3), like (3), gender (3), transactions (3), bolukbasi (3), adam (3), debiasing (3), archived (3), original (3), mohammad (3), pmid (3), elmo (3), richard (3), thought (3), rabii (3), cook (3), 2021 (3), via (3), gameplay (3), asgari (3), mofrad (3), north (3), american (3), chapter (3), papers (3), improve (3), usa (3), process (3), improving (3), 2010 (3), annual (3), tomas (3), nips (3), hierarchical (3), 290 (3), 2004 (3), 1145 (3), jean (3), neurips (3), pentti (3), kanerva (3), means (3), introduction (3), real (3), gerard (3), salton (3), cite (3), 1957 (3), firth (3), theory (3), pca (3), phrases (3), biases (3), proposed (3), way (3), create (3), results (3), not (3), design (3), biovectors (3), underlying (3), similar (3), nlp (3), unsupervised (3), number (3), other (3), homonymy (3), polysemy (3), work (3), high (3), field (3), tools (3), languages (2), table (2), statistics (2), code (2), safety (2), contact (2), privacy (2), organization (2), inc (2), last (2), august (2), hidden (2), volume (2), value (2), all (2), circular (2), lacking (2), reliable (2), 2024 (2), date (2), maint (2), publisher (2), location (2), description (2), impact (2), video (2), games (2), chatbot (2), fiction (2), engine (2), social (2), virtual (2), act (2), adversarial (2), gan (2), mamba (2), rnn (2), convolutional (2), perceptron (2), unit (2), gru (2), memory (2), lstm (2), turing (2), architectures (2), jan (2), ilya (2), sutskever (2), fei (2), geoffrey (2), hinton (2), joseph (2), christopher (2), alan (2), dbpedia (2), conceptnet (2), logic (2), action (2), ibm (2), world (2), alexnet (2), protocol (2), asi (2), intelligent (2), neuro (2), hallucination (2), recursive (2), rlhf (2), sarsa (2), prompt (2), descent (2), variance (2), tradeoff (2), parameter (2), open (2), projects (2), glossary (2), toolkit (2), formal (2), character (2), multimodal (2), user (2), interface (2), interactive (2), checker (2), grammar (2), assisted (2), allocation (2), identification (2), capture (2), wordnet (2), babelnet (2), treebank (2), retrieval (2), simple (2), system (2), transfer (2), statistical (2), summarization (2), induction (2), terminology (2), similarity (2), decomposition (2), labeling (2), tagging (2), syntactic (2), ontology (2), entity (2), concept (2), but (2), they (2), reflecting (2), mark (2), reducing (2), level (2), 1162 (2), spaces (2), tolga (2), zou (2), saligrama (2), venkatesh (2), kalai (2), 1607 (2), 06520 (2), umap (2), uniform (2), manifold (2), approximation (2), projection (2), pmc (2), clinical (2), indra (2), dan (2), how (2), multilingual (2), siamese (2), joint (2), 1506 (2), 194 (2), aaai (2), bibcode (2), proteomics (2), genomics (2), martin (2), technologies (2), jurafsky (2), stroudsburg (2), hdl (2), parametric (2), estimation (2), per (2), 2008 (2), morin (2), eds (2), workshop (2), roweis (2), saul (2), locally (2), 5500 (2), 2323 (2), lavelli (2), acm (2), 2002 (2), 540 (2), 137 (2), studies (2), ducharme (2), réjean (2), vincent (2), pascal (2), holst (2), anders (2), foundations (2), samples (2), paper (2), help (2), book (2), 1962 (2), associations (2), fall (2), university (2), link (2), socher (2), chris (2), compositionality (2), conf (2), levy (2), omer (2), goldberg (2), yoav (2), sparse (2), technique (2), ijcai (2), factorization (2), see (2), done (2), shows (2), introduced (2), even (2), out (2), popular (2), when (2), analogies (2), ethical (2), implications (2), online (2), examples (2), sne (2), reduce (2), idea (2), entire (2), sentences (2), documents (2), form (2), quality (2), more (2), recent (2), pre (2), structures (2), discover (2), actions (2), occur (2), then (2), resulting (2), presented (2), suggest (2), rules (2), grams (2), proteins (2), gene (2), widely (2), late (2), static (2), its (2), performance (2), several (2), tasks (2), approaches (2), two (2), mssg (2), time (2), senses (2), mssa (2), meanings (2), represented (2), than (2), broader (2), led (2), experimentation (2), practical (2), styles (2), expressed (2), published (2), techniques (2), algebraic (2), kernel (2), cca (2), items (2), between (2), series (2), development (2), curve (2), net (2), forest (2), factor (2), anomaly (2), bayes (2), structured (2), prediction (2), shot (2), appearance (2), upload (2), file (2), changes (2), links (2), read (2), log (2), account (2), donate (2), menu (2), add, mobile, cookie, statement, developers, conduct, legal, contacts, disclaimers, under, additional, apply, site, you, agree, registered, trademark, profit, wikimedia, foundation, creative, commons, attribution, sharealike, license, rendered, parsoid, edited, 2026, utc, containing, errors, relations, https, org, index, php, title, word_embedding, oldid, 1369691723, category, workplace, warfare, military, art, marketing, psychosis, healthcare, education, optimization, environmental, competition, arms, race, anthropomorphism, winter, slop, literacy, infrastructure, effect, center, bubble, boom, economic, propaganda, politician, regulation, precautionary, principle, nationalism, ethics, elections, takeover, alignment, government, cold, war, political, graph, gnn, variational, vae, highway, residual, cnn, multilayer, mlp, echo, state, gated, vit, differentiable, françois, chollet, kokotajlo, leike, mustafa, suleyman, schulman, aidan, gomez, noam, shazeer, ashish, vaswani, andrej, karpathy, silver, demis, hassabis, ian, goodfellow, quoc, oriol, vinyals, krizhevsky, goodnight, graves, stephen, grossberg, lotfi, zadeh, yann, lecun, jürgen, schmidhuber, hopfield, paul, werbos, seppo, linnainmaa, seymour, papert, weizenbaum, bernard, widrow, frank, rosenblatt, oliver, selfridge, herbert, simon, cliff, shaw, allen, newell, nathaniel, rochester, mccarthy, marvin, minsky, takeo, kanade, kunihiko, fukushima, shun, ichi, amari, claude, shannon, von, neumann, walter, pitts, warren, sturgis, mcculloch, people, yago, bases, opencog, lida, clarion, soar, reasoners, procedural, programs, inference, engines, deductive, classifiers, robot, control, autogpt, selection, muzero, driving, car, openai, five, alphazero, alphago, decisional, watsonx, watson, project, debater, oasis, genie, udio, suno, riffusion, music, veo, seedance, sora, kling, hailuo, runway, gen, dream, stable, recraft, midjourney, imagen, ideogram, gpt, flux, firefly, dall, aurora, alphafold, facial, whisper, elevenlabs, ocr, hwr, wavenet, audio, implementations, physical, agent2agent, hypothetical, superintelligence, agi, weak, lethal, autonomous, weapons, laws, humanity, exam, companion, nmt, playing, theorem, proving, vibe, coding, improvement, reflection, llm, post, uncanny, valley, rag, adversary, autoregression, imitation, augmentation, regularization, weight, initialization, gating, rectifier, sigmoid, softmax, activation, batchnorm, normalization, convolution, attention, backpropagation, conjugate, quasi, newton, sgd, overfitting, double, loss, functions, hyperparameter, constraint, satisfaction, planning, concepts, source, companies, timeline, spacy, benchmark, optical, clip, voice, assistant, question, answering, spell, pronunciation, assessment, predictive, concordancer, essay, scoring, reviewing, pachinko, dirichlet, dynamic, uby, ngram, framenet, bank, universal, dependencies, thesaurus, propbank, parallel, readable, dictionary, linked, resource, types, standards, resources, seq2seq, small, simplification, stemming, chunking, lemmatization, compound, truecasing, textual, entailment, stylometry, stance, role, distant, reading, coreference, resolution, collocation, argument, stop, trigram, bigram, bag, complete, petreski, davor, hashim, ibrahim, 2022, 982, 249112516, 1435, 5655, s00146, 022, 01443, 975, biased, whose, wang, tianlu, yatskar, ordonez, vicente, 2989, d17, 1323, 2979, men, shopping, amplification, constraints, dieng, adji, ruiz, francisco, blei, 453, tacl_a_00325, 1907, 04907, 439, february, feb, mcinnes, leland, healy, melville, dimension, stat, 1802, 03426, ghassemi, roger, nemati, shamim, 632, 27774487, 5070922, 5090, 0685, 1109, cic, 7410989, 629, cardiology, cinc, visualization, evolving, notes, github, pires, telmo, schlinger, eva, garrette, 1906, 01502, neutral, 1809, 01496, reimers, nils, iryna, gurevych, 9th, ijcnlp, 3982, 3992, kiros, ryan, zhu, yukun, salakhutdinov, ruslan, zemel, torralba, antonio, urtasun, raquel, fidler, sanja, 06726, younès, michael, 248175634, 2334, 0924, 1609, aiide, v17i1, 18907, 187, digital, entertainment, revealing, dynamics, ehsaneddin, e0141287, 26555596, 4640716, 1371, pone, 0141287, 2015ploso, 1041287a, 1503, 05140, plos, continuous, reif, emily, ann, yuan, wattenberg, fernanda, viegas, andy, coenen, pearce, kim, visualizing, measuring, geometry, lucy, bamman, characterizing, variation, across, media, communities, 538, 556, devlin, jacob, ming, lee, kenton, toutanova, kristina, june, 4186, 52967399, n19, 1423, 4171, jiwei, 1732, 6222768, d15, 1200, 01070, 1722, akbik, blythe, duncan, vollgraf, roland, santa, mexico, 1649, 1638, 27th, contextual, string, sequence, agre, gennady, petrov, keskinova, simona, 2078, 2489, 3390, info10030097, studio, flexible, wsd, ruas, terry, grosky, william, aizawa, akiko, 303, 52225306, 0957, 4174, 2027, 145475, 1016, eswa, 026, 2101, 08700, 288, 136, neelakantan, arvind, shankar, jeevan, passos, alexandre, mccallum, efficient, 1069, 15251438, 3115, d14, 1113, 1504, 06654, 1059, camacho, collados, jose, pilehvar, taher, survey, 1805, 04032, huang, eric, 2012, 857900050, oclc, global, prototypes, reisinger, mooney, raymond, los, angeles, california, 117, 932432, 109, prototype, bojanowski, piotr, grave, edouard, joulin, armand, 146, tacl_a_00051, 135, enriching, subword, july, archive, mnih, andriy, 2009, curran, associates, 1088, 1081, scalable, fredric, cowell, robert, ghahramani, zoubin, 252, 246, tenth, יהושע, בנג, sam, lawrence, nonlinear, 5987139, 11125150, 1126, 2000sci, 2323r, alberto, sebastiani, fabrizio, zanoli, roberto, 13th, management, 624, 1031171, 1031284, 615, experimental, comparison, vinkourov, alexei, cristianini, nello, shawe, taylor, inferring, cross, correlation, schwenk, holger, senécal, sébastien, fréderic, gauvain, luc, 2006, springer, 186, 30609, 33486, 6_6, fuzziness, soft, jauvin, christian, 2003, 1155, 1137, 30th, 1300, 1305, permutations, encode, order, 7th, tke, copenhagen, denmark, karlgren, jussi, 2001, uesaka, yoshinori, asoh, hideki, csli, publications, 308, 294, kristoferson, 22nd, 1036, mahwah, jersey, erlbaum, dubin, influential, never, wrote, wong, yang, 1975, 620, 6473756, 1813, 6057, 361219, 361220, 613, communications, incompatibility, experiments, 250, 9937095, 4503, 7879, 1461518, 1461544, 234, december, afips, osgood, suci, tannenbaum, illinois, press, measurement, luhn, 1953, recording, searching, 1002, 5090040104, documentation, reprinted, palmer, 1968, london, longman, selected, 1952, 1959, synopsis, 1930, 1955, brief, perelygin, chuang, jason, potts, over, bauer, proc, acl, compositional, grammars, conll, 180, 171, regularities, qureshi, atif, greene, derek, eve, 165, 10656055, 0925, 9902, s10844, 018, 0511, 1702, 06891, globerson, amir, 2007, euclidean, yitan, linli, int, revisited, perspective, implicit, lebret, rémi, collobert, ronan, emdeddings, hellinger, 1312, 5542, european, eacl, chen, corrado, greg, dean, jeffrey, 1310, 4546, upper, saddle, river, prentice, hall, 095069, relational, database, brown, without, careful, oversight, likely, perpetuates, existing, unaltered, furthermore, amplify, contain, stereotypes, contained, dataset, points, publicly, texts, commonly, consists, written, professional, journalists, still, disproportionate, racial, extracting, generated, aforementioned, news, instance, calculate, sketch, includes, stanford, flair, allennlp, visualize, clusters, stochastic, neighbour, principal, component, deeplearning4j, tomáš, extended, researchers, suggested, representing, sentencetransformers, modifies, triplet, logs, requires, transcribing, during, within, explicitly, stated, chess, emergent, bio, biovec, refer, protein, protvec, amino, acid, genevec, characterize, biochemical, biophysical, interpretations, patterns, bioinformatics, rna, dna, 2010s, contextually, meaningful, developed, unlike, token, own, better, reflect, nature, because, occurrences, situated, regions, known, relation, relatedness, produce, divided, performs, discrimination, simultaneously, while, assuming, specific, vary, depending, combining, prior, databases, suitable, labels, considering, defined, sliding, window, once, disambiguated, standard, produced, allows, performed, recurrently, manner, historically, limitations, conflated, handled, properly, tried, yesterday, great, clear, any, might, necessity, accommodate, motivation, contributions, split, ones, golf, clubhouse, sandwich, later, included, partly, rather, treating, indivisible, adopted, many, groups, theoretical, had, made, speed, well, hardware, allowed, explored, profitably, team, created, train, faster, previous, instrumental, raising, interest, technology, moving, strand, specialised, eventually, paving, come, occurring, another, studied, lle, dimensional, rely, instead, foundational, colleagues, reference, study, both, applying, bilingual, lingual, providing, early, notion, challenges, capturing, characteristics, them, measure, first, implemented, simplest, very, dimensions, 1980s, collecting, provided, titled, singular, curse, quantitative, methodological, observed, aim, quantify, categorize, similarities, properties, characterized, company, keeps, roots, contemporaneous, psychology, rupert, phrase, input, shown, boost, generate, mapping, include, base, appear, typically, encodes, closer, expected, obtained, where, vocabulary, mapped, numbers, valued, outline, jmlr, iclr, icml, iccv, ecml, pkdd, eccv, cvpr, journals, conferences, topological, pac, occam, risk, minimization, machines, mathematical, roc, confusion, coefficient, determination, diagnostics, mechanistic, interpretability, loop, crowdsourcing, active, humans, play, temporal, difference, ecram, electrochemical, ram, memtransistor, spiking, physics, informed, radiance, deepdream, lenet, som, restricted, boltzmann, reservoir, esn, feedforward, isolation, local, outlier, ransac, markov, conditional, graphical, sdl, pgd, nmf, lda, ica, exploratory, mean, shift, optics, dbscan, expectation, maximization, fuzzy, cure, birch, support, svm, relevance, rvm, logistic, naive, boosting, bagging, ensembles, decision, trees, apprenticeship, rank, automl, cleaning, density, problems, quantum, neuromorphic, curriculum, batch, zero, few, meta, semi, paradigms, illustration, point, enables, processor, perform, operations, obtaining, capital, given, country, free, encyclopedia, item, printable, version, download, print, export, switch, legacy, parser, get, shortened, url, permanent, what, here, talk, tiếng, việt, українська, ไทย, српски, srpski, русский, português, polski, norsk, bokmål, 한국어, 日本語, italiano, עברית, français, فارسی, euskara, español, deutsch, čeština, کوردی, català, العربية, subsection, top, personal, special, pages, community, portal, learn, contribute, current, events, navigation, jump, content,
Text of the page (random words):
tm gru esn reservoir computing boltzmann machine restricted gan diffusion model som convolutional neural network u net lenet alexnet deepdream neural field neural radiance field physics informed neural networks transformer vision mamba spiking neural network memtransistor electrochemical ram ecram reinforcement learning q learning policy gradient sarsa temporal difference td multi agent self play learning with humans active learning crowdsourcing human in the loop mechanistic interpretability rlhf model diagnostics coefficient of determination confusion matrix learning curve roc curve mathematical foundations kernel machines bias variance tradeoff computational learning theory empirical risk minimization occam learning pac learning statistical learning vc theory topological deep learning journals and conferences aaai cvpr eccv ecml pkdd emnlp iccv neurips icml iclr ijcai ml jmlr related articles glossary of artificial intelligence list of datasets for machine learning research list of datasets in computer vision and image processing outline of machine learning v t e in natural language processing a word embedding is a representation of a word the embedding is used in text analysis typically the representation is a real valued vector that encodes the meaning of the word in such a way that the words that are closer in the vector space are expected to be similar in meaning 1 word embeddings can be obtained using language modeling and feature learning techniques where words or phrases from the vocabulary are mapped to vectors of real numbers methods to generate this mapping include neural networks 2 dimensionality reduction on the word co occurrence matrix 3 4 5 probabilistic models 6 explainable knowledge base method 7 and explicit representation in terms of the context in which words appear 8 word and phrase embeddings when used as the underlying input representation have been shown to boost the performance in nlp tasks such as syntactic parsing 9 and sentiment analysis 10 development and history of the approach edit in distributional semantics a quantitative methodological approach for understanding meaning in observed language word embeddings or semantic feature space models have been used as a knowledge representation for some time 11 such models aim to quantify and categorize semantic similarities between linguistic items based on their distributional properties in large samples of language data the underlying idea that a word is characterized by the company it keeps was proposed in a 1957 article by john rupert firth 12 but also has roots in the contemporaneous work on search systems 13 and in cognitive psychology 14 the notion of a semantic space with lexical items words or multi word terms represented as vectors or embeddings is based on the computational challenges of capturing distributional characteristics and using them for practical application to measure similarity between words phrases or entire documents the first generation of semantic space models is the vector space model for information retrieval 15 16 17 such vector space models for words and their distributional data implemented in their simplest form results in a very sparse vector space of high dimensionality cf curse of dimensionality reducing the number of dimensions using linear algebraic methods such as singular value decomposition then led to the introduction of latent semantic analysis in the late 1980s and the random indexing approach for collecting word co occurrence contexts 18 19 20 21 in 2000 bengio et al provided in a series of papers titled neural probabilistic language models to reduce the high dimensionality of word representations in contexts by learning a distributed representation for words 22 23 24 a study published in neurips nips 2002 introduced the use of both word and document embeddings applying the method of kernel cca to bilingual and multi lingual corpora also providing an early example of self supervised learning of word embeddings 25 word embeddings come in two different styles one in which words are expressed as vectors of co occurring words and another in which words are expressed as vectors of linguistic contexts in which the words occur these different styles are studied in lavelli et al 2004 26 roweis and saul published in science how to use locally linear embedding lle to discover representations of high dimensional data structures 27 most new word embedding techniques after about 2005 rely on a neural network architecture instead of more probabilistic and algebraic models after foundational work done by yoshua bengio 28 circular reference and colleagues 29 30 the approach has been adopted by many research groups after theoretical advances in 2010 had been made on the quality of vectors and the training speed of the model as well as after hardware advances allowed for a broader parameter space to be explored profitably in 2013 a team at google led by tomas mikolov created word2vec a word embedding toolkit that can train vector space models faster than previous approaches the word2vec approach has been widely used in experimentation and was instrumental in raising interest for word embeddings as a technology moving the research strand out of specialised research into broader experimentation and eventually paving the way for practical application 31 later work on word embeddings included fasttext which represented words partly through character n grams rather than treating each word as a single indivisible unit 32 polysemy and homonymy edit historically one of the main limitations of static word embeddings or word vector space models is that words with multiple meanings are conflated into a single representation a single vector in the semantic space in other words polysemy and homonymy are not handled properly for example in the sentence the club i tried yesterday was great it is not clear if the term club is related to the word sense of a club sandwich clubhouse golf club or any other sense that club might have the necessity to accommodate multiple meanings per word in different vectors multi sense embeddings is the motivation for several contributions in nlp to split single sense embeddings into multi sense ones 33 34 most approaches that produce multi sense embeddings can be divided into two main categories for their word sense representation i e unsupervised and knowledge based 35 based on word2vec skip gram multi sense skip gram mssg 36 performs word sense discrimination and embedding simultaneously improving its training time while assuming a specific number of senses for each word in the non parametric multi sense skip gram np mssg this number can vary depending on each word combining the prior knowledge of lexical databases e g wordnet conceptnet babelnet word embeddings and word sense disambiguation most suitable sense annotation mssa 37 labels word senses through an unsupervised and knowledge based approach considering a word s context in a pre defined sliding window once the words are disambiguated they can be used in a standard word embeddings technique so multi sense embeddings are produced mssa architecture allows the disambiguation and annotation process to be performed recurrently in a self improving manner 38 the use of multi sense embeddings is known to improve performance in several nlp tasks such as part of speech tagging semantic relation identification semantic relatedness named entity recognition and sentiment analysis 39 40 as of the late 2010s contextually meaningful embeddings such as elmo and bert have been developed 41 unlike static word embeddings these embeddings are at the token level in that each occurrence of a word has its own embedding these embeddings better reflect the multi sense nature of words because occurrences of a word in similar contexts are situated in similar regions of bert s embedding space 42 43 for biological sequences biovectors edit word embeddings for n grams in biological sequences e g dna rna and proteins for bioinformatics applications have been proposed by asgari and mofrad 44 named bio vectors biovec to refer to biological sequences in general with protein vectors protvec for proteins amino acid sequences and gene vectors genevec for gene sequences this representation can be widely used in applications of deep learning in proteomics and genomics the results presented by asgari and mofrad 44 suggest that biovectors can characterize biological sequences in terms of biochemical and biophysical interpretations of the underlying patterns game design edit word embeddings with applications in game design have been proposed by rabii and cook 45 as a way to discover emergent gameplay using logs of gameplay data the process requires transcribing actions that occur during a game within a formal language and then using the resulting text to create word embeddings the results presented by rabii and cook 45 suggest that the resulting vectors can capture expert knowledge about games like chess that are not explicitly stated in the game s rules sentence embeddings edit main article sentence embedding the idea has been extended to embeddings of entire sentences or even documents e g in the form of the thought vectors concept in 2015 some researchers suggested skip thought vectors as a means to improve the quality of machine translation 46 a more recent and popular approach for representing sentences is sentence bert or sentencetransformers which modifies pre trained bert with the use of siamese and triplet network structures 47 software edit software for training and using word embeddings includes tomáš mikolov s word2vec stanford university s glove 48 gn glove 49 flair embeddings 39 allennlp s elmo 50 bert 51 fasttext gensim 52 indra 53 and deeplearning4j principal component analysis pca uniform manifold approximation and projection umap and t distributed stochastic neighbour embedding t sne are used to reduce the dimensionality of word vector spaces and visualize word embeddings and clusters 54 55 examples of application edit for instance the fasttext is also used to calculate word embeddings for text corpora in sketch engine that are available online 56 ethical implications edit word embeddings may contain the biases and stereotypes contained in the trained dataset as bolukbasi et al points out in the 2016 paper man is to computer programmer as woman is to homemaker debiasing word embeddings that a publicly available and popular word2vec embedding trained on google news texts a commonly used data corpus which consists of text written by professional journalists still shows disproportionate word associations reflecting gender and racial biases when extracting word analogies 57 for example one of the analogies generated using the aforementioned word embedding is man is to computer programmer as woman is to homemaker 58 59 research done by jieyu zhao et al shows that the applications of these trained word embeddings without careful oversight likely perpetuates existing bias in society which is introduced through unaltered training data furthermore word embeddings can even amplify these biases 60 61 see also edit embedding machine learning brown clustering distributional relational database references edit jurafsky daniel h james martin 2000 speech and language processing an introduction to natural language processing computational linguistics and speech recognition upper saddle river n j prentice hall isbn 978 0 13 095069 7 mikolov tomas sutskever ilya chen kai corrado greg dean jeffrey 2013 distributed representations of words and phrases and their compositionality arxiv 1310 4546 cs cl lebret rémi collobert ronan 2013 word emdeddings through hellinger pca conference of the european chapter of the association for computational linguistics eacl vol 2014 arxiv 1312 5542 levy omer goldberg yoav 2014 neural word embedding as implicit matrix factorization pdf nips li yitan xu linli 2015 word embedding revisited a new representation learning and explicit matrix factorization perspective pdf int l j conf on artificial intelligence ijcai globerson amir 2007 euclidean embedding of co occurrence data pdf journal of machine learning research qureshi m atif greene derek 2018 06 04 eve explainable vector based embedding technique using wikipedia journal of intelligent information systems 53 137 165 arxiv 1702 06891 doi 10 1007 s10844 018 0511 x issn 0925 9902 s2cid 10656055 levy omer goldberg yoav 2014 linguistic regularities in sparse and explicit word representations pdf conll pp 171 180 socher richard bauer john manning christopher ng andrew 2013 parsing with compositional vector grammars pdf proc acl conf archived from the original pdf on 2016 08 11 retrieved 2014 08 14 socher richard perelygin alex wu jean chuang jason manning chris ng andrew potts chris 2013 recursive deep models for semantic compositionality over a sentiment treebank pdf emnlp sahlgren magnus a brief history of word embeddings firth j r 1957 a synopsis of linguistic theory 1930 1955 studies in linguistic analysis 1 32 reprinted in f r palmer ed 1968 selected papers of j r firth 1952 1959 london longman cite book cs1 maint publisher location link luhn h p 1953 a new method of recording and searching information american documentation 4 14 16 doi 10 1002 asi 5090040104 osgood c e suci g j tannenbaum p h 1957 the measurement of meaning university of illinois press salton gerard 1962 some experiments in the generation of word and document associations proceedings of the december 4 6 1962 fall joint computer conference on afips 62 fall pp 234 250 doi 10 1145 1461518 1461544 isbn 978 1 4503 7879 6 s2cid 9937095 cite book isbn date incompatibility help salton gerard wong a yang c s 1975 a vector space model for automatic indexing communications of the acm 18 11 613 620 doi 10 1145 361219 361220 hdl 1813 6057 s2cid 6473756 dubin david 2004 the most influential paper gerard salton never wrote archived from the original on 18 october 2020 retrieved 18 october 2020 kanerva pentti kristoferson jan and holst anders 2000 random indexing of text samples for latent semantic analysis proceedings of the 22nd annual conference of the cognitive science society p 1036 mahwah new jersey erlbaum 2000 karlgren jussi sahlgren magnus 2001 uesaka yoshinori kanerva pentti asoh hideki eds from words to understanding foundations of real world intelligence csli publications 294 308 sahlgren magnus 2005 an introduction to random indexing proceedings of the methods and applications of semantic indexing workshop at the 7th international conference on terminology and knowledge engineering tke 2005 august 16 copenhagen denmark sahlgren magnus holst anders and pentti kanerva 2008 permutations as a means to encode order in word space in proceedings of the 30th annual conference of the cognitive science society 1300 1305 bengio yoshua réjean ducharme pascal vincent 2000 a neural probabilistic language model pdf neurips bengio yoshua ducharme r...
|
