Meta tags:
description= Pre-trained ELMo Representations for Many Languages - HIT-SCIR/ELMoForManyLangs;
Headings (most frequently used words):
uh, oh, the, elmoformanylangs, navigation, use, saved, searches, pre, elmo, languages, to, your, files, footer, parameters, search, code, repositories, users, issues, pull, requests, provide, feedback, hit, scir, trained, representations, for, many, menu, filter, results, more, quickly, folders, and, latest, commit, history, repository, technique, details, downloads, requirements, usage, training, own, citation, about, releases, packages, contributors, install, package, set, up, config_path, in, command, line, programmatically, topics, resources, license, stars, watchers, forks, init, embedder, of, function, sents2elmo,
Text of the page (most frequently used words):
the (62), and (26), for (21), you (18), elmo (18), model (17), your (17), #elmoformanylangs (14), use (12), code (12), github (11), json (11), security (10), all (10), this (9), please (9), reload (8), from (8), with (8), can (7), python (7), cnn_50_100_512_4096_sample (7), more (7), path (7), view (7), there (6), was (6), word (6), training (6), layer (6), output_layer (6), enterprise (6), community (5), navigation (5), languages (5), error (5), while (5), loading (5), page (5), repository (5), license (5), trained (5), 2018 (5), config_path (5), embedder (5), chinese (5), support (5), search (5), readme (4), resources (4), pre (4), representations (4), many (4), raw (4), our (4), open (4), lstm (4), which (4), sents (4), will (4), input (4), change (4), window (4), settings (4), actions (4), hit (4), issues (4), that (3), stars (3), topics (3), vectors (3), text (3), computational (3), linguistics (3), address (3), also (3), better (3), parsing (3), conll (3), shared (3), task (3), need (3), are (3), data (3), bilm (3), train (3), configs (3), output (3), details (3), second (3), list (3), after (3), sents2elmo (3), parameters (3), batch_size (3), into (3), like (3), words (3), allennlp (3), set (3), file (3), see (3), issue (3), configuration (3), lang (3), zht (3), config (3), setup (3), install (3), files (3), commit (3), name (3), scir (3), insights (3), pull (3), requests (3), signed (3), another (3), tab (3), refresh (3), session (3), sign (3), saved (3), documentation (3), feedback (3), grade (3), features (3), copilot (3), platform (3), solutions (3), explore (3), not (2), manage (2), footer (2), releases (2), forks (2), 236 (2), mit (2), multilingual (2), about (2), inproceedings (2), fares (2), etal (2), 2017 (2), author (2), title (2), towards (2), large (2), booktitle (2), proceedings (2), month (2), year (2), publisher (2), association (2), pages (2), url (2), http (2), www (2), aclweb (2), org (2), anthology (2), cite (2), models (2), nlpl (2), che (2), k18 (2), liu (2), embeddings (2), universal (2), dependencies (2), however (2), add (2), very (2), cases (2), actively (2), snip (2), they (2), build (2), nest (2), made (2), example (2), english (2), run (2), own (2), target (2), layers (2), default (2), hidden (2), first (2), lists (2), store (2), sentences (2), segment (2), necessary (2), when (2), according (2), gpu (2), size (2), dir (2), model_dir (2), each (2), programmatically (2), dump (2), hdf5 (2), encoded (2), separated (2), cnn (2), make (2), field (2), vietnamese (2), sue (2), coffee (2), bill (2), tea (2), find (2), package (2), following (2), requirements (2), must (2), simplified (2), norwegian (2), greek (2), language (2), technique (2), changes (2), manifest (2), gitignore (2), commits (2), last (2), message (2), menu (2), quality (2), wiki (2), projects (2), appearance (2), cancel (2), available (2), searches (2), repositories (2), business (2), advanced (2), developer (2), source (2), customer (2), services (2), devops (2), app (2), perform, action, time, share, personal, information, cookies, contact, docs, status, privacy, terms, 2026, inc, 100, contributors, packages, published, report, watching, watchers, custom, properties, activity, nlp, nodalida, murhaf, kutuzov, andrey, oepen, stephan, velldal, erik, reuse, replicability, 21st, nordic, conference, may, gothenburg, sweden, 271, 276, w17, 0237, hosting, wanxiang, yijia, wang, yuxuan, zheng, ting, deep, contextualized, ensemble, treebank, concatenation, october, brussels, belgium, 2005, gave, nice, improvements, citation, process, stable, some, end, loss, working, hopefully, improve, future, nan, less, notable, alumni, aris, kalafatis, acting, labour, party, tree, hole, man, boxes, legacy, train_path, optimizer, adam, 001, lr_decay, max_epoch, max_sent_len, max_vocab_size, 150000, min_count, here, get, average, encoder, def, function, want, inference, specify, properly, cpu, ram, absolute, repo, top, class, init, shape, seq_len, embedding_size, return, numpy, arrays, import, thanks, voidism, contributing, api, using, object, onto, disk, where, key, sentence, value, averaged, representation, changing, interface, adapted, friendly, dict, test, input_format, output_prefix, output_format, fileds, should, only, column, space, supported, contains, spaces, remember, tokenization, likes, prepare, conllu, format, command, line, under, copy, unzip, relative, commends, usage, other, pytorch, python3, encounter, provided, xinhua, proportion, different, wikipedia, traditional, gigawords, hosted, urdu, ukrainian, uyghur, turkish, swedish, slovene, slovak, russian, romanian, portuguese, polish, nynorsk, dutch, bokmål, latvian, latin, korean, japanese, italian, indonesian, hungarian, croatian, hindi, hebrew, ancient, galician, irish, french, finnish, persian, basque, estonian, spanish, german, danish, old, church, slavonic, czech, catalan, bulgarian, arabic, downloads, one, takes, roughly, days, nvidia, p100, vocabulary, feasible, surrounding, negative, samples, shows, performance, preliminary, experiments, jean, 2015, sample, softmax, unicode, characters, same, hyperparameter, character, their, million, randomly, sampled, released, wikidump, common, crawl, largely, based, ourselves, but, peters, release, helps, win, las, items, history, latest, date, folders, tags, branches, master, additional, options, star, fork, notification, notifications, forked, bozheng, public, dismiss, alert, switched, accounts, out, resetting, focus, create, qualifiers, query, filter, results, quickly, submit, include, email, contacted, read, every, piece, take, seriously, provide, syntax, tips, clear, users, jump, pricing, premium, ons, powered, collections, trending, archive, program, accelerator, maintainer, lab, programs, fund, developers, sponsors, partners, trust, center, forum, skills, ebooks, reports, events, webinars, stories, type, software, development, topic, industries, government, manufacturing, financial, healthcare, industry, devsecops, modernization, case, nonprofits, startups, small, medium, teams, enterprises, company, marketplace, changelog, blog, why, stop, leaks, before, start, secret, protection, secure, fix, vulnerabilities, application, review, plan, track, work, instant, dev, environments, codespaces, automate, any, workflow, workflows, integrate, external, tools, mcp, registry, new, direct, agents, merge, write, creation, toggle, skip, content,
Text of the page (random words):
github hit scir elmoformanylangs pre trained elmo representations for many languages github skip to content navigation menu toggle navigation sign in appearance settings platform ai code creation github copilot write better code with ai github copilot app direct agents from issue to merge mcp registry new integrate external tools developer workflows actions automate any workflow codespaces instant dev environments issues plan and track work code review manage code changes application security github advanced security find and fix vulnerabilities code security secure your code as you build secret protection stop leaks before they start explore why github documentation blog changelog marketplace view all features solutions by company size enterprises small and medium teams startups nonprofits by use case app modernization devsecops devops ci cd view all use cases by industry healthcare financial services manufacturing government view all industries view all solutions resources explore by topic ai software development devops security view all topics explore by type customer stories events webinars ebooks reports business insights github skills support services documentation customer support community forum trust center partners view all resources open source community github sponsors fund open source developers programs security lab maintainer community accelerator github stars archive program repositories topics trending collections enterprise enterprise solutions enterprise platform ai powered developer platform available add ons github advanced security enterprise grade security features copilot for business enterprise grade ai features premium support enterprise grade 24 7 support pricing search or jump to search code repositories users issues pull requests search clear search syntax tips provide feedback we read every piece of feedback and take your input very seriously include my email address so i can be contacted cancel submit feedback saved searches use saved searches to filter your results more quickly name query to see all available qualifiers see our documentation cancel create saved search sign in sign up appearance settings resetting focus you signed in with another tab or window reload to refresh your session you signed out in another tab or window reload to refresh your session you switched accounts on another tab or window reload to refresh your session dismiss alert message uh oh there was an error while loading please reload this page hit scir elmoformanylangs public forked from bozheng hit elmo notifications you must be signed in to change notification settings fork 236 star 1 5k code issues 51 pull requests 2 actions projects wiki security and quality 0 insights additional navigation options code issues pull requests actions projects wiki security and quality insights hit scir elmoformanylangs master branches tags go to file code open more actions menu folders and files name name last commit message last commit date latest commit history 51 commits 51 commits elmoformanylangs elmoformanylangs gitignore gitignore license license manifest in manifest in readme md readme md setup py setup py view all files repository files navigation readme mit license more items pre trained elmo representations for many languages we release our elmo representations trained on many languages which helps us win the conll 2018 shared task on universal dependencies parsing according to las technique details we use the same hyperparameter settings as peters et al 2018 for the bilm and the character cnn we train their parameters on a set of 20 million words data randomly sampled from the raw text released by the shared task wikidump common crawl for each language we largely based ourselves on the code of allennlp but made the following changes we support unicode characters we use the sample softmax technique to make training on large vocabulary feasible jean et al 2015 however we use a window of words surrounding the target word as negative samples and it shows better performance in our preliminary experiments the training of elmo on one language takes roughly 3 days on an nvidia p100 gpu downloads arabic bulgarian catalan czech old church slavonic danish german greek english spanish estonian basque persian finnish french irish galician ancient greek hebrew hindi croatian hungarian indonesian italian japanese korean latin latvian norwegian bokmål dutch norwegian nynorsk polish portuguese romanian russian slovak slovene swedish turkish uyghur ukrainian urdu vietnamese chinese the models are hosted on the nlpl vectors repository elmo for simplified chinese we also provided simplified chinese elmo it was trained on xinhua proportion of chinese gigawords v5 which is different from the wikipedia for traditional chinese elmo pre requirements must python 3 6 if you use python3 5 you will encounter this issue 8 pytorch 0 4 other requirements from allennlp usage install the package you need to install the package to use the embeddings with the following commends python setup py install set up the config_path after unzip the model you will find a json file lang model config json please change the config_path field to the relative path to the model configuration cnn_50_100_512_4096_sample json for example if your elmo model is zht model config json and your model configuration is zht model cnn_50_100_512_4096_sample json you need to change config_path in zht model config json to cnn_50_100_512_4096_sample json if there is no configuration cnn_50_100_512_4096_sample json under lang model you can copy the elmoformanylangs configs cnn_50_100_512_4096_sample json into lang model or change the config_path into elmoformanylangs configs cnn_50_100_512_4096_sample json see issue 27 for more details use elmoformanylangs in command line prepare your input file in the conllu format like 1 sue sue _ _ _ _ _ _ _ 2 likes like _ _ _ _ _ _ _ 3 coffee coffee _ _ _ _ _ _ _ 4 and and _ _ _ _ _ _ _ 5 bill bill _ _ _ _ _ _ _ 6 tea tea _ _ _ _ _ _ _ fileds should be separated by t we only use the second column and space is supported in this field for vietnamese a word can contains spaces do remember tokenization when it s all set run python m elmoformanylangs test input_format conll input path to your input model path to your model output_prefix path to your output output_format hdf5 output_layer 1 it will dump an hdf5 encoded dict onto the disk where the key is t separated words in the sentence and the value is it s 3 layer averaged elmo representation you can also dump the cnn encoded word with output_layer 0 the first layer of the lstm with output_layer 1 and the second layer of the lstm with output_layer 2 we are actively changing the interface to make it more adapted to the allennlp elmo and more programmatically friendly use elmoformanylangs programmatically thanks voidism for contributing the api by using embedder python object you can use elmo into your own code like this from elmoformanylangs import embedder e embedder path to your model sents 今 天 天氣 真 好 阿 潮水 退 了 就 知道 誰 沒 穿 褲子 the list of lists which store the sentences after segment if necessary e sents2elmo sents will return a list of numpy arrays each with the shape seq_len embedding_size the parameters to init embedder class embedder model_dir path to your model batch_size 64 model_dir the absolute path from the repo top dir to you model dir batch_size the batch_size you want when the model inference you can specify it properly according to your gpu cpu ram size default 64 the parameters of the function sents2elmo def sents2elmo sents output_layer 1 sents the list of lists which store the sentences after segment if necessary output_layer the target layer to output 0 for the word encoder 1 for the first lstm hidden layer 2 for the second lstm hidden layer 1 for an average of 3 layers default 2 for all 3 layers training your own elmo please run python m elmoformanylangs bilm train h to get more details about the elmo training here is an example for training english elmo less data en raw snip notable alumni aris kalafatis acting labour party they build an open nest in a tree hole or man made nest boxes legacy snip python m elmoformanylangs bilm train train_path data en raw config_path elmoformanylangs configs cnn_50_100_512_4096_sample json model output en optimizer adam lr 0 001 lr_decay 0 8 max_epoch 10 max_sent_len 20 max_vocab_size 150000 min_count 3 however we need to add that the training process is not very stable in some cases we end up with a loss of nan we are actively working on that and hopefully improve it in the future citation if our elmo gave you nice improvements please cite us inproceedings che etal 2018 k18 2 author che wanxiang and liu yijia and wang yuxuan and zheng bo and liu ting title towards better ud parsing deep contextualized word embeddings ensemble and treebank concatenation booktitle proceedings of the conll 2018 shared task multilingual parsing from raw text to universal dependencies month october year 2018 address brussels belgium publisher association for computational linguistics pages 55 64 url http www aclweb org anthology k18 2005 please also cite the nlpl vectors repository for hosting the models inproceedings fares etal 2017 nodalida author fares murhaf and kutuzov andrey and oepen stephan and velldal erik title word vectors reuse and replicability towards a community repository of large text resources booktitle proceedings of the 21st nordic conference on computational linguistics month may year 2017 address gothenburg sweden publisher association for computational linguistics pages 271 276 url http www aclweb org anthology w17 0237 about pre trained elmo representations for many languages topics multilingual nlp elmo resources readme license mit license uh oh there was an error while loading please reload this page activity custom properties stars 1 5k stars watchers 41 watching forks 236 forks report repository releases no releases published packages 0 uh oh there was an error while loading please reload this page uh oh there was an error while loading please reload this page contributors uh oh there was an error while loading please reload this page languages python 100 0 footer 2026 github inc footer navigation terms privacy security status community docs contact manage cookies do not share my personal information you can t perform that action at this time
|
