Meta tags:
description=PhoBERT: Pre-trained language models for Vietnamese (EMNLP-2020 Findings) - VinAIResearch/PhoBERT;

Headings (most frequently used words):

uh, oh, phobert, navigation, saved, searches, pre, trained, models, files, using, with, footer, installation, example, usage, search, code, repositories, users, issues, pull, requests, provide, feedback, vinairesearch, language, for, vietnamese, menu, use, to, filter, your, results, more, quickly, folders, and, latest, commit, history, repository, transformers, fairseq, notes, about, releases, packages, contributors, topics, resources, license, licenses, found, stars, watchers, forks, table, of, contents,

Text of the page (most frequently used words):
phobert (33), the (19), and (19), transformers (13), pre (13), github (12), for (12), #models (11), security (10), vietnamese (10), with (10), code (10), license (9), language (9), reload (8), trained (8), install (8), base (8), all (8), this (7), please (7), word (7), from (7), texts (7), view (7), you (6), vncorenlp (6), vinai (6), your (6), enterprise (6), navigation (5), there (5), was (5), error (5), while (5), loading (5), page (5), mit (5), tokenizer (5), search (5), community (4), readme (4), rdrsegmenter (4), 2020 (4), ông (4), tại (4), py_vncorenlp (4), sentence (4), import (4), pip (4), input (4), are (4), training (4), use (4), using (4), features (4), large (4), pull (4), more (4), actions (4), issues (4), support (4), can (3), repository (3), stars (3), license_for_phobert_v2 (3), agpl (3), found (3), resources (3), tagging (3), bert (3), fairseq (3), topics (3), emnlp (3), findings (3), chúc (3), text (3), segmentation (3), example (3), usage (3), installation (3), must (3), see (3), from_pretrained (3), torch (3), 20gb (3), wikipedia (3), news (3), 256 (3), branch (3), users (3), source (3), our (3), results (3), files (3), commit (3), name (3), open (3), vinairesearch (3), insights (3), requests (3), signed (3), settings (3), another (3), tab (3), window (3), refresh (3), session (3), sign (3), saved (3), documentation (3), feedback (3), grade (3), copilot (3), platform (3), solutions (3), explore (3), that (2), manage (2), footer (2), releases (2), published (2), forks (2), 113 (2), 791 (2), licenses (2), part (2), speech (2), roberta (2), nlp (2), natural (2), inference (2), named (2), entity (2), recognition (2), đang (2), làm_việc (2), lan (2), cũng (2), đây (2), output (2), làm (2), việc (2), absolute (2), path (2), save_dir (2), case (2), segmenter (2), produce (2), segmented (2), before (2), process (2), data (2), based (2), downstream (2), raw (2), notes (2), tfautomodel (2), input_ids (2), autotokenizer (2), automodel (2), 135m (2), pip3 (2), tokenizers (2), git (2), into (2), fast (2), paper (2), software (2), nguyen (2), monolingual (2), new (2), state (2), art (2), public (2), readme_fairseq (2), commits (2), last (2), message (2), menu (2), quality (2), projects (2), appearance (2), cancel (2), available (2), searches (2), repositories (2), business (2), advanced (2), developer (2), customer (2), services (2), devops (2), app (2), perform, action, time, not, share, personal, information, cookies, contact, docs, status, privacy, terms, 2026, inc, contributors, packages, report, watching, watchers, custom, properties, activity, library, embeddings, nli, pos, ner, python3, deep, learning, about, nguyễn_khắc_chúc, đại_học, quốc_gia, hà_nội, print, word_segment, nguyễn, khắc, đại, học, quốc, gia, nội, wseg, annotators, load, component, download_model, save, them, some, local, machine, folder, automatically, download, components, original, without, applied, feeding, employed, including, recommended, also, same, applications, tone, normalization, details, here, tensorflow, outputs, now, tuples, no_grad, encode, tensor, chúng_tôi, những, nghiên_cứu_viên, already, 370m, gnu, affero, gpl, 120gb, oscar, 2301, max, length, arch, params, model, clone, single, fast_tokenizers_bartpho_phobert_bertweet, https, com, datquocnguyen, note, merged, slow, main, merging, discussion, mentioned, would, like, utilize, might, follows, request, when, used, help, incorporated, other, cite, inproceedings, title, author, dat, quoc, anh, tuan, booktitle, association, computational, linguistics, year, pages, 1037, 1042, general, architecture, experimental, outperforms, previous, multilingual, approaches, obtaining, performances, four, tasks, dependency, parsing, two, versions, first, scale, approach, which, optimizes, procedure, robust, performance, phở, popular, food, vietnam, pho, introduction, table, contents, items, history, latest, date, folders, file, tags, branches, master, additional, options, star, fork, change, notification, notifications, dismiss, alert, switched, accounts, out, resetting, focus, create, qualifiers, query, filter, quickly, submit, include, email, address, contacted, read, every, piece, take, very, seriously, provide, syntax, tips, clear, jump, pricing, premium, add, ons, powered, collections, trending, archive, program, accelerator, maintainer, lab, programs, fund, developers, sponsors, partners, trust, center, forum, skills, ebooks, reports, events, webinars, stories, type, development, topic, industries, government, manufacturing, financial, healthcare, industry, cases, devsecops, modernization, nonprofits, startups, small, medium, teams, enterprises, company, size, marketplace, changelog, blog, why, stop, leaks, they, start, secret, protection, secure, build, find, fix, vulnerabilities, application, changes, review, plan, track, work, instant, dev, environments, codespaces, automate, any, workflow, workflows, integrate, external, tools, mcp, registry, direct, agents, issue, merge, write, better, creation, toggle, skip, content,

Text of the page (random words):
github vinairesearch phobert phobert pre trained language models for vietnamese emnlp 2020 findings github skip to content navigation menu toggle navigation sign in appearance settings platform ai code creation github copilot write better code with ai github copilot app direct agents from issue to merge mcp registry new integrate external tools developer workflows actions automate any workflow codespaces instant dev environments issues plan and track work code review manage code changes application security github advanced security find and fix vulnerabilities code security secure your code as you build secret protection stop leaks before they start explore why github documentation blog changelog marketplace view all features solutions by company size enterprises small and medium teams startups nonprofits by use case app modernization devsecops devops ci cd view all use cases by industry healthcare financial services manufacturing government view all industries view all solutions resources explore by topic ai software development devops security view all topics explore by type customer stories events webinars ebooks reports business insights github skills support services documentation customer support community forum trust center partners view all resources open source community github sponsors fund open source developers programs security lab maintainer community accelerator github stars archive program repositories topics trending collections enterprise enterprise solutions enterprise platform ai powered developer platform available add ons github advanced security enterprise grade security features copilot for business enterprise grade ai features premium support enterprise grade 24 7 support pricing search or jump to search code repositories users issues pull requests search clear search syntax tips provide feedback we read every piece of feedback and take your input very seriously include my email address so i can be contacted cancel submit feedback saved searches use saved searches to filter your results more quickly name query to see all available qualifiers see our documentation cancel create saved search sign in sign up appearance settings resetting focus you signed in with another tab or window reload to refresh your session you signed out in another tab or window reload to refresh your session you switched accounts on another tab or window reload to refresh your session dismiss alert message uh oh there was an error while loading please reload this page vinairesearch phobert public notifications you must be signed in to change notification settings fork 113 star 791 code issues 4 pull requests 0 actions projects security and quality 0 insights additional navigation options code issues pull requests actions projects security and quality insights vinairesearch phobert master branches tags go to file code open more actions menu folders and files name name last commit message last commit date latest commit history 46 commits 46 commits license license license_for_phobert_v2 license_for_phobert_v2 readme md readme md readme_fairseq md readme_fairseq md view all files repository files navigation readme mit license agpl 3 0 license more items table of contents introduction using phobert with transformers installation pre trained models example usage using phobert with fairseq notes phobert pre trained language models for vietnamese pre trained phobert models are the state of the art language models for vietnamese pho i e phở is a popular food in vietnam two phobert versions of base and large are the first public large scale monolingual language models pre trained for vietnamese phobert pre training approach is based on roberta which optimizes the bert pre training procedure for more robust performance phobert outperforms previous monolingual and multilingual approaches obtaining new state of the art performances on four downstream vietnamese nlp tasks of part of speech tagging dependency parsing named entity recognition and natural language inference the general architecture and experimental results of phobert can be found in our paper inproceedings phobert title phobert pre trained language models for vietnamese author dat quoc nguyen and anh tuan nguyen booktitle findings of the association for computational linguistics emnlp 2020 year 2020 pages 1037 1042 please cite our paper when phobert is used to help produce published results or is incorporated into other software using phobert with transformers installation install transformers with pip pip install transformers or install transformers from source note that we merged a slow tokenizer for phobert into the main transformers branch the process of merging a fast tokenizer for phobert is in the discussion as mentioned in this pull request if users would like to utilize the fast tokenizer the users might install transformers as follows git clone single branch branch fast_tokenizers_bartpho_phobert_bertweet https github com datquocnguyen transformers git cd transformers pip3 install e install tokenizers with pip pip3 install tokenizers pre trained models model params arch max length pre training data license vinai phobert base v2 135m base 256 20gb of wikipedia and news texts 120gb of texts from oscar 2301 gnu affero gpl v3 vinai phobert base 135m base 256 20gb of wikipedia and news texts mit license vinai phobert large 370m large 256 20gb of wikipedia and news texts mit license example usage import torch from transformers import automodel autotokenizer phobert automodel from_pretrained vinai phobert base v2 tokenizer autotokenizer from_pretrained vinai phobert base v2 input text must be already word segmented sentence chúng_tôi là những nghiên_cứu_viên input_ids torch tensor tokenizer encode sentence with torch no_grad features phobert input_ids models outputs are now tuples with tensorflow 2 0 from transformers import tfautomodel phobert tfautomodel from_pretrained vinai phobert base using phobert with fairseq please see details at here notes in case the input texts are raw i e without word segmentation a word segmenter must be applied to produce word segmented texts before feeding to phobert as phobert employed the rdrsegmenter from vncorenlp to pre process the pre training data including vietnamese tone normalization and word and sentence segmentation it is recommended to also use the same word segmenter for phobert based downstream applications w r t the input raw texts installation pip install py_vncorenlp example usage import py_vncorenlp automatically download vncorenlp components from the original repository and save them in some local machine folder py_vncorenlp download_model save_dir absolute path to vncorenlp load the word and sentence segmentation component rdrsegmenter py_vncorenlp vncorenlp annotators wseg save_dir absolute path to vncorenlp text ông nguyễn khắc chúc đang làm việc tại đại học quốc gia hà nội bà lan vợ ông chúc cũng làm việc tại đây output rdrsegmenter word_segment text print output ông nguyễn_khắc_chúc đang làm_việc tại đại_học quốc_gia hà_nội bà lan vợ ông chúc cũng làm_việc tại đây about phobert pre trained language models for vietnamese emnlp 2020 findings topics deep learning vietnamese transformers python3 named entity recognition language models ner natural language inference bert pos tagging vietnamese nlp nli fairseq vncorenlp roberta bert embeddings rdrsegmenter part of speech tagging transformers library phobert resources readme license mit agpl 3 0 licenses found licenses found mit license agpl 3 0 license_for_phobert_v2 uh oh there was an error while loading please reload this page activity custom properties stars 791 stars watchers 19 watching forks 113 forks report repository releases no releases published packages 0 uh oh there was an error while loading please reload this page uh oh there was an error while loading please reload this page contributors uh oh there was an error while loading please reload this page footer 2026 github inc footer navigation terms privacy security status community docs contact manage cookies do not share my personal information you can t perform that action at this time