Meta tags:
description=BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese (INTERSPEECH 2022) - VinAIResearch/BARTpho;

Headings (most frequently used words):

bartpho, uh, oh, pre, trained, models, navigation, saved, searches, sequence, to, files, using, in, license, footer, installation, example, usage, search, code, repositories, users, issues, pull, requests, provide, feedback, vinairesearch, for, vietnamese, menu, use, filter, your, results, more, quickly, folders, and, latest, commit, history, repository, transformers, fairseq, notes, about, releases, packages, contributors, topics, resources, stars, watchers, forks, table, of, contents,

Text of the page (most frequently used words):
bartpho (51), the (37), and (26), word (19), syllable (17), fairseq (15), #sequence (14), for (14), github (13), this (11), vietnamese (11), pre (11), software (11), code (11), security (10), transformers (10), models (9), vinai (9), all (9), with (9), install (9), text (8), input (8), level (8), reload (7), license (7), trained (7), from (7), features (7), model (7), view (7), you (6), was (6), please (6), use (6), from_pretrained (6), pip (6), pull (6), input_ids (6), your (6), enterprise (6), navigation (5), there (5), while (5), users (5), bartpho_word (5), sentence (5), bpe (5), bartpho_syllable (5), search (5), that (4), community (4), error (4), loading (4), page (4), readme (4), bart (4), any (4), documentation (4), files (4), tokenids (4), những (4), model_folder_path (4), path (4), folder (4), sentencepiece (4), zip (4), git (4), using (4), txt (4), 1024 (4), large (4), base (4), our (4), mbart (4), more (4), actions (4), quality (4), issues (4), support (4), can (3), perform (3), stars (3), mit (3), resources (3), topics (3), 2022 (3), copyright (3), without (3), merge (3), other (3), tone (3), normalization (3), training (3), downstream (3), task (3), available (3), encode (3), bartmodel (3), import (3), request (3), automodel (3), autotokenizer (3), branch (3), tokenizer (3), source (3), results (3), tasks (3), commit (3), name (3), open (3), vinairesearch (3), insights (3), requests (3), signed (3), settings (3), another (3), tab (3), window (3), refresh (3), session (3), sign (3), saved (3), feedback (3), grade (3), copilot (3), platform (3), solutions (3), explore (3), action (2), not (2), manage (2), footer (2), releases (2), published (2), repository (2), forks (2), 105 (2), nlp (2), summarization (2), interspeech (2), permission (2), copy (2), including (2), copies (2), notice (2), shall (2), out (2), should (2), raw (2), used (2), both (2), corpus (2), before (2), process (2), also (2), notes (2), extract_features (2), last_layer_features (2), extract (2), false (2), add_if_not_exist (2), apply (2), chúng_tôi (2), nghiên_cứu_viên (2), eval (2), bpe_codes_path (2), checkpoint_file (2), load (2), chúng (2), tôi (2), nghiên (2), cứu (2), viên (2), spm_model_path (2), example (2), usage (2), unzip (2), 420m (2), 396m (2), params (2), clone (2), https (2), com (2), datquocnguyen (2), issue (2), follows (2), installation (2), return_tensors (2), word_tokenizer (2), syllable_tokenizer (2), tokenizers (2), into (2), main (2), fast (2), paper (2), nguyen (2), architecture (2), two (2), public (2), compare (2), find (2), vietnamesetonenormalization (2), commits (2), last (2), message (2), menu (2), projects (2), appearance (2), cancel (2), see (2), searches (2), repositories (2), business (2), advanced (2), developer (2), customer (2), services (2), devops (2), app (2), time, share, personal, information, cookies, contact, docs, status, privacy, terms, 2026, inc, contributors, packages, report, watching, watchers, custom, properties, activity, pretrained, about, 2021, hereby, granted, free, charge, person, obtaining, associated, deal, restriction, limitation, rights, modify, publish, distribute, sublicense, sell, permit, persons, whom, furnished, subject, following, conditions, above, included, substantial, portions, provided, warranty, kind, express, implied, but, limited, warranties, merchantability, fitness, particular, purpose, noninfringement, event, authors, holders, liable, claim, damages, liability, whether, contract, tort, otherwise, arising, connection, dealings, segment, texts, segmentation, vncorenlp, fine, tuning, data, applied, python, script, here, bpe_codes, fastbpe, codes, sentencepiece_model, download, editable, function, hub_interface, discussed, waiting, approval, facebookresearch, 3905, torch, 150m, 132m, max, length, arch, single, fast_tokenizers_bartpho_phobert_bertweet, note, merged, slow, merging, discussion, detailed, would, like, utilize, might, when, help, produce, incorporated, cite, inproceedings, title, author, luong, tran, duong, minh, dat, quoc, booktitle, proceedings, 23rd, annual, conference, international, speech, communication, association, year, general, experimental, found, present, versions, which, are, first, scale, monolingual, uses, scheme, denoising, autoencoder, thus, especially, suitable, generative, conduct, experiments, its, competitor, show, automatic, human, evaluations, outperforms, strong, baseline, improves, state, art, further, evaluate, capitalization, punctuation, restoration, effective, than, these, introduction, table, contents, items, history, latest, date, folders, file, tags, branches, additional, options, star, fork, must, change, notification, notifications, dismiss, alert, switched, accounts, resetting, focus, create, qualifiers, query, filter, quickly, submit, include, email, address, contacted, read, every, piece, take, very, seriously, provide, syntax, tips, clear, jump, pricing, premium, add, ons, powered, collections, trending, archive, program, accelerator, maintainer, lab, programs, fund, developers, sponsors, partners, trust, center, forum, skills, ebooks, reports, events, webinars, stories, type, development, topic, industries, government, manufacturing, financial, healthcare, industry, cases, devsecops, modernization, case, nonprofits, startups, small, medium, teams, enterprises, company, size, marketplace, changelog, blog, why, stop, leaks, they, start, secret, protection, secure, build, fix, vulnerabilities, application, enforce, changes, review, plan, track, work, instant, dev, environments, codespaces, automate, workflow, workflows, integrate, external, tools, mcp, registry, new, direct, agents, write, better, creation, toggle, skip, content,

Text of the page (random words):
github vinairesearch bartpho bartpho pre trained sequence to sequence models for vietnamese interspeech 2022 github skip to content navigation menu toggle navigation sign in appearance settings platform ai code creation github copilot write better code with ai github copilot app direct agents from issue to merge mcp registry new integrate external tools developer workflows actions automate any workflow codespaces instant dev environments issues plan and track work code review manage code changes code quality enforce quality at merge application security github advanced security find and fix vulnerabilities code security secure your code as you build secret protection stop leaks before they start explore why github documentation blog changelog marketplace view all features solutions by company size enterprises small and medium teams startups nonprofits by use case app modernization devsecops devops ci cd view all use cases by industry healthcare financial services manufacturing government view all industries view all solutions resources explore by topic ai software development devops security view all topics explore by type customer stories events webinars ebooks reports business insights github skills support services documentation customer support community forum trust center partners view all resources open source community github sponsors fund open source developers programs security lab maintainer community accelerator github stars archive program repositories topics trending collections enterprise enterprise solutions enterprise platform ai powered developer platform available add ons github advanced security enterprise grade security features copilot for business enterprise grade ai features premium support enterprise grade 24 7 support pricing search or jump to search code repositories users issues pull requests search clear search syntax tips provide feedback we read every piece of feedback and take your input very seriously include my email address so i can be contacted cancel submit feedback saved searches use saved searches to filter your results more quickly name query to see all available qualifiers see our documentation cancel create saved search sign in sign up appearance settings resetting focus you signed in with another tab or window reload to refresh your session you signed out in another tab or window reload to refresh your session you switched accounts on another tab or window reload to refresh your session dismiss alert message uh oh there was an error while loading please reload this page vinairesearch bartpho public notifications you must be signed in to change notification settings fork 7 star 105 code issues 0 pull requests 0 actions projects security and quality 0 insights additional navigation options code issues pull requests actions projects security and quality insights vinairesearch bartpho main branches tags go to file code open more actions menu folders and files name name last commit message last commit date latest commit history 41 commits 41 commits license license readme md readme md vietnamesetonenormalization md vietnamesetonenormalization md view all files repository files navigation readme mit license more items table of contents introduction using bartpho with transformers using bartpho with fairseq notes bartpho pre trained sequence to sequence models for vietnamese we present bartpho with two versions bartpho syllable and bartpho word which are the first public large scale monolingual sequence to sequence models pre trained for vietnamese bartpho uses the large architecture and the pre training scheme of the sequence to sequence denoising autoencoder bart thus it is especially suitable for generative nlp tasks we conduct experiments to compare our bartpho with its competitor mbart on a downstream task of vietnamese text summarization and show that in both automatic and human evaluations bartpho outperforms the strong baseline mbart and improves the state of the art we further evaluate and compare bartpho and mbart on the vietnamese capitalization and punctuation restoration tasks and also find that bartpho is more effective than mbart on these two tasks the general architecture and experimental results of bartpho can be found in our paper inproceedings bartpho title bartpho pre trained sequence to sequence models for vietnamese author nguyen luong tran and duong minh le and dat quoc nguyen booktitle proceedings of the 23rd annual conference of the international speech communication association year 2022 please cite our paper when bartpho is used to help produce published results or incorporated into other software using bartpho in transformers installation install transformers with pip pip install transformers or install transformers from source note that we merged a slow tokenizer for bartpho into the main transformers branch the process of merging a fast tokenizer for bartpho is in the discussion as detailed in this pull request if users would like to utilize the fast tokenizer the users might install transformers as follows git clone single branch branch fast_tokenizers_bartpho_phobert_bertweet https github com datquocnguyen transformers git cd transformers pip install e install sentencepiece and tokenizers with pip pip install sentencepiece tokenizers pre trained models model params arch max length input text vinai bartpho syllable base 132m base 1024 syllable level vinai bartpho syllable 396m large 1024 syllable level vinai bartpho word base 150m base 1024 word level vinai bartpho word 420m large 1024 word level example usage import torch from transformers import automodel autotokenizer bartpho syllable syllable_tokenizer autotokenizer from_pretrained vinai bartpho syllable bartpho_syllable automodel from_pretrained vinai bartpho syllable txt chúng tôi là những nghiên cứu viên input_ids syllable_tokenizer txt return_tensors pt input_ids features bartpho_syllable input_ids bartpho word word_tokenizer autotokenizer from_pretrained vinai bartpho word bartpho_word automodel from_pretrained vinai bartpho word txt chúng_tôi là những nghiên_cứu_viên input_ids word_tokenizer txt return_tensors pt input_ids features bartpho_word input_ids using bartpho in fairseq installation there is an issue w r t the encode function in the bart hub_interface as discussed in this pull request facebookresearch fairseq 3905 while waiting for this pull request s approval please install fairseq as follows git clone https github com datquocnguyen fairseq git cd fairseq pip install editable pre trained models model params download input text bartpho syllable 396m fairseq bartpho syllable zip syllable level bartpho word 420m fairseq bartpho word zip word level unzip fairseq bartpho syllable zip unzip fairseq bartpho word zip example usage from fairseq models bart import bartmodel load bartpho syllable model model_folder_path path to folder fairseq bartpho syllable spm_model_path path to folder fairseq bartpho syllable sentence bpe model bartpho_syllable bartmodel from_pretrained model_folder_path checkpoint_file model pt bpe sentencepiece sentencepiece_model spm_model_path eval input syllable level raw text sentence chúng tôi là những nghiên cứu viên apply sentencepiece to the input text tokenids bartpho_syllable encode sentence add_if_not_exist false extract features from bartpho syllable last_layer_features bartpho_syllable extract_features tokenids load bartpho word model model_folder_path path to folder fairseq bartpho word bpe_codes_path path to folder fairseq bartpho word bpe codes bartpho_word bartmodel from_pretrained model_folder_path checkpoint_file model pt bpe fastbpe bpe_codes bpe_codes_path eval input word level text sentence chúng_tôi là những nghiên_cứu_viên apply bpe to the input text tokenids bartpho_word encode sentence add_if_not_exist false extract features from bartpho word last_layer_features bartpho_word extract_features tokenids notes before fine tuning bartpho on a downstream task users should perform vietnamese tone normalization on the downstream task s data as this pre process was also applied to the pre training corpus a python script for vietnamese tone normalization is available at here for bartpho word users should use vncorenlp to segment input raw texts as it was used to perform both vietnamese tone normalization and word segmentation on the pre training corpus license mit license copyright c 2021 vinai permission is hereby granted free of charge to any person obtaining a copy of this software and associated documentation files the software to deal in the software without restriction including without limitation the rights to use copy modify merge publish distribute sublicense and or sell copies of the software and to permit persons to whom the software is furnished to do so subject to the following conditions the above copyright notice and this permission notice shall be included in all copies or substantial portions of the software the software is provided as is without warranty of any kind express or implied including but not limited to the warranties of merchantability fitness for a particular purpose and noninfringement in no event shall the authors or copyright holders be liable for any claim damages or other liability whether in an action of contract tort or otherwise arising from out of or in connection with the software or the use or other dealings in the software about bartpho pre trained sequence to sequence models for vietnamese interspeech 2022 topics bart text summarization sequence to sequence pretrained models vietnamese nlp bartpho resources readme license mit license uh oh there was an error while loading please reload this page activity custom properties stars 105 stars watchers 4 watching forks 7 forks report repository releases no releases published packages 0 uh oh there was an error while loading please reload this page contributors uh oh there was an error while loading please reload this page footer 2026 github inc footer navigation terms privacy security status community docs contact manage cookies do not share my personal information you can t perform that action at this time