Meta tags:

Headings (most frequently used words):

gpt, contents, architecture, references,

Text of the page (most frequently used words):
gpt (37), the (25), 2023 (11), model (11), this (10), wikipedia (9), transformer (9), and (8), from (8), 2021 (8), with (7), articles (7), article (7), text (6), page (6), 2026 (6), language (6), edit (6), search (5), was (5), march (5), references (5), notability (5), source (5), pre (5), trained (5), retrieved (5), how (5), learn (5), sources (5), topic (4), contents (4), may (4), non (4), all (4), generative (4), large (4), github (4), 2025 (4), june (4), jax (4), designed (4), like (4), primary (4), has (4), when (4), remove (4), hide (4), move (4), sidebar (4), languages (3), view (3), foundation (3), license (3), issues (3), reliable (3), models (3), openai (3), 2017 (3), current (3), codex (3), mesh (3), not (3), for (3), needed (3), size (3), that (3), eleutherai (3), message (3), please (3), help (3), tools (3), main (3), toggle (2), table (2), contact (2), about (2), privacy (2), policy (2), available (2), terms (2), using (2), categories (2), needing (2), factual (2), verification (2), pages (2), multiple (2), lacking (2), topics (2), unclear (2), short (2), description (2), wikidata (2), open (2), intelligence (2), copilot (2), deep (2), image (2), related (2), fidji (2), simo (2), 2019 (2), will (2), greg (2), brockman (2), former (2), sam (2), altman (2), chatgpt (2), wang (2), ben (2), hugging (2), face (2), generate (2), english (2), prompt (2), other (2), information (2), only (2), tokens (2), billion (2), parameters (2), architecture (2), developed (2), name (2), these (2), improve (2), jstor (2), scholar (2), books (2), newspapers (2), news (2), find (2), secondary (2), general (2), talk (2), appearance (2), upload (2), file (2), changes (2), links (2), history (2), read (2), log (2), create (2), account (2), donate (2), menu (2), add, mobile, cookie, statement, statistics, developers, code, conduct, legal, safety, contacts, disclaimers, under, additional, apply, site, you, agree, registered, trademark, profit, organization, wikimedia, inc, use, creative, commons, attribution, sharealike, rendered, parsoid, last, edited, april, utc, hidden, maintenance, different, artificial, transformers, https, org, index, php, title, oldid, 1350337778, category, five, trade, unions, microsoft, langchain, learning, contrastive, training, autogpt, dungeon, apple, stargate, llc, jvs, 2024, larry, summers, shivon, zilis, helen, toner, ilya, sutskever, 2015, 2018, elon, musk, holden, karnofsky, hurd, reid, hoffman, chair, bret, taylor, nicole, seligman, adebayo, ogunlesi, paul, nakasone, zico, kolter, sue, desmond, hellmann, adam, angelo, board, directors, emmett, shear, mira, murati, dario, amodei, scott, schools, jakub, pachocki, sarah, friar, removal, senior, management, people, operator, agent, intelligent, agents, whisper, sora, dall, specialized, astra, oss, mini, store, research, atlas, products, komatsuzaki, aran, kingoflolz, translating, generating, performance, without, first, specific, task, llms, programmed, give, factually, accurate, based, probability, fine, tuning, dataset, library, handle, parallelization, scheme, pile, layers, attention, heads, its, vocabulary, 50257, same, 2048, context, window, decoder, solve, nlp, tasks, predicting, piece, continue, natural, processing, autoregressive, llm, suggests, produce, human, continues, optional, refers, fact, but, web, interface, longer, communicates, development, stopped, eleuther, website, apache, type, years, ago, release, developer, logo, discuss, adding, tertiary, relies, excessively, demonstrate, citing, are, provide, significant, coverage, beyond, mere, trivial, mention, cannot, shown, likely, deleted, redirected, merged, independent, meet, guideline, messages, free, encyclopedia, item, projects, printable, version, download, pdf, print, export, switch, legacy, parser, get, shortened, url, cite, permanent, link, what, here, actions, українська, türkçe, русский, မြန်မာဘာသာ, 한국어, ido, suomi, فارسی, català, العربية, top, personal, special, recent, community, portal, contribute, random, events, navigation, jump, content,

Text of the page (random words):
gpt j wikipedia jump to content main menu main menu move to sidebar hide navigation main page contents current events random article about wikipedia contact us contribute help learn to edit community portal recent changes upload file special pages search search appearance donate create account log in personal tools donate create account log in contents move to sidebar hide top 1 architecture 2 references toggle the table of contents gpt j 11 languages العربية català فارسی suomi ido 한국어 မြန်မာဘာသာ русский türkçe українська 中文 edit links article talk english read edit view history tools tools move to sidebar hide actions read edit view history general what links here related changes upload file permanent link page information cite this page get shortened url switch to legacy parser print export download as pdf printable version in other projects wikidata item appearance move to sidebar hide from wikipedia the free encyclopedia large language model developed by eleutherai this article has multiple issues please help improve it or discuss these issues on the talk page learn how and when to remove these messages the topic of this article may not meet wikipedia s general notability guideline please help demonstrate the notability of the topic by citing reliable secondary sources that are independent of the topic and provide significant coverage of it beyond a mere trivial mention if notability cannot be shown the article is likely to be merged redirected or deleted find sources gpt j news newspapers books scholar jstor march 2026 learn how and when to remove this message this article relies excessively on references to primary sources please improve this article by adding secondary or tertiary sources find sources gpt j news newspapers books scholar jstor march 2026 learn how and when to remove this message learn how and when to remove this message gpt j logo developer eleutherai release june 9 2021 5 years ago 2021 06 09 type large language model generative pre trained transformer foundation model license apache license 2 0 website 6b eleuther ai gpt j or gpt j 6b is an open source large language model llm developed by eleutherai in 2021 as the name suggests it is a generative pre trained transformer model designed to produce human like text that continues from a prompt the optional 6b in the name refers to the fact that it has 6 billion parameters 1 the model is available on github but the web interface no longer communicates with the model development stopped in 2021 2 architecture edit gpt j is a gpt 3 like model with 6 billion parameters like gpt 3 it is an autoregressive decoder only transformer model designed to solve natural language processing nlp tasks by predicting how a piece of text will continue the model has 28 transformer layers and 16 attention heads its vocabulary size is 50257 tokens the same size as gpt 2 s 1 it has a context window size of 2048 tokens 3 non primary source needed it was trained on the pile dataset 1 using the mesh transformer jax library in jax to handle the parallelization scheme 1 4 non primary source needed gpt j was designed to generate english text from a prompt it was not designed for translating or generating text in other languages or for performance without first fine tuning the model for a specific task like all llms it is not programmed to give factually accurate information only to generate text based on probability 1 non primary source needed references edit 1 2 3 4 5 gpt j 6b hugging face 3 may 2023 retrieved 13 june 2023 wang ben 2025 01 25 kingoflolz mesh transformer jax retrieved 2025 01 27 gpt j github hugging face retrieved 23 june 2023 wang ben komatsuzaki aran may 2021 mesh transformer jax github retrieved 13 june 2023 v t e openai products chatgpt atlas deep research gpt store search foundation models gpt models gpt 1 gpt 2 gpt 3 gpt 3 5 gpt 4 gpt 4o o1 o3 gpt 4 5 gpt 4 1 o4 mini gpt oss gpt 5 gpt 5 1 gpt 5 2 gpt 5 4 gpt 5 5 gpt 5 6 gpt 6 astra specialized dall e codex gpt image sora whisper intelligent agents chatgpt agent codex gpt 5 3 codex operator people senior management current sam altman removal greg brockman sarah friar jakub pachocki scott schools fidji simo former dario amodei mira murati emmett shear board of directors current sam altman adam d angelo sue desmond hellmann zico kolter paul nakasone adebayo ogunlesi nicole seligman bret taylor chair former greg brockman 2017 2023 reid hoffman 2019 2023 will hurd 2021 2023 holden karnofsky 2017 2021 elon musk 2015 2018 ilya sutskever 2017 2023 helen toner 2021 2023 shivon zilis 2019 2023 larry summers 2023 2025 fidji simo 2024 2025 jvs stargate llc related apple intelligence ai dungeon autogpt generative pre trained transformer github copilot contrastive language image pre training deep learning langchain microsoft copilot openai and trade unions openai five transformer gpt j category retrieved from https en wikipedia org w index php title gpt j oldid 1350337778 categories large language models generative pre trained transformers open source artificial intelligence hidden categories articles with short description short description is different from wikidata articles with topics of unclear notability from march 2026 all articles with topics of unclear notability articles lacking reliable references from march 2026 all articles lacking reliable references articles with multiple maintenance issues all pages needing factual verification wikipedia articles needing factual verification from march 2026 this page was last edited on 21 april 2026 at 10 22 utc page was rendered with parsoid text is available under the creative commons attribution sharealike 4 0 license additional terms may apply by using this site you agree to the terms of use and privacy policy wikipedia is a registered trademark of the wikimedia foundation inc a non profit organization privacy policy about wikipedia disclaimers contact wikipedia legal safety contacts code of conduct developers statistics cookie statement mobile view search search toggle the table of contents gpt j 11 languages add topic