Meta tags:
description= GitHub is where people build software. More than 225 million people use GitHub to discover, fork, and contribute to over 800 million projects.;
Headings (most frequently used words):
llm, tensorrt, navigation, this, topic, footer, ai, dynamo, awesome, inference, netease, media, menu, here, are, 102, public, repositories, matching, add, to, your, repo, xlite, dev, collabora, whisperlive, semianalysisai, inferencex, shashikg, whispers2t, coderonion, cuda, and, hpc, huggingface, optimum, benchmark, npuichigo, openai_trtllm, grps, grps_trtllm, yanght27, gpu, perf, playground, guidance, llgtrt, vossr, chat, with, rtx, python, api, modal, labs, stopwatch, argonne, lcf, bench, fgblanch, outlookllm, janhq, cortex, zrzrzrzrzrzrzr, lm, fly, lix19937, deploy, wcks13589, tutorial,
Text of the page (most frequently used words):
llm (65), #tensorrt (43), code (24), updated (22), inference (21), star (21), and (20), pull (20), requests (20), issues (20), python (17), github (14), with (13), vllm (12), for (11), cuda (11), 2026 (9), gpu (9), 2025 (8), nvidia (8), triton (8), openai (8), your (7), chat (6), rust (6), pytorch (6), deepseek (6), support (6), awesome (6), nemo (5), server (5), aug (5), sglang (5), api (5), whisper (5), attention (5), all (5), you (4), this (4), security (4), the (4), may (4), 2024 (4), openvino (4), features (4), rtx (4), performance (4), discussions (4), serving (4), nov (4), text (4), speech (4), flash (4), copilot (4), that (3), learn (3), repository (3), topic (3), topics (3), add (3), tutorial (3), jupyter (3), notebook (3), deepspeed (3), benchmark (3), learning (3), benchmarking (3), modal (3), swift (3), engine (3), multi (3), high (3), language (3), hpc (3), most (3), stars (3), another (3), tab (3), window (3), refresh (3), session (3), reload (3), sign (3), developers (3), changelog (3), customer (3), enterprise (3), grade (3), build (3), can (2), not (2), manage (2), community (2), status (2), footer (2), navigation (2), more (2), repo (2), guardrails (2), infra (2), deploy (2), sep (2), cortex (2), outlook (2), new (2), bench (2), llms (2), structured (2), generation (2), guidance (2), json (2), programming (2), trt (2), qwen3 (2), qwen2 (2), llama (2), agent (2), function (2), call (2), service (2), supporting (2), distributed (2), netease (2), media (2), dynamic (2), batching (2), torch (2), deep (2), framework (2), compatible (2), services (2), backend (2), transformers (2), optimum (2), mlir (2), tvm (2), ptx (2), vlm (2), cudnn (2), cutlass (2), cublas (2), public (2), projects (2), voice (2), detection (2), recognition (2), agentic (2), rocm (2), source (2), platform (2), mla (2), paged (2), dynamo (2), recently (2), fewest (2), forks (2), sort (2), 102 (2), sponsors (2), events (2), trending (2), explore (2), signed (2), universe (2), training (2), integrations (2), stories (2), center (2), documentation (2), business (2), fixes (2), products (2), secret (2), quality (2), perform, action, time, share, personal, information, cookies, contact, docs, privacy, terms, inc, associate, visit, landing, page, select, load, jun, materials, include, but, limited, wcks13589, infer, lix19937, tgi, mlx, 大模型推理框架加速, 飞起来, sponsor, fly, zrzrzrzrzrzrzr, jan, library, loaded, any, runtime, submodules, accelerated, gpus, janhq, addin, adds, composition, summarizing, uses, local, via, outlookllm, fgblanch, jul, llamacpp, argonne, lcf, machine, tool, stopwatch, labs, llama2, 13b, mistral, vossr, apr, cfg, regex, outputs, built, llgtrt, kernel, optimization, toolkit, kernels, 性能与, 学习项目, ncu, nsys, 并行架构, 126, perf, playground, yanght27, dec, olmocr, janus, pro, internvl, minicpm, llama3, internvideo, chatglm, index, qwq, phi, higher, than, serve, pure, implemented, gprs, tokenizers, cpp, agents, multimodal, capabilities, gradio, interface, 161, grps_trtllm, tensorflow, deployment, supports, trtllm, other, frameworks, streaming, modes, dual, offering, scalability, extensibility, helps, users, quickly, models, provide, through, http, rpc, interfaces, 172, grps, langchain, 221, openai_trtllm, npuichigo, neural, compressor, onnxruntime, unified, utility, timm, peft, diffusers, sentence, full, hardware, optimizations, quantization, schemes, 342, huggingface, gemm, blas, lists, some, computing, 525, coderonion, activity, asr, vad, optimized, pipeline, model, multiple, 582, whispers2t, shashikg, qwen, mlops, kimi, benchmarks, amd, open, accelerator, research, standard, 开源推理研究平台, inferencex, semianalysisai, intel, dictation, obs, translation, nearly, live, implementation, whisperlive, collabora, oct, minimax, curated, list, papers, codes, wint8, parallelism, etc, xlite, dev, disaggregated, diffusion, omni, routing, kubernetes, datacenter, scale, least, options, javascript, shell, html, typescript, filter, here, are, repositories, matching, collections, message, dismiss, alert, switched, accounts, out, appearance, settings, search, pricing, event, talks, workshops, what, next, partners, about, company, forum, webinars, blog, resource, connect, resources, premium, advanced, ons, trust, insights, why, evaluate, see, industries, government, manufacturing, financial, healthcare, industry, nonprofits, startups, team, enterprises, stage, businesses, latest, improvements, across, one, place, developer, articles, repos, customize, certifications, scanning, leak, prevention, protection, vulnerability, remediation, secure, cloud, development, environments, codespaces, analysis, enforcement, automation, actions, ship, coding, tasks, automated, work, hosting, review, collaboration, menu, skip, content,
Text of the page (random words):
tensorrt llm github topics github skip to content navigation menu sign in products platform github repository hosting code review and collaboration copilot ai coding agent for tasks projects and automated work build ship actions ci cd and automation code quality quality analysis fixes and enforcement codespaces cloud development environments secure code security vulnerability detection and remediation secret protection secret scanning and leak prevention all features developers learn documentation training certifications build build with copilot customize copilot integrations explore trending repos github sponsors developer articles changelog changelog your source for the latest features improvements and fixes across github products all in one place businesses by stage enterprises team startups nonprofits by industry healthcare financial services manufacturing government see all industries evaluate github why github customer stories business insights trust center add ons github advanced security enterprise grade code security copilot for business enterprise grade ai features premium support enterprise grade 24 7 support resources learn connect documentation resource center blog events webinars community forum customer stories integrations support customer support training github status company about partners changelog github universe github universe the event for developers by developers talks workshops and what s next for github pricing search sign in sign up appearance settings you signed in with another tab or window reload to refresh your session you signed out in another tab or window reload to refresh your session you switched accounts on another tab or window reload to refresh your session dismiss alert message explore topics trending collections events github sponsors tensorrt llm star here are 102 public repositories matching this topic language all filter by language all 102 python 63 c 8 jupyter notebook 5 rust 5 typescript 5 html 3 shell 3 go 2 javascript 1 d2 1 sort most stars sort options most stars fewest stars most forks fewest forks recently updated least recently updated ai dynamo dynamo star 8 5k code issues pull requests discussions a datacenter scale distributed inference serving framework kubernetes rust routing engine omni diffusion vllm llm inference tensorrt llm sglang disaggregated serving updated nov 17 2026 rust xlite dev awesome llm inference star 5 6k code issues pull requests a curated list of awesome llm vlm inference papers with codes flash attention paged attention wint8 4 parallelism etc mla vllm llm inference awesome llm flash attention tensorrt llm paged attention deepseek flash attention 3 deepseek v3 minimax 01 deepseek r1 flash mla qwen3 updated oct 20 2026 python collabora whisperlive star 4 4k code issues pull requests a nearly live implementation of openai s whisper text to speech translation voice recognition openai obs dictation whisper rocm tensorrt openvino openvino intel tensorrt llm whisper tensorrt updated nov 15 2026 python semianalysisai inferencex star 2k code issues pull requests discussions open source ai accelerator research platform standard 开源推理研究平台 benchmarking amd gpu cuda inference pytorch nvidia benchmarks rocm kimi mlops llm serving vllm llm inference agentic qwen tensorrt llm deepseek sglang agentic ai updated nov 17 2026 python shashikg whispers2t star 582 code issues pull requests an optimized speech to text pipeline for the whisper model supporting multiple inference engine deep learning speech recognition vad speech to text whisper asr tensorrt voice activity detection tensorrt llm updated aug 27 2024 jupyter notebook coderonion awesome cuda and hpc star 525 code issues pull requests this repository lists some awesome public cuda cuda python cublas cudnn cutlass tensorrt tensorrt llm triton tvm mlir ptx and high performance computing hpc projects awesome hpc gpu cuda pytorch cublas triton blas llama cutlass cudnn gemm vlm tensorrt ptx tvm mlir llm tensorrt llm deepseek updated aug 2 2025 huggingface optimum benchmark star 342 code issues pull requests ️ a unified multi backend utility for benchmarking transformers timm peft diffusers and sentence transformers with full support of optimum s hardware optimizations quantization schemes benchmark pytorch openvino onnxruntime text generation inference neural compressor tensorrt llm updated nov 10 2026 python npuichigo openai_trtllm star 221 code issues pull requests openai compatible api for tensorrt llm triton backend triton inference server openai api llm langchain tensorrt llm updated aug 1 2024 rust netease media grps star 172 code issues pull requests discussions deep learning deployment framework supports tf torch trt trtllm vllm and other nn frameworks support dynamic batching and streaming modes it is dual language compatible with python and c offering scalability extensibility and high performance it helps users quickly deploy models and provide services through http rpc interfaces tensorflow torch tensorrt serving triton inference server dynamic batching vllm tensorrt llm updated may 8 2025 c netease media grps_trtllm star 161 code issues pull requests discussions higher performance openai llm service than vllm serve a pure c high performance openai llm service implemented with gprs tensorrt llm tokenizers cpp supporting chat and function call ai agents distributed multi gpu inference multimodal capabilities and a gradio chat interface openai multi modal phi function call qwq ai agent llm llama index chatglm internvideo tensorrt llm qwen2 llama3 minicpm v internvl qwen2 vl deepseek r1 janus pro olmocr qwen3 updated dec 8 2025 python yanght27 gpu perf playground star 126 code issues pull requests gpu 性能与 ai infra 学习项目 cuda triton 算子 ncu nsys vllm sglang trt llm ms swift pytorch deepspeed ms swift 训练 并行架构 ai gpu cuda pytorch triton cuda kernels cuda toolkit tensorrt pytorch tutorial gpu programming gpu optimization cuda programming deepspeed cuda kernel tensorrt engine llm vllm tensorrt llm ms swift sglang updated sep 8 2026 python guidance ai llgtrt star 73 code issues pull requests tensorrt llm server with structured outputs json built with rust json regex guidance cfg openai api tensorrt llm structured generation updated apr 25 2025 rust vossr chat with rtx python api star 67 code issues pull requests chat with rtx python api tensorrt llm llm inference tensorrt llm mistral 7b llama2 13b chat with rtx nvidia chat with rtx updated may 11 2025 python modal labs stopwatch star 66 code issues pull requests a tool for benchmarking llms on modal machine learning llms vllm tensorrt llm sglang updated aug 29 2025 python argonne lcf llm inference bench star 63 code issues pull requests llm inference bench benchmark inference deepspeed llm llamacpp vllm tensorrt llm updated jul 18 2025 jupyter notebook fgblanch outlookllm star 49 code issues pull requests add in for new outlook that adds llm new features composition summarizing q a it uses a local llm via nvidia tensorrt llm outlook addin tensorrt llm updated may 12 2026 python janhq cortex tensorrt llm star 41 code issues pull requests cortex tensorrt llm is a c inference library that can be loaded by any server at runtime it submodules nvidia s tensorrt llm for gpu accelerated inference on nvidia s gpus nvidia jan tensorrt llm tensorrt llm updated sep 26 2024 c zrzrzrzrzrzrzr lm fly sponsor star 25 code issues pull requests 大模型推理框架加速 让 llm 飞起来 mlx tgi openvino llm vllm llm inference tensorrt llm updated may 10 2024 python lix19937 llm deploy star 24 code issues pull requests ai infra llm infer tensorrt llm vllm llm llm inference tensorrt llm updated aug 10 2026 python wcks13589 llm tutorial star 17 code issues pull requests llm tutorial materials include but not limited to nvidia nemo tensorrt llm triton inference server and nemo guardrails nemo nvidia nemo llm nemo guardrails tensorrt llm updated jun 26 2025 python load more add this topic to your repo to associate your repository with the tensorrt llm topic visit your repo s landing page and select manage topics learn more footer 2026 github inc footer navigation terms privacy security status community docs contact manage cookies do not share my personal information you can t perform that action at this time
|