Meta tags:
description= GitHub is where people build software. More than 225 million people use GitHub to discover, fork, and contribute to over 800 million projects.;
Headings (most frequently used words):
distributed, inference, navigation, public, this, topic, footer, flashinfer, ai, gpustack, llm, menu, here, are, 222, repositories, matching, add, to, your, repo, sgl, project, sglang, omni, nehanth, pooled, awslabs, awsome, christopherowen, spark, ds41f, justvugg, lumabri, opencpil, prima, cpp, shootthesound, comfyui, mesh, mzbac, mlx_sharding, adt109119, llamacpp, yuvrajsingh, mist, smolcluster, enapt, swarmllm, ninnix, q36, trademav, ramdeck, core, deen, media, dgx, monarch, wn, clusterchat, dharshaneshwaran, hyperpod, labscommunity, cascadia,
Text of the page (most frequently used words):
inference (44), #distributed (34), llm (28), and (27), code (25), updated (22), star (21), 2026 (20), pull (20), requests (20), issues (20), for (17), github (14), nov (14), local (14), your (11), the (11), python (11), models (9), model (7), gpu (7), llama (7), generation (7), training (7), cpp (6), cuda (6), across (6), comfyui (6), peer (6), with (5), rust (5), dgx (5), openai (5), discussions (5), parallelism (5), serving (5), all (5), you (4), security (4), run (4), gguf (4), metal (4), one (4), rpc (4), javascript (4), open (4), large (4), spark (4), video (4), api (4), computing (4), server (4), pytorch (4), node (4), vllm (4), sglang (4), performance (4), support (4), copilot (4), this (3), learn (3), topic (3), topics (3), silicon (3), machine (3), devices (3), source (3), language (3), browser (3), image (3), nvidia (3), self (3), hosted (3), cluster (3), coding (3), sponsor (3), deepseek (3), kubernetes (3), high (3), flashinfer (3), most (3), stars (3), another (3), tab (3), window (3), refresh (3), session (3), reload (3), sign (3), developers (3), changelog (3), customer (3), enterprise (3), grade (3), features (3), build (3), that (2), manage (2), community (2), status (2), footer (2), navigation (2), more (2), repository (2), repo (2), add (2), intel (2), into (2), runtime (2), free (2), pipeline (2), out (2), hyperpod (2), webgpu (2), p2p (2), chatbot (2), webrtc (2), split (2), diffusion (2), two (2), oct (2), pool (2), public (2), streaming (2), agent (2), compatible (2), qwen (2), apple (2), engine (2), moe (2), vulkan (2), qwen3 (2), alternative (2), anthropic (2), chat (2), computers (2), team (2), over (2), big (2), expert (2), grpo (2), fsdp (2), mac (2), heterogeneous (2), parameter (2), data (2), educational (2), raspberry (2), library (2), gpus (2), aug (2), llamacpp (2), mlx (2), ltx (2), nvenc (2), flux (2), device (2), prima (2), colibri (2), 228 (2), docker (2), shell (2), aws (2), speech (2), interactive (2), gpustack (2), recently (2), fewest (2), forks (2), sort (2), 222 (2), sponsors (2), events (2), trending (2), explore (2), signed (2), universe (2), integrations (2), stories (2), center (2), documentation (2), business (2), fixes (2), products (2), secret (2), quality (2), can, perform, action, time, not, share, personal, information, cookies, contact, docs, privacy, terms, inc, associate, visit, landing, page, select, load, openvino, any, cascadia, labscommunity, telemetry, fault, tolerance, golang, react, pools, consumer, laptops, meta, measures, vram, each, reports, splits, through, rebuilds, when, worker, drops, dharshaneshwaran, its, users, entirely, clusterchat, gb10, sparks, monarch, deen, media, ggml, memory, pooling, homelab, ram, old, pcs, macs, android, unified, ramdeck, core, trademav, ssd, 250, ps5, quantization, amd, 27b, 35b, a3b, q36, ninnix, openclaw, ollama, claude, chatgpt, huggingface, libp2p, candle, decentralized, mcp, app, runs, own, computer, lets, internet, too, account, crypto, swarmllm, enapt, mini, edge, socket, programming, zero, hardware, minis, pis, plain, sockets, smolcluster, yuvrajsingh, mist, 一個基於, 的分佈式, 推理程式, 讓您能夠利用區域網路內的多台電腦協同進行大型語言模型的分佈式推理, electron, 的製作跨平台桌面應用程式操作, adt109119, 2024, 106, mlx_sharding, mzbac, may, flux2, multi, lan, same, compresses, activations, live, wire, icarus, daedalus, back, half, 126, mesh, shootthesound, sep, iclr2026, official, scale, beyond, 131, opencpil, mixture, experts, servers, powered, pure, lumabri, justvugg, roce, reproducible, deployment, tuning, benchmarks, flash, switchless, three, four, fabric, ds41f, christopherowen, physical, generative, parallelcluster, eks, efa, slurm, best, practices, reference, architectures, examples, 498, awsome, awslabs, speculative, decoding, wgsl, agents, 657, pooled, nehanth, audio, multimodal, asr, music, recognition, tts, text, super, intelligence, system, omni, sgl, project, intelligent, routing, cncf, state, art, modern, accelerators, mindie, genai, ascend, rocm, maas, manager, demand, ssh, accessible, instances, attention, jit, kernel, least, options, html, typescript, swift, 111, filter, here, are, repositories, matching, collections, message, dismiss, alert, switched, accounts, appearance, settings, search, pricing, event, talks, workshops, what, next, partners, about, company, forum, webinars, blog, resource, connect, resources, premium, advanced, ons, trust, insights, why, evaluate, see, industries, government, manufacturing, financial, services, healthcare, industry, nonprofits, startups, enterprises, stage, businesses, latest, improvements, place, developer, articles, repos, customize, certifications, scanning, leak, prevention, protection, vulnerability, detection, remediation, secure, cloud, development, environments, codespaces, analysis, enforcement, automation, actions, ship, tasks, projects, automated, work, hosting, review, collaboration, platform, menu, skip, content,
Text of the page (random words):
distributed inference github topics github skip to content navigation menu sign in products platform github repository hosting code review and collaboration copilot ai coding agent for tasks projects and automated work build ship actions ci cd and automation code quality quality analysis fixes and enforcement codespaces cloud development environments secure code security vulnerability detection and remediation secret protection secret scanning and leak prevention all features developers learn documentation training certifications build build with copilot customize copilot integrations explore trending repos github sponsors developer articles changelog changelog your source for the latest features improvements and fixes across github products all in one place businesses by stage enterprises team startups nonprofits by industry healthcare financial services manufacturing government see all industries evaluate github why github customer stories business insights trust center add ons github advanced security enterprise grade code security copilot for business enterprise grade ai features premium support enterprise grade 24 7 support resources learn connect documentation resource center blog events webinars community forum customer stories integrations support customer support training github status company about partners changelog github universe github universe the event for developers by developers talks workshops and what s next for github pricing search sign in sign up appearance settings you signed in with another tab or window reload to refresh your session you signed out in another tab or window reload to refresh your session you switched accounts on another tab or window reload to refresh your session dismiss alert message explore topics trending collections events github sponsors distributed inference star here are 222 public repositories matching this topic language all filter by language all 222 python 111 c 16 rust 15 go 11 shell 10 c 8 javascript 8 swift 7 typescript 7 html 6 sort most stars sort options most stars fewest stars most forks fewest forks recently updated least recently updated flashinfer ai flashinfer star 6 9k code issues pull requests discussions flashinfer kernel library for llm serving gpu cuda jit pytorch nvidia moe attention llm inference large large models distributed inference updated nov 17 2026 cuda gpustack gpustack star 6 1k code issues pull requests discussions a gpu cluster manager for high performance ai model serving vllm sglang and on demand ssh accessible gpu instances cuda inference openai llama maas rocm ascend llm llm serving vllm genai llm inference qwen deepseek sglang distributed inference high performance inference mindie updated nov 17 2026 python llm d llm d star 5 3k code issues pull requests state of the art inference performance with modern accelerators on kubernetes kubernetes ai gpu cncf inference model server llm distributed inference intelligent routing updated nov 17 2026 python sgl project sglang omni star 1 8k code issues pull requests discussions high performance serving runtime for interactive super intelligence system text to speech streaming interactive cuda inference pytorch tts speech recognition music generation asr model serving multimodal openai api audio generation sglang distributed inference updated nov 17 2026 python nehanth pooled star 657 code issues pull requests discussions peer to peer llm inference in the browser pool your devices to run big open models for chat and coding agents browser webrtc peer to peer inference webgpu wgsl llm local llm speculative decoding distributed inference updated nov 3 2026 javascript awslabs awsome distributed ai star 498 code issues pull requests best practices reference architectures and examples for distributed ai training and inference on aws kubernetes aws gpu slurm efa distributed training eks parallelcluster hyperpod generative ai distributed inference physical ai updated nov 14 2026 shell christopherowen spark ds41f star 228 code issues pull requests reproducible docker vllm deployment tuning and benchmarks for deepseek v4 1 flash on a switchless three node or four node dgx spark fabric docker roce vllm deepseek distributed inference dgx spark updated nov 17 2026 python justvugg lumabri star 228 code issues pull requests run large ai models across your computers and servers powered by colibri pure c c peer to peer self hosted colibri cluster computing mixture of experts llm inference local ai distributed inference updated nov 17 2026 c opencpil prima cpp star 131 code issues pull requests official prima cpp scale your local ai beyond one device llm inference distributed ai distributed inference iclr2026 on device llm updated sep 6 2026 c shootthesound comfyui mesh sponsor star 126 code issues pull requests split flux 2 and ltx 2 3 across two gpus lan or same machine nvenc compresses activations live on the wire icarus comfyui node daedalus back half server flux pytorch image generation multi gpu nvenc video generation pipeline parallelism diffusion models flux2 comfyui distributed inference comfyui node ltx video updated may 17 2026 python mzbac mlx_sharding star 106 code issues pull requests distributed inference for mlx llm mlx distributed inference updated aug 1 2024 python adt109119 llamacpp distributed inference star 95 code issues pull requests 一個基於 llama cpp 的分佈式 llm 推理程式 讓您能夠利用區域網路內的多台電腦協同進行大型語言模型的分佈式推理 使用 electron 的製作跨平台桌面應用程式操作 ui rpc llm llamacpp llm inference gguf distributed llm distributed inference updated aug 23 2026 javascript yuvrajsingh mist smolcluster sponsor star 90 code issues pull requests educational distributed training and inference library for heterogeneous local hardware mac minis raspberry pis and gpus over plain python sockets fsdp grpo parameter server expert and data parallelism raspberry pi distributed computing pytorch educational zero socket programming data parallelism parameter server edge computing model parallelism distributed training heterogeneous computing mac mini apple silicon fsdp llm training local ai distributed inference grpo expert parallelism updated oct 7 2026 python enapt swarmllm star 88 code issues pull requests discussions free open source app that runs ai chat models on your own computer and lets computers team up over the internet to run models too big for one machine no account no crypto openai and anthropic compatible api rust ai mcp decentralized p2p inference self hosted candle libp2p huggingface openai api llm chatgpt alternative local llm gguf anthropic api distributed inference claude code ollama alternative openclaw updated nov 17 2026 rust ninnix q36 sponsor star 83 code issues pull requests qwen3 8 27b and qwen3 6 35b a3b local inference engine for vulkan and metal c metal amd vulkan moe quantization inference engine ps5 apple silicon bc 250 local llm llm inference qwen gguf distributed inference openai compatible coding agent ssd streaming updated nov 13 2026 c trademav ramdeck core public star 72 code issues pull requests pool ram across old pcs macs and android devices into a unified local ai cluster self hosted rpc homelab memory pooling llama cpp local llm local ai distributed inference ggml rpc updated oct 26 2026 python deen media dgx monarch star 56 code issues pull requests distributed image and video generation in comfyui for two nvidia dgx sparks nvidia image generation video generation diffusion models comfyui distributed inference gb10 dgx spark updated nov 16 2026 python l wn clusterchat star 56 code issues pull requests a chatbot with a large language model split across its users entirely in the browser open source webrtc chatbot p2p webgpu open models llm distributed inference updated nov 10 2026 javascript dharshaneshwaran hyperpod star 56 code issues pull requests pools consumer laptops into one local runtime for meta llama models measures the free vram each machine reports splits the model across cuda and metal devices through llama cpp rpc and rebuilds the pipeline when a worker drops out react rust golang metal fault tolerance cuda llama llama cpp local llm gguf distributed inference gpu telemetry updated nov 1 2026 go labscommunity cascadia star 52 code issues pull requests run any model on intel silicon intel openvino llm inference local ai distributed inference updated nov 17 2026 rust load more add this topic to your repo to associate your repository with the distributed inference topic visit your repo s landing page and select manage topics learn more footer 2026 github inc footer navigation terms privacy security status community docs contact manage cookies do not share my personal information you can t perform that action at this time
|