Meta tags:
description=Privacy-first LLM proxy and AI gateway - load balancing, multi-provider routing, API key management, usage tracking, rate limiting. Self-hosted. Zero knowledge of your prompts. - voidmind-io/voidllm;

Headings (most frequently used words):

uh, oh, code, voidllm, navigation, saved, searches, files, it, mcp, tools, license, footer, docker, search, repositories, users, issues, pull, requests, provide, feedback, voidmind, io, menu, use, to, filter, your, results, more, quickly, folders, and, latest, commit, history, repository, why, how, works, who, for, quick, start, features, gateway, documentation, configuration, deployment, production, checklist, privacy, cli, project, about, releases, 39, packages, contributors, languages, binary, no, needed, one, click, deploy, built, in, external, servers, mode, ide, setup, known, limitations, compose, kubernetes, helm, from, source, topics, resources, of, conduct, contributing, security, policy, stars, watchers, forks,

Text of the page (most frequently used words):
#voidllm (46), and (44), the (30), api (26), code (24), proxy (24), mcp (24), your (22), #security (18), with (18), for (16), per (16), openai (15), key (15), github (14), docker (14), you (13), usage (13), enterprise (13), yaml (13), servers (13), license (12), llm (12), provider (12), tools (12), model (12), all (12), privacy (11), gateway (11), ollama (10), rate (10), support (10), are (10), this (9), load (9), management (9), knowledge (9), from (9), org (9), use (9), name (9), azure (9), tool (9), reload (8), view (8), first (8), rbac (8), self (8), hosted (8), balancing (8), multi (8), available (8), user (8), only (8), content (8), any (8), limits (8), models (8), openssl (8), rand (8), base64 (8), compose (8), export (8), mode (8), start (8), not (7), vllm (7), anthropic (7), tracking (7), zero (7), voidmind (7), team (7), budgets (7), http (7), deployment (7), providers (7), access (7), control (7), teams (7), that (6), there (6), was (6), error (6), loading (6), page (6), contributing (6), routing (6), prompts (6), set (6), voidllm_admin_key (6), voidllm_encryption_key (6), config (6), settings (6), alias (6), 8080 (6), external (6), one (6), session (6), keys (6), navigation (5), while (5), please (5), latest (5), custom (5), helm (5), source (5), who (5), how (5), cost (5), request (5), see (5), admin (5), run (5), deployments (5), chart (5), priority (5), aliases (5), failover (5), default (5), issues (5), documentation (5), across (5), requests (5), features (5), email (5), open (5), app (5), search (5), can (4), time (4), community (4), releases (4), conduct (4), readme (4), limiting (4), kubernetes (4), built (4), changelog (4), sqlite (4), data (4), database (4), tokens (4), prompt (4), logs (4), response (4), multiple (4), redis (4), secrets (4), postgresql (4), base_url (4), never (4), supported (4), https (4), url (4), pricing (4), endpoint (4), architecture (4), wasm (4), auto (4), type (4), execution (4), token (4), automatic (4), health (4), every (4), amd64 (4), platform (4), need (4), more (4), yml (4), actions (4), quality (4), docs (3), status (3), dockerfile (3), stars (3), policy (3), resources (3), topics (3), services (3), business (3), host (3), through (3), memory (3), toggle (3), full (3), list (3), prometheus (3), metrics (3), production (3), bootstrap (3), instead (3), cmd (3), example (3), aws (3), com (3), timeout (3), localhost (3), single (3), configuration (3), sso (3), ide (3), instance (3), oauth (3), slack (3), bearer (3), authorization (3), description (3), calls (3), call (3), proxies (3), level (3), everything (3), latency (3), routes (3), download (3), compatible (3), sdk (3), log (3), credentials (3), linux (3), arm64 (3), binary (3), file (3), metadata (3), files (3), dev (3), commit (3), insights (3), pull (3), signed (3), another (3), tab (3), window (3), refresh (3), sign (3), saved (3), feedback (3), grade (3), copilot (3), solutions (3), explore (3), action (2), share (2), personal (2), manage (2), footer (2), 2026 (2), javascript (2), typescript (2), report (2), repository (2), forks (2), 122 (2), project (2), claude (2), each (2), vulnerabilities (2), compliance (2), what (2), events (2), passes (2), persistent (2), storage (2), body (2), feature (2), guide (2), replicas (2), they (2), network (2), public (2), tls (2), port (2), npm (2), build (2), 11434 (2), syntax (2), true (2), enabled (2), code_mode (2), global (2), api_key (2), gpt (2), round (2), robin (2), 30s (2), audit (2), roles (2), setup (2), circuit (2), breakers (2), quick (2), topic (2), blog (2), runtime (2), google (2), yet (2), work (2), upstream (2), sse (2), list_models (2), get_usage (2), specific (2), via (2), generated (2), llms (2), execute_code (2), find (2), counts (2), exposes (2), write (2), sandboxed (2), scoped (2), details (2), create (2), infrastructure (2), direct (2), oidc (2), users (2), organization (2), pro (2), plus (2), organizations (2), daily (2), budget (2), reports (2), above (2), clients (2), json (2), monthly (2), chat (2), completions (2), works (2), out (2), just (2), change (2), curl (2), application (2), vl_uk_ (2), apps (2), gives (2), password (2), these (2), shown (2), them (2), prints (2), stdout (2), tar (2), needed (2), required (2), add (2), want (2), design (2), right (2), fit (2), which (2), gate (2), router (2), why (2), take (2), seriously (2), sum (2), mod (2), codecov (2), artifacthub (2), repo (2), code_of_conduct (2), gitignore (2), dockerignore (2), air (2), toml (2), bench (2), scripts (2), pkg (2), internal (2), 175 (2), commits (2), last (2), message (2), menu (2), discussions (2), appearance (2), cancel (2), searches (2), repositories (2), advanced (2), developer (2), customer (2), devops (2), merge (2), perform, information, cookies, contact, terms, inc, css, template, languages, contributors, packages, jul, watching, watchers, properties, activity, golang, about, significant, assistance, hosting, permitted, competing, prohibited, converts, apache, four, years, after, release, privately, migrate, postgres, pass, verify, jwt, bidirectional, migration, cli, designed, gdpr, stored, processed, option, doesn, exist, enable_content_logging, contain, much, caching, architectural, decision, makes, hardening, shared, process, without, configure, backups, scrape, policies, resource, don, strong, bytes, keep, manager, traffic, separate, isolate, put, behind, reverse, checklist, prerequisites, node, optional, subcharts, install, adminkey, encryptionkey, environment, variables, interpolated, hardcoded, var, encryption_key, admin_key, none, auth_type, mcp_servers, openai_key, fallback, azure_deployment, azure_east_key, eastus, east, smart, strategy, balanced, output_per_1m, input_per_1m, dolphin, mistral, server, common, troubleshooting, otel, endpoints, codes, reference, permissions, overview, strategies, getting, started, faq, pool, pod, require, coming, soon, requiring, jira, header, auth, using, deprecated, protocol, pre, 2025, spec, detected, deactivated, streamable, transport, known, limitations, connects, cursor, windsurf, etc, vl_uk_your_key, headers, mcpservers, admins, block, blocklist, declarations, schemas, included, argument, types, await, toolname, args, keyword, search_tools, discover, list_servers, three, max_tool_calls, memory_limit_mb, concurrent, runtimes, pool_size, lets, orchestrates, turn, runs, quickjs, filesystem, reduces, register, system_admin, list_deployments, temporary, create_key, visible, list_keys, stats, get_model_health, flat, fees, charges, current, future, lifetime, product, advisory, board, founder, early, limited, spots, founding, member, 999, dedicated, 24h, backed, otlp, grpc, correlation, opentelemetry, filterable, groups, mapped, group, sync, created, allowed, domains, provisioning, gets, its, own, identity, okta, keycloak, 149, 1490, 48h, cross, analytics, breakdown, trends, alerts, limit, unlimited, orgs, 490, graceful, shutdown, active, streams, orchestration, where, csv, duration, ttft, real, enforcement, minute, day, most, restrictive, wins, levels, hierarchy, dashboard, playground, web, retry, 5xx, aware, least, weighted, embeddings, images, audio, streaming, box, base, messages, role, hello, point, railway, adding, click, deploy, proxying, used, once, save, complete, copy, now, vl_uk_a3f2, local, random, windows, macos, below, mount, declare, voidllm_data, ghcr, boots, sensible, defaults, generate, today, logging, observability, saas, cannot, responses, reasons, managed, plane, good, speak, authenticates, applies, resolves, called, written, persisted, flowchart, touches, disk, existing, goes, down, breaks, anywhere, switching, means, changing, enforced, runaway, script, burns, estimation, visibility, into, spending, virtual, scoping, raw, solves, problem, stores, persists, setting, tracked, made, many, long, took, stays, yours, screenshots, sits, between, applications, wide, sub, 2ms, overhead, items, history, date, folders, tags, branches, main, additional, options, star, fork, must, notification, notifications, dismiss, alert, switched, accounts, resetting, focus, qualifiers, our, query, filter, results, quickly, submit, include, address, contacted, read, piece, input, very, provide, tips, clear, jump, premium, ons, powered, collections, trending, archive, program, accelerator, maintainer, lab, programs, fund, developers, sponsors, partners, trust, center, forum, skills, ebooks, webinars, stories, software, development, industries, government, manufacturing, financial, healthcare, industry, cases, devsecops, modernization, case, nonprofits, startups, small, medium, enterprises, company, size, marketplace, stop, leaks, before, secret, protection, secure, fix, enforce, changes, review, plan, track, instant, environments, codespaces, automate, workflow, workflows, integrate, registry, new, agents, issue, better, creation, skip,

Text of the page (random words):
tion security github advanced security find and fix vulnerabilities code security secure your code as you build secret protection stop leaks before they start explore why github documentation blog changelog marketplace view all features solutions by company size enterprises small and medium teams startups nonprofits by use case app modernization devsecops devops ci cd view all use cases by industry healthcare financial services manufacturing government view all industries view all solutions resources explore by topic ai software development devops security view all topics explore by type customer stories events webinars ebooks reports business insights github skills support services documentation customer support community forum trust center partners view all resources open source community github sponsors fund open source developers programs security lab maintainer community accelerator github stars archive program repositories topics trending collections enterprise enterprise solutions enterprise platform ai powered developer platform available add ons github advanced security enterprise grade security features copilot for business enterprise grade ai features premium support enterprise grade 24 7 support pricing search or jump to search code repositories users issues pull requests search clear search syntax tips provide feedback we read every piece of feedback and take your input very seriously include my email address so i can be contacted cancel submit feedback saved searches use saved searches to filter your results more quickly name query to see all available qualifiers see our documentation cancel create saved search sign in sign up appearance settings resetting focus you signed in with another tab or window reload to refresh your session you signed out in another tab or window reload to refresh your session you switched accounts on another tab or window reload to refresh your session dismiss alert message uh oh there was an error while loading please reload this page voidmind io voidllm public notifications you must be signed in to change notification settings fork 14 star 122 code issues 1 pull requests 0 discussions actions security and quality 0 insights additional navigation options code issues pull requests discussions actions security and quality insights voidmind io voidllm main branches tags go to file code open more actions menu folders and files name name last commit message last commit date latest commit history 175 commits 175 commits github github chart voidllm chart voidllm cmd voidllm cmd voidllm docs docs internal internal pkg pkg scripts bench scripts bench ui ui air toml air toml dockerignore dockerignore gitignore gitignore changelog md changelog md code_of_conduct md code_of_conduct md contributing md contributing md dockerfile dockerfile license license readme md readme md security md security md artifacthub repo yml artifacthub repo yml codecov yml codecov yml docker compose dev yaml docker compose dev yaml docker compose enterprise yaml docker compose enterprise yaml docker compose yaml docker compose yaml go mod go mod go sum go sum voidllm yaml example voidllm yaml example view all files repository files navigation readme code of conduct contributing license security more items voidllm a privacy first llm proxy and ai gateway for teams that take control seriously voidllm is a self hosted llm proxy that sits between your applications and llm providers openai anthropic azure ollama vllm or any custom endpoint it gives you organization wide access control api key management usage tracking rate limiting and multi deployment load balancing one go binary sub 2ms proxy overhead zero knowledge of your prompts more screenshots privacy first by design voidllm is a zero knowledge llm proxy it never stores logs or persists any prompt or response content not as a setting you can toggle by architecture only metadata is tracked who made the request which model how many tokens how long it took your data stays yours why voidllm problem how voidllm solves it teams share raw api keys in slack virtual keys with org team user scoping and rbac no visibility into who s spending what per key per team per org usage tracking cost estimation one runaway script burns the monthly budget rate limits token budgets enforced by the proxy at every level switching providers means changing every app model aliases clients call default the proxy routes it anywhere provider goes down everything breaks multi deployment load balancing with automatic failover existing proxies log your prompts zero knowledge proxy architecture content never touches disk how it works flowchart lr app your app sdk openai compatible proxy voidllm proxy proxy gate api key rbac br rate limits budgets gate router model alias load balancing router providers openai anthropic azure br ollama vllm custom proxy metadata only db sqlite postgresql loading your apps speak the openai api to voidllm it authenticates the key applies rbac rate limits and budgets resolves the model alias and routes to the right provider only metadata who called which model token counts latency is written to the database prompt and response content passes through memory and is never persisted who it s for a good fit if you self host llm infrastructure vllm ollama or use managed providers and need one control plane cannot log prompts or responses for privacy or compliance reasons need org team user key rbac budgets and model routing run multiple providers and want aliases load balancing and failover not the right fit if you want a hosted saas gateway voidllm is self hosted by design need full prompt response logging or content level observability it s zero knowledge by architecture need upstream mcp servers with per user oauth today not yet supported quick start generate required keys export voidllm_admin_key openssl rand base64 32 export voidllm_encryption_key openssl rand base64 32 start the proxy no config file needed voidllm boots with sensible defaults docker run p 8080 8080 e voidllm_admin_key e voidllm_encryption_key v voidllm_data data ghcr io voidmind io voidllm latest on first start voidllm prints bootstrap credentials to stdout shown below no config file is required add models in the ui or mount a voidllm yaml to declare them see configuration binary no docker needed download the latest binary for your platform from the releases page linux curl sl https github com voidmind io voidllm releases latest download voidllm linux amd64 tar gz tar xz export voidllm_admin_key openssl rand base64 32 export voidllm_encryption_key openssl rand base64 32 voidllm available for linux amd64 arm64 windows amd64 arm64 macos amd64 arm64 on first start voidllm prints your credentials to stdout bootstrap complete copy these now api key vl_uk_a3f2 email admin voidllm local password random open http localhost 8080 log in with the email and password above and start proxying the api key is used for sdk calls authorization bearer vl_uk_ these credentials are shown once save them one click deploy keys are auto generated open the url railway gives you and start adding models your apps just point at the proxy instead of the provider curl http localhost 8080 v1 chat completions h authorization bearer vl_uk_ h content type application json d model default messages role user content hello any openai compatible sdk works out of the box just change the base url to your voidllm proxy features feature details openai compatible proxy v1 chat completions embeddings images audio streaming multi provider routing openai anthropic azure ollama vllm any custom endpoint load balancing round robin least latency weighted priority across deployments automatic failover retry on 5xx timeout circuit breakers health aware routing web ui dashboard playground api keys teams models usage settings rbac org team user key hierarchy 4 roles rate limits requests per minute day most restrictive wins across levels token budgets daily monthly limits real time enforcement usage tracking tokens cost duration ttft per request usage export csv json download model aliases clients call default you control where it routes mcp gateway proxy external mcp servers with access control and session management code mode wasm sandboxed js for multi tool orchestration prometheus metrics latency tokens active streams routing health database sqlite default or postgresql deployment docker helm chart graceful shutdown pro 49 mo 490 yr everything above plus unlimited orgs teams no limit on organizations or teams cost reports model breakdown daily trends budget alerts cross org analytics usage and cost across all organizations support priority email 48h enterprise 149 mo 1490 yr everything in pro plus sso oidc google azure ad okta keycloak any provider per org sso each organization gets its own identity provider auto provisioning users created from allowed email domains group sync oidc groups mapped to voidllm teams audit logs every admin action filterable api ui opentelemetry otlp grpc export request id correlation multi instance redis backed rate limits and budgets across replicas support dedicated slack 24h founding member 999 one time all enterprise features current and future lifetime license product advisory board direct founder access priority support early access limited spots flat pricing no per user fees no per request charges self hosted on your infrastructure mcp gateway voidllm is an mcp gateway it exposes built in management tools and proxies requests to external mcp servers with access control usage tracking and automatic session management built in tools tool description list_models list models with health status rbac scoped get_model_health health status for a specific model or deployment get_usage usage stats for your key team org list_keys api keys visible to you create_key create a temporary api key list_deployments deployment details system_admin only external mcp servers register external mcp servers via the admin ui or api voidllm proxies tool calls through api v1 mcp alias with scoped access control global org or team level automatic session management usage tracking and prometheus metrics code mode code mode lets llms write javascript that orchestrates multiple mcp tool calls in a single execution instead of one tool call per llm turn the js runs in a wasm sandboxed quickjs runtime with no filesystem no network and no host access reduces token usage by 30 80 mcp code_mode enabled true pool_size 8 concurrent wasm runtimes memory_limit_mb 16 per execution timeout 30s per execution max_tool_calls 50 per execution code mode exposes three tools on api v1 mcp tool description list_servers discover available mcp servers and tool counts search_tools find tools by keyword across all servers execute_code run js with mcp tools as await tools alias toolname args typescript type declarations are auto generated from tool schemas and included in the execute_code description so llms see available tools and argument types at tools list time admins can block specific tools from code mode via the per tool blocklist api and ui ide setup mcpservers voidllm type http url http your voidllm instance 8080 api v1 mcp headers authorization bearer vl_uk_your_key this connects your ide claude code cursor windsurf to the code mode endpoint management tools list_models get_usage etc are available at api v1 mcp voidllm external mcp servers at api v1 mcp alias known limitations sse transport not supported mcp servers using the deprecated sse protocol pre 2025 03 26 spec are auto detected and deactivated use servers that support streamable http no oauth for upstream mcp servers servers requiring per user oauth jira slack google are not yet supported api key and header auth work single instance only code mode s wasm runtime pool is in memory multi pod deployments require redis support coming soon documentation full documentation blog faq topic guide getting started quick start configuration all yaml settings docker docker deployment kubernetes helm chart providers openai anthropic azure ollama vllm load balancing strategies failover circuit breakers mcp gateway overview servers code mode ide setup rbac roles and permissions privacy zero knowledge architecture api reference endpoints and error codes enterprise license sso audit otel pricing troubleshooting common issues configuration server proxy port 8080 models single endpoint name dolphin mistral provider ollama base_url http localhost 11434 v1 timeout 30s aliases default pricing input_per_1m 0 15 output_per_1m 0 60 load balanced multiple deployments with failover name gpt 4o strategy round robin aliases smart deployments name azure east provider azure base_url https eastus openai azure com api_key azure_east_key azure_deployment gpt 4o priority 1 name openai fallback provider openai base_url https api openai com v1 api_key openai_key priority 2 mcp_servers name aws knowledge alias aws url https knowledge mcp global api aws auth_type none settings admin_key voidllm_admin_key encryption_key voidllm_encryption_key mcp code_mode enabled true supported providers openai anthropic azure vllm ollama custom environment variables are interpolated with var syntax secrets never hardcoded deployment docker compose cp voidllm yaml example voidllm yaml export voidllm_admin_key openssl rand base64 32 export voidllm_encryption_key openssl rand base64 32 docker compose up kubernetes helm helm install voidllm chart voidllm set secrets adminkey openssl rand base64 32 set secrets encryptionkey openssl rand base64 32 set config models 0 name my model set config models 0 provider ollama set config models 0 base_url http ollama 11434 v1 postgresql and redis are available as optional subcharts for production deployments from source prerequisites go 1 23 node 20 cd ui npm ci npm run build cd go run cmd voidllm config voidllm yaml production checklist use postgresql instead of sqlite put voidllm behind tls a reverse proxy isolate the admin ui api from public proxy traffic separate admin port with tls use a strong voidllm_encryption_key 32 bytes and keep it in a secrets manager don t use voidllm_admin_key as a production api key it s for bootstrap only set resource limits and network policies in kubernetes scrape metrics with prometheus configure database backups for multiple replicas use redis so rate limits and budgets are shared they are per process without it enterprise see the security hardening guide for the full list privacy this is not a feature toggle it s an architectural decision that makes voidllm a privacy first llm proxy no request body in logs db or any persistent storage no response body in logs db or any persistent storage no prompt caching content passes through memory only usage events contain only who key org team what model how much tokens cost there is no enable_content_logging option it doesn t exist designed to support gdpr compliance no personal data in prompts is stored or processed cli tools bidirectional...