Meta tags:

Headings (most frequently used words):

tools, custom, llm, calling, tool, with, vapi, and, it, your, response, generation, code, without, assistant, native, attached, transfercall, processorder, integration, what, is, why, use, setting, up, for, adding, testing, curl, integrating, conclusion, complete, how, works, sample, snippet, attaching, to, an, existing, in, get_payment_link, only, including, appointment, scheduling,

Text of the page (most frequently used words):
#custom (61), the (43), tools (42), call (37), tool (37), llm (33), your (29), vapi (28), with (25), for (23), function (20), const (20), chat (19), response (19), and (18), quickstart (17), assistant (16), model (16), completions (15), name (15), content (15), calling (15), json (14), payload (14), openai (14), url (13), processorder (13), integration (12), type (12), transfercall (12), messages (12), res (12), code (11), server (11), calls (11), that (10), curl (10), https (10), api (10), streaming (10), native (9), result (9), request (9), data (9), support (9), can (8), testing (8), transfer (8), generation (8), works (8), destination (8), this (7), here (7), provider (7), user (7), application (7), temperature (7), toolcall (7), how (7), chunk (7), toolcallinfo (7), requestargs (7), func (7), voice (7), management (7), page (6), you (6), like (6), attached (6), transcriber (6), number (6), conversation (6), role (6), insert (6), configuration (6), arguments (6), logevent (6), req (6), back (6), await (6), overview (6), assistants (5), scheduling (5), payment (5), language (5), azure (5), continue (5), gpt (5), without (5), post (5), get_payment_link (5), snippet (5), stringify (5), write (5), let (5), any (5), setting (5), speech (5), web (5), phone (5), built (4), complete (4), use (4), link (4), into (4), guide (4), message (4), ask (4), they (4), want (4), trigger (4), not (4), patch (4), are (4), these (4), body (4), endpoint (4), detection (4), argumentsstr (4), stream (4), toolcalls (4), text (4), what (4), llms (4), voices (4), examples (4), multilingual (4), sms (4), sip (4), documentation (4), cli (4), new (4), appointment (3), fine (3), transferring (3), responses (3), description (3), async (3), task (3), system (3), authorization (3), bearer (3), private (3), key (3), only (3), locally (3), toolcallid (3), expected (3), toolcalllist (3), chunks (3), endpoints (3), return (3), process (3), sent (3), includes (3), example (3), functioncallpayload (3), pre (3), null (3), parsedargs (3), choice (3), delta (3), openairesponse (3), variables (3), external (3), adding (3), end (3), setheader (3), control (3), events (3), functions (3), why (3), compliance (3), plan (3), authentication (3), enterprise (3), google (3), models (3), agents (3), rejection (3), triage (3), handoff (3), squads (3), widget (3), troubleshoot (3), errors (3), hooks (3), inbound (3), step (2), before (2), processing (2), generate (2), conversational (2), helper (2), conclusion (2), named (2), false (2), customer (2), also (2), asks (2), including (2), chatting (2), integrating (2), customllm (2), always (2), 12345 (2), results (2), immediately (2), specified (2), parameters (2), returned (2), get (2), set (2), try (2), hardcodedresult (2), hardcoded (2), vapipayload (2), received (2), app (2), dedicated (2), special (2), when (2), via (2), followupchunk (2), followupresponse (2), auto (2), tool_choice (2), true (2), functionmessage (2), create (2), tool_functions (2), err (2), parse (2), tool_calls (2), finishreason (2), choices (2), asynciterable (2), unknown (2), string (2), information (2), actions (2), just (2), attaching (2), existing (2), cache (2), modeltools (2), optionally (2), sample (2), reply (2), structured (2), dynamic (2), dashboard (2), status (2), website (2), consent (2), static (2), recording (2), deepgram (2), cartesia (2), playht (2), elevenlabs (2), core (2), faq (2), ivr (2), navigation (2), environments (2), dev (2), uat (2), prod (2), debugging (2), prompting (2), boards (2), advanced (2), developing (2), spam (2), urls (2), webhooks (2), property (2), order (2), clinic (2), passing (2), between (2), lead (2), qualification (2), migrate (2), variable (2), substitution (2), session (2), compatibility (2), non (2), reasons (2), forwarding (2), voicemail (2), outbound (2), sdk (2), telnyx (2), twilio (2), telephony (2), free (2), agent (2), bring (2), own (2), tuned (2), tts (2), keywords (2), knowledge (2), fallback (2), background (2), configurations (2), composer (2), guides (2), introduction (2), integrations (2), release (2), sdks (2), mcp (2), reference (2), index (2), level (2), txt (2), next, build, receptionist, books, reschedules, cancels, appointments, using, previous, edit, yes, was, helpful, convenience, find, source, building, each, layer, tune, rolling, out, production, orders, leverage, fetching, everyday, turns, basic, interactive, according, service, department, numbere164checkenabled, xxxxxx, destinations, agree, about, after, integrate, choose, fits, needs, customtools, param, value, returns, instructs, 555, 1234, please, eventually, include, need, once, them, commands, returning, customtool, expect, have, array, list, requests, unique, handled, skip, further, handling, function_call, functionname, sends, appears, isn, executed, come, failed, error, console, catch, finish_reason, length, accumulate, packaged, follow, streamed, parses, execution, right, now, got, working, expand, abilities, three, different, ways, update, done, keep, alive, connection, event, headers, successful, ourtools, merge, definitions, prepare, log, incoming, generated, sse, builds, context, gets, reception, start, basics, simply, mode, receives, details, streams, lets, plug, quickly, easily, seamless, instructions, running, needed, interactions, combine, attach, even, add, flexibility, mixes, natural, actionable, enhanced, functionality, more, than, generator, processes, handle, logic, all, while, users, think, smart, answers, questions, but, takes, copy, scroll, top, light, tcpa, guidelines, addresses, proxy, pci, retrieve, artifacts, zero, retention, zdr, hipaa, gdpr, jwt, single, sign, sso, flow, rss, feed, glossary, report, issues, sympana, connector, voiceaiwrapper, voicerr, vapify, chatdash, voiceflow, langfuse, supabase, cloudflare, gcp, cloud, storage, aws, assemblyai, talkscriber, speechmatics, gladia, openrouter, togetherai, perplexity, deepinfra, groq, gemini, anthropic, bedrock, inworld, rimeai, minimax, lmnt, legacy, migration, test, suites, silent, handoffs, routing, commerce, medical, analysis, concurrency, queue, debug, drops, hold, specialist, transfers, based, warm, join, timeout, real, time, websocket, transport, trunk, credential, amazon, chime, plivo, zadarma, networking, firewall, trunking, escalation, argument, encryption, base, query, retrieval, gohighlevel, slack, sheets, calendar, troubleshooting, aliases, client, side, keys, realtime, pipeline, email, address, reading, pronunciation, dictionaries, denoising, idle, flush, syntax, formatting, personalization, transient, permanent, legal, security, privacy, community, resources, best, practices, monitoring, scorecard, outputs, simulations, evals, observability, workflows, campaigns, insights, place, numbers, behavior, concepts, started, search, available, root, append, markdown, version,

Text of the page (random words):
custom keywords custom voices custom transcriber custom tts custom llms fine tuned openai models bring your own server tool calling integration examples phone numbers free vapi number inbound sms telephony integrations sip integration phone number hooks calls place calls in call control call end reasons troubleshoot call errors call insights outbound campaigns quickstart overview chat quickstart streaming non streaming openai compatibility session management variable substitution sms chat web widget workflows migrate to squads quickstart overview examples squads quickstart overview handoff tool passing data between assistants examples webhooks setting server urls server events spam call rejection developing locally server authentication observability evals simulations pre release boards structured outputs scorecard monitoring best practices prompting guide debugging voice agents enterprise environments dev uat prod ivr navigation testing resources faq how vapi works integrations community support security and privacy legal introduction phone calls web calls vapi guides composer cli quickstart quickstart transient vs permanent configurations variables multilingual support personalization with user information voice formatting plan flush syntax background messages idle messages assistant hooks background speech denoising pronunciation dictionaries email address reading speech configuration voice pipeline configuration voice fallback configuration transcriber fallback configuration openai realtime speech to speech provider keys tools built in call tools voicemail tool custom tools code tool client side tools web sdk tool rejection plan static variables and aliases custom tools troubleshooting google calendar google sheets slack gohighlevel knowledge retrieval query tool custom knowledge base argument encryption custom keywords custom voices elevenlabs playht cartesia custom transcriber custom tts fine tuned openai models bring your own server tool calling integration appointment scheduling lead qualification inbound support voice widget documentation agent support escalation multilingual agent free vapi number inbound sms twilio telnyx sip telephony sip trunking networking and firewall twilio telnyx zadarma plivo amazon chime sdk troubleshoot sip trunk credential errors phone number hooks calls outbound calls websocket transport real time call control customer join timeout voicemail detection call forwarding assistant based warm transfer dynamic call transfers on hold specialist transfer debug forwarding drops call queue management call concurrency call end reasons troubleshoot call errors call analysis call recording quickstart overview quickstart streaming non streaming openai compatibility session management variable substitution sms chat web widget migrate to squads quickstart overview appointment scheduling lead qualification medical triage order management property management multilingual support quickstart overview handoff tool passing data between assistants clinic triage scheduling handoff tool clinic triage scheduling e commerce order management property management routing multilingual support silent handoffs webhooks setting server urls server events spam call rejection developing locally server authentication quickstart advanced quickstart advanced boards quickstart examples quickstart quickstart prompting guide debugging voice agents enterprise environments dev uat prod ivr navigation test suites chat testing voice testing faq core models voices text to speech vapi voices legacy migration elevenlabs playht azure openai cartesia lmnt minimax rimeai deepgram inworld openai azure openai anthropic bedrock gemini groq deepinfra perplexity togetherai openrouter deepgram google gladia speechmatics talkscriber assemblyai aws s3 gcp cloud storage cloudflare r2 supabase langfuse voiceflow chatdash vapify voicerr ai voiceaiwrapper sympana connector support how to report issues enterprise glossary rss feed data flow single sign on sso jwt authentication recording consent plan gdpr compliance hipaa compliance zero data retention zdr retrieve call artifacts pci compliance proxy server guide static ip addresses tcpa consent guidelines website status support dashboard light on this page what is a custom llm and why use it setting up your custom llm for response generation how it works sample code snippet attaching custom llm without tools to an existing assistant in vapi adding tools calling with your custom llm a native llm tools b vapi attached tools c custom tools testing tool calling with curl a native tool calling get_payment_link b vapi attached tool calling transfercall c custom tool calling processorder integrating tools with vapi a without tools response generation only b with tools including transfercall and processorder conclusion complete code scroll to top assistants custom llms custom llm tool calling integration copy page what is a custom llm and why use it a custom llm is more than just a text generator it s a conversational assistant that can call external functions trigger processes and handle special logic all while chatting with your users think of it as your smart helper that not only answers questions but also takes actions why use a custom llm enhanced functionality it mixes natural language responses with actionable functions flexibility you can combine built in functions attach external tools via vapi or even add custom endpoints dynamic interactions the assistant can return structured instructions like transferring a call or running a custom process when needed seamless integration vapi lets you plug these custom endpoints into your assistant quickly and easily setting up your custom llm for response generation before adding tool calls let s start with the basics setting up your custom llm to simply generate conversation responses in this mode your llm receives conversation details asks the model for a reply and streams that text back how it works request reception your endpoint e g chat completions gets a payload with the model messages temperature and optionally tools content generation the code builds an openai api request that includes the conversation context response streaming the generated reply is sent back as server sent events sse sample code snippet 1 app post chat completions async req request res response 2 log the incoming request 3 logevent request received at chat completions req body 4 const payload req body 5 6 prepare the api request to openai 7 const requestargs any 8 model payload model 9 messages payload messages 10 temperature payload temperature 1 0 11 stream true 12 tools payload tools 13 tool_choice auto 14 15 16 optionally merge in native tool definitions 17 const modeltools payload tools 18 requestargs tools modeltools ourtools 19 20 logevent calling openai api for content generation 21 const openairesponse await openai chat completions create requestargs 22 logevent openai api call successful streaming response 23 24 set up streaming headers 25 res setheader content type text event stream 26 res setheader cache control no cache 27 res setheader connection keep alive 28 29 stream the response chunks back 30 for await const chunk of openairesponse as unknown as asynciterable any 31 res write data json stringify chunk n n 32 33 res write data done n n 34 res end 35 attaching custom llm without tools to an existing assistant in vapi if you just want response generation without tool calls update your vapi model with a patch request like this curl x patch https api vapi ai assistant insert your assistant id here h authorization bearer insert your private key here h content type application json d model provider custom llm model gpt 4o url https custom llm url chat completions messages role system content task ask the user if they want to transfer the call if not continue the conversation transcriber provider azure language en ca adding tools calling with your custom llm now that you ve got response generation working let s expand your assistant s abilities your custom llm can trigger external actions in three different ways a native llm tools these tools are built right into your llm integration for example a native function like get_payment_link can return a payment url how it works detection the llm s streaming response includes a tool call for get_payment_link execution the integration parses the arguments and calls the native function response the result is packaged into a follow up api call and streamed back code snippet 1 variables to accumulate tool call information 2 let argumentsstr 3 let toolcallinfo name string id string null null 4 5 process streaming chunks 6 for await const chunk of openairesponse as unknown as asynciterable any 7 const choice chunk choices chunk choices 0 8 const delta choice delta 9 const toolcalls delta tool_calls 10 11 if toolcalls toolcalls length 0 12 for const toolcall of toolcalls 13 const func toolcall function 14 if func func name 15 toolcallinfo name func name id toolcall id 16 17 if func func arguments 18 argumentsstr func arguments 19 20 21 22 23 const finishreason choice finish_reason 24 if finishreason tool_calls toolcallinfo 25 let parsedargs 26 try 27 parsedargs json parse argumentsstr 28 catch err 29 console error failed to parse arguments err 30 31 if tool_functions toolcallinfo name 32 const result await tool_functions toolcallinfo name parsedargs 33 const functionmessage 34 role function 35 name toolcallinfo name 36 content json stringify result 37 38 39 const followupresponse await openai chat completions create 40 model requestargs model 41 messages requestargs messages functionmessage 42 temperature requestargs temperature 43 stream true 44 tools requestargs tools 45 tool_choice auto 46 47 48 for await const followupchunk of followupresponse 49 res write data json stringify followupchunk n n 50 51 argumentsstr 52 toolcallinfo null 53 continue 54 55 56 res write data json stringify chunk n n 57 b vapi attached tools these tools come pre attached via your vapi configuration for example the transfercall tool how it works detection when a tool call for transfercall appears with a destination in the payload the function isn t executed response the integration immediately sends a function call payload with the destination back to vapi code snippet 1 if functionname transfercall payload destination 2 const functioncallpayload 3 function_call 4 name transfercall 5 arguments 6 destination payload destination 7 8 9 10 logevent special handling for transfercall functioncallpayload 11 res write data json stringify functioncallpayload n n 12 skip further processing for this chunk 13 continue 14 c custom tools custom tools are unique to your application and are handled by a dedicated endpoint for example a custom function named processorder how it works dedicated endpoint requests for custom tools go to chat completions custom tool detection the payload includes a tool call list if the function name is processorder a hardcoded result is returned response a json response is sent back with the result code snippet custom endpoint 1 app post chat completions custom tool async req request res response 2 logevent received request at chat completions custom tool req body 3 expect the payload to have a message with a toolcalllist array 4 const vapipayload req body message 5 6 process tool call 7 for const toolcall of vapipayload toolcalllist 8 if toolcall function name processorder 9 const hardcodedresult customtool processorder with customllm always works 10 logevent returning hardcoded result for processorder toolcallid toolcall id 11 return res json 12 results 13 14 toolcallid toolcall id 15 result hardcodedresult 16 17 18 19 20 21 testing tool calling with curl once your endpoints are set up try testing them with these curl commands a native tool calling get_payment_link curl x post https custom llm url chat completions h content type application json d model gpt 4o messages role user content i need a payment link temperature 0 7 tools type function function name get_payment_link description get a payment link parameters expected response streaming chunks eventually include the result e g a payment link returned by the native tool function b vapi attached tool calling transfercall curl x post https custom llm url chat completions h content type application json d model gpt 4o messages role user content please transfer my call temperature 0 7 tools type function function name transfercall description transfer call to a specified destination parameters destination 555 1234 expected response immediately returns a function call payload that instructs vapi to transfer the call to the specified destination c custom tool calling processorder curl x post https custom llm url chat completions custom tool h content type application json d message toolcalllist id 12345 function name processorder arguments param value expected response 1 2 results 3 4 toolcallid 12345 5 result customtools with customllm always works 6 7 8 integrating tools with vapi after testing locally integrate your custom llm with vapi choose the configuration that fits your needs a without tools response generation only curl x patch https api vapi ai assistant insert your assistant id here h authorization bearer insert your private key here h content type application json d model provider custom llm model gpt 4o url https custom llm url chat completions messages role system content task ask the user if they want to transfer the call if not continue chatting transcriber provider azure language en ca b with tools including transfercall and processorder curl x patch https api vapi ai assistant insert your assistant id here h authorization bearer insert your private key here h content type application json d model provider custom llm model gpt 4o url https custom llm url chat completions messages role system content task ask the user if they want to transfer the call if they agree trigger the transfercall tool if not continue the conversation also if the user asks about the custom function processorder trigger that tool tools type transfercall destinations type number number xxxxxx numbere164checkenabled false message transferring call to customer service department type function async false function name processorder description it s a custom tool function named processorder according to vapi ai custom tools guide server url https custom llm url chat completions custom tool transcriber provider azure language en ca conclusion a custom llm turns a basic conversational assistant into an interactive helper that can generate everyday language responses call native tools like fetching a payment link use vapi attached tools like transferring a call and leverage custom tools like processing orders by building each layer step by step and testing with curl you can fine tune your integration before rolling it out in production complete code for your convenience you can find the complet...