Meta tags:

Headings (most frequently used words):

with, gemma, run, agents, on, and, the, models, cloud, stay, organized, collections, save, categorize, content, based, your, preferences, before, you, begin, deploy, model, vllm, container, test, deployed, service, curl, set, concurrency, for, optimal, performance, build, ai, agent, development, kit, using, clean, up, what, next, products, pricing, support, resources, engage, learn, how, to, grant, roles, console, gcloud, optimize, utilization,

Text of the page (most frequently used words):
the (93), cloud (93), run (75), with (61), and (59), service (46), for (44), project (44), google (32), you (32), your (32), #deploy (30), from (29), gemma (28), gpu (28), services (26), overview (25), create (24), using (21), requests (21), code (16), use (16), vllm (16), role (16), container (15), that (14), worker (14), this (13), configure (13), model (13), agents (13), development (13), iam (13), build (13), vpc (13), roles (13), functions (13), resources (12), request (12), memory (12), jobs (12), pools (12), environment (11), select (11), inference (10), gcloud (10), can (10), concurrency (10), function (10), samples (9), more (9), send (9), models (9), best (9), practices (9), agent (9), how (9), instances (9), max (9), instance (9), trigger (9), such (9), cpu (9), maximum (9), storage (9), view (9), volumes (9), execute (9), see (8), sample (8), are (8), delete (8), set (8), triggers (8), thumb (7), gpus (7), identity (7), kit (7), python (7), num (7), seqs (7), application (7), region (7), tutorial (7), limits (7), account (7), console (7), pub (7), sub (7), all (6), other (6), content (6), page (6), java (6), startup (6), deployed (6), traffic (6), settings (6), proxy (6), add (6), direct (6), configuration (6), grant (6), migrate (6), metrics (6), manage (5), about (5), need (5), rtx (5), 6000 (5), pro (5), following (5), adk (5), optimal (5), optimize (5), scaling (5), command (5), usage (5), reasoning (5), probe (5), timeout (5), serving (5), secure (5), variables (5), serve (5), management (5), projects (5), access (5), resourcemanager (5), tools (5), security (5), networking (5), migration (5), network (5), source (5), job (5), node (5), eventarc (5), invoke (5), português (4), pricing (4), products (4), down (4), frameworks (4), true (4), model_name (4), token (4), authentication (4), learn (4), after (4), endpoint (4), within (4), utilization (4), note (4), new (4), per (4), enabled (4), http (4), type (4), port (4), app (4), 240 (4), checks (4), nvidia (4), host (4), enable (4), cost (4), based (4), admin (4), selector (4), plan (4), selecting (4), containers (4), custom (4), health (4), dependencies (4), sign (3), events (3), system (3), support (3), understand (3), information (3), under (3), developers (3), time (3), assistant (3), name (3), get (3), import (3), example (3), ensure (3), have (3), value (3), leads (3), queuing (3), also (3), should (3), latency (3), auto (3), don (3), scale (3), start (3), provides (3), recommended (3), performance (3), chat (3), completion (3), 9090 (3), user (3), used (3), service_name (3), when (3), install (3), llm (3), api (3), call (3), test (3), which (3), throttling (3), number (3), description (3), supported (3), image (3), execution (3), allow (3), zonal (3), redundancy (3), gemma4 (3), tool (3), require (3), efficient (3), continuous (3), agentic (3), address (3), required (3), click (3), granted (3), billing (3), existing (3), workflows (3), guides (3), hosting (3), monitoring (3), solutions (3), distributed (3), databases (3), local (3), introduction (3), web (3), mcp (3), log (3), write (3), authenticate (3), connectors (3), optimization (3), autoscale (3), labels (3), secrets (3), ephemeral (3), disk (3), cifs (3), smb (3), nfs (3), volume (3), mounts (3), entrypoint (3), retries (3), connect (3), firestore (3), concurrent (3), product (3), 한국어 (2), 日本語 (2), עברית (2), brasil (2), italiano (2), indonesia (2), français (2), español (2), américa (2), latina (2), deutsch (2), english (2), our (2), terms (2), site (2), youtube (2), started (2), github (2), last (2), updated (2), 2026 (2), utc (2), licensed (2), details (2), license (2), feedback (2), runtime (2), streamer (2), next (2), created (2), gemma_model (2), id_token (2), authorization (2), false (2), skip_special_tokens (2), enable_thinking (2), chat_template_kwargs (2), https (2), openai (2), litellm (2), subprocess (2), before (2), incoming (2), authenticating (2), help (2), increasing (2), parallel (2), available (2), queued (2), determines (2), many (2), each (2), handle (2), context (2), setting (2), success (2), stop (2), why (2), sky (2), blue (2), message (2), e2b (2), output (2), curl (2), localhost (2), running (2), choose (2), end (2), not (2), seconds (2), egress (2), 600 (2), invoker (2), recommendations (2), amount (2), 80gi (2), allocated (2), full (2), list (2), regions (2), replace (2), container_args (2), 8080 (2), vertex (2), parser (2), generation (2), cli (2), advanced (2), capabilities (2), calling (2), adding (2), binding (2), typically (2), email (2), principal (2), project_id (2), save (2), additional (2), verify (2), keep (2), procedure (2), instead (2), finish (2), these (2), steps (2), removing (2), associated (2), creator (2), contains (2), permission (2), projectcreator (2), doesn (2), specific (2), any (2), been (2), perform (2), world (2), free (2), multi (2), coding (2), open (2), built (2), documentation (2), sdk (2), languages (2), infrastructure (2), costs (2), observability (2), industry (2), hybrid (2), multicloud (2), data (2), analytics (2), pipelines (2), compute (2), troubleshoot (2), oci (2), assisted (2), remote (2), server (2), a2a (2), logging (2), prometheus (2), controls (2), static (2), shared (2), connector (2), private (2), automate (2), external (2), metadata (2), file (2), systems (2), capacity (2), rollbacks (2), pool (2), revisions (2), tags (2), tasks (2), testing (2), integrate (2), workflow (2), base (2), runtimes (2), images (2), configurations (2), git (2), returns (2), results (2), php (2), ruby (2), prepare (2), develop (2), cases (2), cross (2), reference (2), technology (2), areas (2), close (2), subscribe, newsletter, third, decade, climate, action, join, cookies, privacy, tech, twitter, blog, engage, training, certification, architecture, center, getting, status, release, notes, community, forums, contact, sales, marketplace, easy, easytounderstand, solved, problem, solvedmyproblem, otherup, hard, hardtounderstand, incorrect, incorrectinformationorsamplecode, missing, missingtheinformationsamplesineed, otherdown, tell, except, otherwise, noted, registered, trademark, oracle, its, affiliates, policies, apache, creative, commons, attribution, various, larger, faster, codelab, what, clean, helpful, instruction, root_agent, bearer, extra_headers, extra_body, your_cloud_run_service_url, base_url, strip, stdout, text, capture_output, print, auth, lite_llm, shows, pass, appropriate, increase, keeping, twice, while, improve, immediately, process, their, queue, queues, absorb, spikes, important, makes, take, longer, requires, cache, completely, avoid, match, exceeds, than, has, slots, less, responsive, out, sends, same, sequences, concurrently, section, equal, line, argument, sent, 856, completion_tokens, 877, total_tokens, prompt_tokens, 106, stop_reason, finish_reason, question, tool_calls, null, function_call, short, answer, phenomenon, called, rayleigh, scattering, index, choices, 1774904187, object, chatcmpl, 9cf1ab1450487047, provide, similar, completions, messages, json, separate, terminal, tab, leaving, runs, specify, previously, prompted, component, header, valid, oidc, developer, now, however, directly, responds, intentional, because, intended, front, refer, 401, unauthorized, displayed, along, ending, url, modify, default, customized, comma, separated, form, key, sizes, least, response, must, returned, disables, actively, disable, better, check, hf_token, env, vars, processed, simultaneously, given, option, follows, e4b, variant, where, deployments, central1, unique, beta, gen2, unauthenticated, tcpsocket, initialdelayseconds, failurethreshold, timeoutseconds, periodseconds, args, echo, ifs, docker, pkg, dev, vision, garden, dockers, pytorch, tensor, size, dtype, bfloat16, choice, config, prefix, caching, chunked, prefill, 26b, 31b, fast, generative, featuring, state, art, throughput, batching, quantization, optimized, cuda, kernels, pagedattention, helps, fully, functional, offers, including, structured, review, generate, estimate, projected, calculator, necessary, quota, quotas, total, allocation, milli, without, deployer, project_number, policy, member, another, field, enter, identifier, principals, person_add, serviceusage, serviceusageconsumer, consumer, projectiamadmin, initialize, evaluate, real, scenarios, customers, 300, credits, workloads, begin, long, multimodality, complex, logic, step, planning, most, weight, family, delivering, strong, guide, describes, prebuilt, library, guidance, categorize, preferences, stay, organized, collections, home, known, issues, troubleshooting, errors, gke, kubernetes, vmware, tanzu, spring, music, compliant, strategy, foundry, heroku, aws, lambda, 1st, gen, engine, cookbook, vibe, accelerated, video, transcoding, ffmpeg, batch, fine, tune, llms, hugging, face, transformers, opencv, acceleration, ollama, browser, automation, servers, n8n, explore, tracing, error, reporting, audit, logs, opentelemetry, monitor, tenant, platforms, untrusted, software, supply, chain, insights, constraints, customer, managed, encryption, keys, threat, detection, binary, protect, armor, iap, control, users, audiences, public, design, mesh, restrict, ingress, outbound, standard, dual, stack, ipv4, ipv6, register, ips, dns, pull, subscriptions, runners, kafka, autoscaler, count, splits, background, work, checkpoints, executions, task, parallelism, scheduled, event, driven, load, general, tips, grpc, database, routed, entries, processing, into, push, subscription, asynchronous, series, part, schedule, asynchronously, websocket, stream, websockets, webhook, target, automatic, updates, language, manual, minimum, autoscaling, sandboxes, recommender, gradual, rollouts, copy, frontend, proxying, nginx, session, affinity, failover, multiple, assets, cdn, mapping, domains, compose, deployment, sources, codelabs, spanner, bigquery, tutorials, net, compare, commands, package, containerize, shell, sveltekit, nuxt, angular, ssr, kotlin, streamlit, smolagents, langchain, gradio, fastapi, flask, hello, repository, good, fit, contract, resource, discover, skip, main,

Text of the page (random words):
g controls manual scaling metadata description labels tags source deploy configurations supported language runtimes and base images configure automatic base image updates build environment variables build service account build worker pools invoke and trigger services invoke with https requests host a webhook target stream with websockets overview build a websocket chat service tutorial invoke asynchronously invoke services on a schedule create a workflow invoke services as part of a workflow connect a series of services from cloud functions and cloud run tutorial execute asynchronous tasks call a service from a pub sub push subscription trigger service from pub sub integrate image processing into pub sub sample tutorial trigger from events create triggers with eventarc pub sub triggers create pub sub eventarc triggers trigger functions from pub sub using eventarc trigger functions from routed log entries cloud storage triggers create triggers with cloud storage trigger services from cloud storage using eventarc trigger functions from cloud storage using eventarc firestore triggers create triggers with firestore trigger functions from events in a firestore database connect with other services using grpc best practices general development tips for services cost optimization optimize java services optimize python services optimize node js services load testing best practices understand zonal redundancy functions best practices overview configure event driven function retries execute job tasks to completion create jobs execute jobs execute jobs execute scheduled jobs execute jobs from workflows configure jobs container entrypoint cpu limits memory limits gpu gpu configuration gpu best practices environment variables container health checks volume mounts cloud storage volumes nfs volumes in memory volumes using cifs smb network file systems ephemeral disk labels maximum retries parallelism secrets service identity task timeout tags manage jobs view or delete jobs view or stop job executions best practices jobs retries and checkpoints cost optimization perform continuous background work deploy worker pools deploy worker pools deploy worker pools from source code manage worker pools view or delete worker pools view or delete worker pool revisions instance splits and rollbacks configure worker pools capacity memory limits cpu limits gpu gpu configuration gpu best practices environment container and entrypoint environment variables volume mounts cloud storage volumes nfs volumes in memory volumes using cifs smb network file systems ephemeral disk container health checks secrets service identity instance count metadata description labels scale based on external metrics autoscale worker pools with external metrics kafka autoscaler host github runners with worker pools autoscale worker pools based on prometheus metrics autoscale worker pools with pub sub pull subscriptions automate scaling with workflows cost optimization configure networking best practices for cloud run networking configure private networking send traffic to vpc network overview direct vpc register private ips for worker pools using cloud dns dual stack ipv4 and ipv6 migrate standard vpc connector to direct vpc vpc connectors send traffic to shared vpc network overview direct vpc migrate shared vpc connector to direct vpc connectors in service projects connectors in host project static outbound ip address network security restrict endpoint ingress services use vpc service controls vpc sc cloud service mesh secure security design overview authenticate requests overview allow public access custom audiences authenticate developers service to service authenticate users end user authentication tutorial secure your resources access control with iam configure iap for cloud run introduction to service identity protect services with cloud armor use binary authorization use cloud run threat detection use customer managed encryption keys manage custom constraints for projects view software supply chain security insights secure cloud run services tutorial multi tenant platforms running untrusted code monitor and log monitoring and logging overview view built in metrics write prometheus metrics write opentelemetry metrics log and view logs audit logging error reporting use distributed tracing for services run ai solutions overview explore resources ai agents overview build and deploy a2a agents overview deploy a2a agents build and deploy adk agents build and deploy n8n agents mcp servers overview build and deploy a remote mcp server tools code execution browser automation inference with gpus overview services run llm inference on cloud run gpus with ollama run agents with gemma 4 models on cloud run run opencv on cloud run with gpu acceleration run llm inference on cloud run gpus with hugging face transformers js jobs fine tune llms using gpus with cloud run jobs run batch inference using gpus with cloud run jobs gpu accelerated video transcoding with ffmpeg ai assisted development and vibe coding introduction to cloud run for ai assisted developers cookbook migrate an existing web service from app engine from cloud run functions 1st gen from aws lambda from heroku from cloud foundry migration overview choose an oci compliant strategy migrate to oci containers migrate configuration sample migration spring music from vmware tanzu from a vm using migrate to containers from kubernetes to gke troubleshoot introduction troubleshoot errors local troubleshooting tutorial known issues samples all cloud run code samples all cloud run functions code samples code samples for all products ai and ml application development application hosting compute data analytics and pipelines databases distributed hybrid and multicloud industry solutions migration networking observability and monitoring security storage access and resources management costs and usage management infrastructure as code sdk languages frameworks and tools home documentation application hosting cloud run guides send feedback run agents with gemma 4 models on cloud run stay organized with collections save and categorize content based on your preferences this guide describes how to deploy gemma 4 open models on cloud run using a prebuilt container with vllm inference library and provides guidance on using the deployed cloud run service with ai agents built using agent development kit gemma 4 is the google s most efficient open weight model family delivering strong reasoning and agentic capabilities long context multimodality reasoning and tool calling allow gemma 4 handle complex logic multi step planning coding and agentic workflows before you begin sign in to your google cloud account if you re new to google cloud create an account to evaluate how our products perform in real world scenarios new customers also get 300 in free credits to run test and deploy workloads in the google cloud console on the project selector page select or create a google cloud project roles required to select or create a project select a project selecting a project doesn t require a specific iam role you can select any project that you ve been granted a role on create a project to create a project you need the project creator role roles resourcemanager projectcreator which contains the resourcemanager projects create permission learn how to grant roles note if you don t plan to keep the resources that you create in this procedure create a project instead of selecting an existing project after you finish these steps you can delete the project removing all resources associated with the project go to project selector verify that billing is enabled for your google cloud project in the google cloud console on the project selector page select or create a google cloud project roles required to select or create a project select a project selecting a project doesn t require a specific iam role you can select any project that you ve been granted a role on create a project to create a project you need the project creator role roles resourcemanager projectcreator which contains the resourcemanager projects create permission learn how to grant roles note if you don t plan to keep the resources that you create in this procedure create a project instead of selecting an existing project after you finish these steps you can delete the project removing all resources associated with the project go to project selector verify that billing is enabled for your google cloud project set up your cloud run development environment in your google cloud project install and initialize the gcloud cli ensure you have the following iam roles granted to your account cloud run admin roles run admin project iam admin roles resourcemanager projectiamadmin service usage consumer roles serviceusage serviceusageconsumer learn how to grant the roles console in the google cloud console go to the iam page go to iam select the project click person_add grant access in the new principals field enter your user identifier this is typically the email address that is used to deploy the cloud run service in the select a role list select a role to grant additional roles click add add another role and add each additional role click save gcloud to grant the required iam roles to your account on your project gcloud projects add iam policy binding project_id member principal role role replace project_number with your google cloud project number project_id with your google cloud project id principal with the account you are adding the binding for this is typically the email address that is used to deploy the cloud run service role with the role you are adding to the deployer account if necessary request total nvidia rtx pro 6000 gpu allocation in milli gpu without zonal redundancy per project per region quota under cloud run admin api in the quotas and system limits page review the cloud run pricing page to generate a cost estimate based on your projected usage use the pricing calculator deploy a gemma 4 model with a vllm container gemma 4 offers advanced agentic capabilities including reasoning function calling code generation and structured output agent development kit adk helps you build fully functional ai agents with gemma 4 use vllm to serve gemma as an openai api endpoint vllm provides fast and efficient serving for generative models at scale featuring state of the art serving throughput efficient memory management with pagedattention continuous batching of incoming requests quantization support and optimized cuda kernels note gemma 4 26b and 31b require more advanced cloud run and vllm configuration with direct vpc egress and run ai model streamer to deploy gemma models on cloud run use the following gcloud cli command with the recommended settings container_args serve model_name enable chunked prefill enable prefix caching generation config auto enable auto tool choice tool call parser gemma4 reasoning parser gemma4 dtype bfloat16 max num seqs 64 gpu memory utilization 0 95 tensor parallel size 1 port 8080 host 0 0 0 0 gcloud beta run deploy service_name image us docker pkg dev vertex ai vertex vision model garden dockers pytorch vllm serve gemma4 project project region region execution environment gen2 no allow unauthenticated cpu 20 memory 80gi gpu 1 gpu type nvidia rtx pro 6000 no gpu zonal redundancy no cpu throttling max instances 3 concurrency 64 timeout 600 startup probe tcpsocket port 8080 initialdelayseconds 240 failurethreshold 1 timeoutseconds 240 periodseconds 240 command vllm args ifs echo container_args replace service_name with a unique name for the cloud run service project with your google cloud project id region with a google cloud region where nvidia rtx pro 6000 gpus are supported for cloud run such as us central1 for a full list of supported regions for gpu enabled deployments see gpu configuration model_name with the full name of a gemma 4 variant gemma 4 2b google gemma 4 e2b it gemma 4 4b google gemma 4 e4b it the other settings are as follows option description concurrency the maximum number of requests that can be processed simultaneously by a given instance such as 8 see set concurrency for optimal performance for recommendations on optimal request latency cpu the amount of allocated cpu for your service such as 20 set env vars the environment variables set for your service for example hf_token gpu the gpu value for your service such as 1 gpu type the type of gpu to use for your service such as nvidia rtx pro 6000 max instances the maximum number of container instances for your service such as 1 memory the amount of allocated memory for your service such as 80gi no invoker iam check disable invoker iam checks see secure cloud run services tutorial for recommendations on how to better secure your app no cpu throttling this setting disables cpu throttling when the container is not actively serving requests timeout the time within which a response must be returned such as 600 seconds startup probe comma separated settings for startup probe in the form key value see cloud run startup probe for details with model sizes of gemma 4 if you are not using direct vpc egress it s recommended to set your startup probe timeout to at least 240 seconds if you need to modify the default settings or add more customized settings to your cloud run service see configure services after completion of the deployed service a success message is displayed along with the cloud run endpoint url ending with run app test the deployed gemma service with curl now that you have deployed the gemma service you can send requests to it however if you send a request directly cloud run responds with http 401 unauthorized this is intentional because an llm inference api is intended for other services to call such as a front end application for more information on service to service authentication on cloud run refer to authenticating service to service to send requests to the gemma service add a header with a valid oidc token to the requests for example using the cloud run developer proxy start the proxy and when prompted to install the cloud run proxy component choose y gcloud run services proxy service_name project project region region port 9090 run the following command to send a request in a separate terminal tab leaving the proxy running the proxy runs on localhost 9090 specify the gemma model you previously used curl http localhost 9090 v1 chat completions h content type application json d model model_name messages role user content why is the sky blue chat_template_kwargs enable_thinking true skip_special_tokens false this command should provide an output similar to this id chatcmpl 9cf1ab1450487047 object chat completion created 1774904187 model google gemma 4 e2b it choices index 0 message role assistant content the short answer is a phenomenon called rayleigh scattering function_call null tool_calls reasoning question why is the sky blue n finish_reason stop stop_reason 106 usage prompt_toke...