Meta tags:
description= Understand instance options available to support GPU-accelerated workloads such as machine learning, data processing, and graphics workloads on Compute Engine.;
keywords= compute engine, gce, gcp, google cloud, nvidia gpu models, nvidia gpus, gpu machine types, gpu types, gcp gpu, a4x max a4X, a4, a3, a2, g2, a3 ultra, a3 mega, a3 high, a3 edge, a2 ultra, a2 standard, g4, n1, nvidia gb200, rtx pro 6000, b200, h200, h100, a100, l4, t4, p100, v100, machine series, gpu accelerated vms, gpu memory, gpu count;
Headings (most frequently used words):
machine, nvidia, and, series, a3, n1, core, standard, a4x, tensor, cuda, gpus, types, a2, cores, max, type, ultra, mega, high, edge, architectures, gpu, stay, organized, with, collections, save, categorize, content, based, on, your, preferences, overview, a4, b200, a100, g4, rtx, pro, 6000, g2, l4, general, comparison, chart, performance, what, next, gb300, gb200, h200, h100, t4, p4, v100, p100, blackwell, architecture, hopper, ada, lovelace, ampere, volta, pascal, turing, products, pricing, support, resources, engage,
Text of the page (most frequently used words):
and (218), the (210), for (173), create (141), gpu (117), nvidia (116), #machine (113), vms (112), with (98), instance (95), memory (81), about (72), troubleshoot (62), bandwidth (59), types (58), gpus (56), performance (53), that (52), use (52), network (48), disks (47), manage (46), type (44), attached (43), disk (43), instances (42), workloads (41), server (41), view (40), compute (40), you (40), can (39), using (39), count (38), mig (37), from (36), engine (35), standard (35), overview (35), available (33), set (33), windows (33), a4x (32), configure (32), see (30), google (29), training (29), data (28), ssd (28), optimized (28), cloud (27), model (27), reservation (27), inference (26), local (26), hpc (25), series (25), gbps (25), storage (25), are (23), vcpu (23), number (23), h100 (22), 000 (22), virtual (22), your (22), maximum (22), cores (21), ultra (21), high (21), accelerator (21), hyperdisk (21), sql (21), support (20), following (20), graphics (20), cpu (20), monitor (20), images (20), p100 (19), a100 (19), yes (19), host (19), information (18), more (18), this (18), tflops (18), these (18), models (18), higher (18), cluster (18), access (18), resources (17), rtx (17), max (17), workload (17), intensive (17), custom (17), image (17), linux (17), policies (16), zones (16), tensor (16), when (16), blackwell (16), single (16), designed (16), tesla (16), egress (16), reservations (16), availability (15), not (15), best (15), vws (15), snapshots (15), groups (15), ssh (15), recommendations (15), configuration (15), other (14), 80gb (14), specifically (14), based (14), creation (14), load (14), apply (14), connect (14), used (13), pro (13), general (13), device (13), handle (13), add (13), highgpu (13), start (13), management (13), bulk (13), microsoft (13), architecture (12), have (12), core (12), operations (12), large (12), workstations (12), scale (12), temporary (12), separate (12), demands (12), attach (12), sole (12), persistent (12), regional (12), regions (11), fp16 (11), mega (11), h200 (11), gb300 (11), choose (11), spot (11), options (11), delete (11), practices (11), stateful (11), v100 (10), cuda (10), also (10), 6000 (10), b200 (10), gb200 (10), multiple (10), nvlink (10), one (10), platforms (10), gib (10), physical (10), provisioning (10), request (10), managed (10), metadata (10), migs (10), machines (10), migrate (10), logs (10), license (9), fp64 (9), table (9), architectures (9), edge (9), purpose (9), hardware (9), full (9), ideal (9), supported (9), depends (9), resource (9), creating (9), address (9), nic (9), flex (9), deploy (9), capacity (9), clusters (9), networking (9), application (9), licenses (9), tenant (9), openshift (9), shutdown (9), time (9), all (8), int8 (8), precision (8), which (8), throughput (8), uses (8), run (8), 500 (8), 192 (8), hbm3e (8), configurations (8), implemented (8), hyper (8), thread (8), cannot (8), exceed (8), given (8), actual (8), destination (8), factors (8), note (8), applications (8), serving (8), service (8), import (8), login (8), optimize (8), internal (8), install (8), disaster (8), recovery (8), agent (8), existing (8), update (8), guest (8), boot (8), keys (8), h4d (8), flexibility (8), dns (8), tpu (8), pricing (7), thumb (7), details (7), review (7), indicates (7), metric (7), applicable (7), mixed (7), provide (7), learning (7), metrics (7), video (7), transcoding (7), visualization (7), usage (7), 100 (7), capabilities (7), mesh (7), tbps (7), distributed (7), workstation (7), zone (7), 375 (7), default (7), resize (7), group (7), sxm (7), commitments (7), startup (7), balancing (7), benchmark (7), enable (7), manually (7), idle (7), addresses (7), back (7), mysql (7), zonal (7), remove (7), maintenance (7), templates (7), snapshot (7), backup (7), events (6), send (6), fp32 (6), matrix (6), 624 (6), 312 (6), foundation (6), hbm2 (6), 600 (6), 384 (6), get (6), peer (6), communication (6), multi (6), reserve (6), grace (6), superchips (6), hypercomputer (6), security (6), migration (6), updates (6), instant (6), files (6), nested (6), across (6), mode (6), demand (6), pools (6), failover (6), extension (6), change (6), new (6), transfer (6), customized (6), code (5), samples (5), understand (5), accelerate (5), int4 (5), 40gb (5), hopper (5), ada (5), lovelace (5), ampere (5), tf32 (5), fp8 (5), each (5), different (5), remote (5), 208 (5), some (5), has (5), 640 (5), ultragpu (5), must (5), kubernetes (5), slurm (5), deployment (5), cpus (5), bound (5), automatically (5), monitoring (5), databases (5), accounts (5), errors (5), future (5), drivers (5), virtualization (5), placement (5), node (5), highly (5), build (5), modify (5), schedules (5), audit (5), symphony (5), email (5), scripts (5), manager (5), between (5), operating (5), systems (5), enhanced (5), stop (5), tenancy (5), restore (5), control (5), faq (5), português (4), español (4), started (4), down (4), volta (4), turing (4), range (4), dense (4), structural (4), sparsity (4), 156 (4), 979 (4), fp4 (4), such (4), llms (4), gddr6 (4), 320 (4), hbm3 (4), 800 (4), size (4), end (4), shared (4), 750 (4), require (4), cost (4), instructions (4), deploying (4), services (4), documentation (4), platform (4), arm (4), chip (4), metal (4), infrastructure (4), tools (4), authentication (4), issues (4), ubuntu (4), console (4), tcp (4), pmu (4), share (4), projects (4), upgrade (4), live (4), autoscaling (4), cross (4), region (4), requests (4), calendar (4), health (4), active (4), containers (4), container (4), database (4), postgresql (4), extensions (4), work (4), suspend (4), location (4), graceful (4), locations (4), plans (4), replication (4), terms (3), system (3), 2026 (3), content (3), apache (3), check (3), pascal (3), tops (3), don (3), deep (3), feature (3), advanced (3), balanced (3), 250 (3), vector (3), units (3), 900 (3), massive (3), tables (3), bert (3), dlrm (3), recommenders (3), 128 (3), only (3), limit (3), 360 (3), without (3), tuning (3), megagpu (3), 160 (3), two (3), 872 (3), detailed (3), gke (3), how (3), features (3), exascale (3), bare (3), generation (3), document (3), family (3), specific (3), guides (3), observability (3), analytics (3), automatic (3), troubleshooting (3), encryption (3), serial (3), patterns (3), discounts (3), cuds (3), manual (3), scalable (3), resilient (3), autoscale (3), balancer (3), prevent (3), project (3), policy (3), list (3), testing (3), ibm (3), spectrum (3), perform (3), plan (3), during (3), prepare (3), rhel (3), environment (3), protect (3), certificates (3), query (3), state (3), disable (3), properties (3), account (3), increase (3), schedule (3), lifecycle (3), design (3), non (3), pool (3), move (3), tpus (3), product (3), 한국어 (2), 日本語 (2), עברית (2), brasil (2), italiano (2), indonesia (2), français (2), américa (2), latina (2), deutsch (2), english (2), sign (2), site (2), youtube (2), center (2), products (2), need (2), last (2), updated (2), utc (2), except (2), page (2), licensed (2), under (2), feedback (2), next (2), introduced (2), primarily (2), because (2), acceleration (2), demanding (2), double (2), computational (2), values (2), section (2), assume (2), multiplication (2), doubled (2), bfloat16 (2), 121 (2), 989 (2), 494 (2), providing (2), specialized (2), provides (2), reduced (2), 117 (2), rendering (2), requirements (2), 935 (2), refers (2), achieve (2), threads (2), performing (2), same (2), compare (2), 300 (2), gddr5 (2), gddr7 (2), hbm2e (2), 180 (2), europe (2), west1 (2), east1 (2), 104 (2), runs (2), northamerica (2), vcpus (2), come (2), specify (2), computing (2), titanium (2), 400 (2), 440 (2), direct (2), within (2), exchange (2), over (2), bus (2), omniverse (2), simulation (2), desktops (2), 680 (2), 340 (2), 170 (2), offers (2), fine (2), gpudirect (2), enabled (2), methods (2), 224 (2), 141gb (2), then (2), sockets (2), neoverse (2), connected (2), four (2), fast (2), c2c (2), nvl72 (2), artificial (2), intelligence (2), integrations (2), smaller (2), pre (2), accelerators (2), adds (2), sdk (2), languages (2), frameworks (2), costs (2), industry (2), solutions (2), hybrid (2), multicloud (2), pipelines (2), hosting (2), development (2), quota (2), commitment (2), consumption (2), registration (2), sles (2), export (2), port (2), collecting (2), nodes (2), report (2), api (2), rdp (2), app (2), latency (2), analyze (2), visible (2), committed (2), underutilized (2), insights (2), utilization (2), globally (2), web (2), autohealing (2), iis (2), external (2), autoscaler (2), autoscalers (2), activity (2), provider (2), place (2), running (2), directory (2), setting (2), redis (2), cloning (2), writer (2), always (2), alwayson (2), name (2), file (2), volumes (2), authenticate (2), method (2), synchronization (2), global (2), switch (2), byol (2), payg (2), byos (2), changes (2), families (2), guide (2), securing (2), event (2), simulate (2), topology (2), together (2), repairs (2), repair (2), failures (2), template (2), suspended (2), stopped (2), target (2), distribution (2), cancel (2), once (2), interfaces (2), ipv6 (2), static (2), decrease (2), basic (2), consistent (2), scoped (2), copy (2), appliances (2), asynchronous (2), encrypt (2), names (2), format (2), mount (2), extreme (2), credentials (2), organization (2), user (2), through (2), rdma (2), created (2), preemptible (2), constraints (2), apis (2), reference (2), technology (2), areas (2), close (2), subscribe, newsletter, our, third, decade, climate, action, join, cookies, privacy, tech, twitter, blog, engage, certification, getting, github, status, release, notes, community, forums, contact, sales, marketplace, easy, easytounderstand, solved, problem, solvedmyproblem, otherup, hard, hardtounderstand, incorrect, sample, incorrectinformationorsamplecode, missing, missingtheinformationsamplesineed, otherdown, tell, otherwise, noted, java, registered, trademark, oracle, its, affiliates, developers, creative, commons, attribution, bandwidths, learn, what, equipped, 260, 130, 125, first, generations, quantization, doesn, include, they, offering, foundational, 242, engines, transformer, versatile, energy, efficient, prioritize, accelerated, included, benefit, even, supports, 1871, 467, 233, introduces, expanded, breakthrough, involve, math, operate, instruction, simt, typically, individual, elements, vectors, language, any, expressed, significantly, than, dedicated, often, called, multiply, accumulate, form, backbone, sections, separated, into, 732, ring, ecc, 1597, 141, 186, moe, 279, interconnect, describes, comparison, chart, reach, september, caution, aren, northeast1, central1, fewer, lets, unlike, instead, exception, 432, 216, 108, defines, amount, allocate, 768, 200, 720, key, p2p, allows, directly, pcie, involving, suitable, low, solution, compared, edition, 16g, asia, south1, northeast2, edgegpu, tcpx, 936, 468, 234, standalone, flex_start, recommend, scheduler, either, limited, well, suited, both, tasks, 1128, 952, parameters, highest, 968, 744, 884, 140, 116, 960, 144, maxgpu, b300, differ, their, components, hypervisor, layer, rack, supercomputing, option, recommended, densely, allocated, schedulers, approaching, 4ogb, select, small, micro, later, involves, while, various, including, find, matches, outlines, processing, save, categorize, preferences, stay, organized, collections, home, concurrent, operation, renewal, nvme, resizing, common, pay, viewing, output, diagnostic, sudoers, screenshots, generate, bug, soft, lockups, locks, rescue, inaccessible, dumps, kernel, panic, fstab, unresponsive, suspension, reboots, shutdowns, connectivity, tips, nics, dpdk, improve, resiliency, reduce, compact, idpf, interface, irdma, driver, customize, per, sustained, merge, split, extend, renew, purchase, savings, dynamic, overcommit, floating, reliable, udp, backends, routing, https, scaling, consuming, consume, combine, cud, allow, sharing, decisions, signals, predictions, organize, labels, replica, states, observe, reports, tensorflow, tensorrt5, monte, carlo, spark, forwarding, others, integrate, clustering, 2016, sharepoint, strategies, passive, inactive, setups, built, validation, were, deployed, transition, test, hammerdb, bucket, aws, ec2, pacemaker, s2d, block, netapp, hot, standby, drbd, client, private, mailjet, mailgun, sendgrid, sending, blue, green, deployments, lamp, joomla, asp, net, interactive, mongodb, flask, terraform, servers, mtls, random, generator, virtio, rng, accurate, protocol, ntp, append, els, after, restrictions, packages, deprecate, versions, trusted, red, hat, knowledgebase, functionality, risks, command, vpc, controls, confidential, kek, secure, expiration, shielded, attributes, predefined, notices, process, faulty, unmanaged, affect, preserved, turn, off, alternate, fails, repairing, maintain, click, upgrades, override, selectively, applying, out, outage, rebalance, reenable, proactive, redistribution, shape, distribute, define, info, accidental, deletion, spread, edit, rename, reset, resume, restart, referrers, source, uuid, recover, corrupted, duplicate, clones, alerts, backups, vss, regionally, protection, considerations, synchronous, consistency, failback, replicate, make, provisioned, iops, limits, evaluate, settings, help, kms, customer, supplied, symbolic, links, practice, ram, additional, exapools, verify, identity, ptr, record, failure, rates, tags, restrict, automate, password, powershell, sac, securely, apps, root, vpn, bastion, iap, connection, browser, connections, networks, iso, imported, prerequisites, importing, exporting, bring, own, path, detach, reattach, constraint, accounting, provision, examples, reserved, team, slice, slices, scenarios, deterministic, base, ready, aci, configured, preempted, historical, alternative, similar, displays, ops, logging, subnet, public, minimum, hostname, quickstarts, iam, roles, permissions, office, licensing, premium, strategy, gemini, scores, discover, free, skip, main,
Text of the page (random words):
local ssd disks a2 ultra attached nvidia a100 80gb gpus machine type vcpu count 1 instance memory gb attached local ssd gib maximum network bandwidth gbps 2 gpu count gpu memory 3 gb hbm2e a2 ultragpu 1g 12 170 375 24 1 80 a2 ultragpu 2g 24 340 750 32 2 160 a2 ultragpu 4g 48 680 1 500 50 4 320 a2 ultragpu 8g 96 1 360 3 000 100 8 640 a2 standard attached nvidia a100 40gb gpus machine type vcpu count 1 instance memory gb local ssd supported maximum network bandwidth gbps 2 gpu count gpu memory 3 gb hbm2 a2 highgpu 1g 12 85 yes 24 1 40 a2 highgpu 2g 24 170 yes 32 2 80 a2 highgpu 4g 48 340 yes 50 4 160 a2 highgpu 8g 96 680 yes 100 8 320 a2 megagpu 16g 96 1 360 yes 100 16 640 1 a vcpu is implemented as a single hardware hyper thread on one of the available cpu platforms 2 maximum egress bandwidth cannot exceed the number given actual egress bandwidth depends on the destination ip address and other factors for more information about network bandwidth see network bandwidth 3 gpu memory is the memory on a gpu device that can be used for temporary storage of data it is separate from the instance s memory and is specifically designed to handle the higher bandwidth demands of your graphics intensive workloads g4 machine series nvidia rtx pro 6000 g4 accelerator optimized machine types use nvidia rtx pro 6000 blackwell server edition gpus nvidia rtx pro 6000 and are suitable for nvidia omniverse simulation workloads graphics intensive applications video transcoding and virtual desktops g4 machine types also provide a low cost solution for performing single host inference and model tuning compared with a series machine types a key feature of the g4 series is support for direct gpu peer to peer p2p communication on multi gpu machine types g4 standard 96 g4 standard 192 g4 standard 384 this allows gpus within the same instance to exchange data directly over the pcie bus without involving the cpu host for more information about g4 gpu peer to peer communication see g4 gpu peer to peer communication note to get started with g4 instances see create a g4 instance attached nvidia rtx pro 6000 gpus machine type vcpu count 1 instance memory gb maximum titanium ssd supported gib 2 physical nic count maximum network bandwidth gbps 3 gpu count gpu memory 4 gb gddr7 g4 standard 6 6 22 0 1 20 1 8 12 g4 standard 12 12 45 375 1 20 1 4 24 g4 standard 24 24 90 750 1 20 1 2 48 g4 standard 48 48 180 1 500 1 50 1 96 g4 standard 96 96 360 3 000 1 100 2 192 g4 standard 192 192 720 6 000 1 200 4 384 g4 standard 384 384 1 440 12 000 2 400 8 768 1 a vcpu is implemented as a single hardware hyper thread on one of the available cpu platforms 2 you can add titanium ssd disks when creating a g4 instance for the number of disks you can attach see machine types that require you to choose a number of local ssd disks 3 maximum egress bandwidth cannot exceed the number given actual egress bandwidth depends on the destination ip address and other factors see network bandwidth 4 gpu memory is the memory on a gpu device that can be used for temporary storage of data it is separate from the instance s memory and is specifically designed to handle the higher bandwidth demands of your graphics intensive workloads g2 machine series nvidia l4 g2 accelerator optimized machine types have nvidia l4 gpus attached and are ideal for cost optimized inference graphics intensive and high performance computing workloads each g2 machine type also has a default memory and a custom memory range the custom memory range defines the amount of memory that you can allocate to your instance for each machine type you can also add local ssd disks when creating a g2 instance for the number of disks you can attach see machine types that require you to choose a number of local ssd disks attached nvidia l4 gpus machine type vcpu count 1 default instance memory gb custom instance memory range gb max local ssd supported gib maximum network bandwidth gbps 2 gpu count gpu memory 3 gb gddr6 g2 standard 4 4 16 16 to 32 375 10 1 24 g2 standard 8 8 32 32 to 54 375 16 1 24 g2 standard 12 12 48 48 to 54 375 16 1 24 g2 standard 16 16 64 54 to 64 375 32 1 24 g2 standard 24 24 96 96 to 108 750 32 2 48 g2 standard 32 32 128 96 to 128 375 32 1 24 g2 standard 48 48 192 192 to 216 1 500 50 4 96 g2 standard 96 96 384 384 to 432 3 000 100 8 192 1 a vcpu is implemented as a single hardware hyper thread on one of the available cpu platforms 2 maximum egress bandwidth cannot exceed the number given actual egress bandwidth depends on the destination ip address and other factors for more information about network bandwidth see network bandwidth 3 gpu memory is the memory on a gpu device that can be used for temporary storage of data it is separate from the instance s memory and is specifically designed to handle the higher bandwidth demands of your graphics intensive workloads n1 machine series you can attach the following gpu models to an n1 machine type with the exception of the n1 shared core machine types unlike the machine types in the accelerator optimized machine series n1 machine types don t come with a set number of attached gpus instead you specify the number of gpus to attach when creating the instance n1 instances with fewer gpus limit the maximum number of vcpus in general a higher number of gpus lets you create instances with a higher number of vcpus and memory n1 t4 gpus you can attach nvidia t4 gpus to n1 general purpose instances with the following instance configurations accelerator type gpu count gpu memory 1 gb gddr6 vcpu count instance memory gb local ssd supported nvidia tesla t4 or nvidia tesla t4 vws 1 16 1 to 48 1 to 312 yes 2 32 1 to 48 1 to 312 yes 4 64 1 to 96 1 to 624 yes 1 gpu memory is the memory available on a gpu device that you can use for temporary data storage it is separate from the instance s memory and is specifically designed to handle the higher bandwidth demands of your graphics intensive workloads n1 p4 gpus you can attach nvidia p4 gpus to n1 general purpose instances with the following instance configurations accelerator type gpu count gpu memory 1 gb gddr5 vcpu count instance memory gb local ssd supported 2 nvidia tesla p4 or nvidia tesla p4 vws 1 8 1 to 24 1 to 156 yes 2 16 1 to 48 1 to 312 yes 4 32 1 to 96 1 to 624 yes 1 gpu memory is the memory that is available on a gpu device that you can use for temporary data storage it is separate from the instance s memory and is specifically designed to handle the higher bandwidth demands of your graphics intensive workloads 2 for instances with attached nvidia p4 gpus local ssd disks are only supported in zones us central1 c and northamerica northeast1 b n1 v100 gpus you can attach nvidia v100 gpus to n1 general purpose instances with the following instance configurations accelerator type gpu count gpu memory 1 gb hbm2 vcpu count instance memory gb local ssd supported 2 nvidia tesla v100 1 16 1 to 12 1 to 78 yes 2 32 1 to 24 1 to 156 yes 4 64 1 to 48 1 to 312 yes 8 128 1 to 96 1 to 624 yes 1 gpu memory is the memory available on a gpu device that you can use for temporary data storage it is separate from the instance s memory and is specifically designed to handle the higher bandwidth demands of your graphics intensive workloads 2 for instances with attached nvidia v100 gpus local ssd disks aren t supported in us east1 c n1 p100 gpus you can attach nvidia p100 gpus to n1 general purpose instances with the following instance configurations for some nvidia p100 gpus the maximum cpu and memory available for some configurations depends on the zone in which the gpu resource runs caution nvidia p100 gpus reach end of support on september 15 2026 for more information see nvidia p100 end of support accelerator type gpu count gpu memory 1 gb hbm2 zone vcpu count instance memory gb local ssd supported nvidia tesla p100 or nvidia tesla p100 vws 1 16 all p100 zones 1 to 16 1 to 104 yes 2 32 all p100 zones 1 to 32 1 to 208 yes 4 64 us east1 c europe west1 d europe west1 b 1 to 64 1 to 208 yes all other p100 zones 1 to 96 1 to 624 yes 1 gpu memory is the memory available on a gpu device that you can use for temporary data storage it is separate from the instance s memory and is specifically designed to handle the higher bandwidth demands of your graphics intensive workloads general comparison chart the following table describes the gpu memory size feature availability and ideal workload types of different gpu models on compute engine in the following table n a indicates that the metric is not applicable or not available for this gpu model machine type gpu model gpu memory interconnect nvidia rtx virtual workstation vws support best used for a4x max gb300 279 gb hbm3e 8 tbps nvlink full mesh 1 800 gbps large scale distributed training and inference of moe llms recommenders hpc a4x gb200 186 gb hbm3e 8 tbps nvlink full mesh 1 800 gbps large scale distributed training and inference of llms recommenders hpc a4 b200 180 gb hbm3e 8 tbps nvlink full mesh 1 800 gbps large scale distributed training and inference of llms recommenders hpc a3 ultra h200 141 gb hbm3e 4 8 tbps nvlink full mesh 900 gbps large models with massive data tables for ml training inference hpc bert dlrm a3 mega a3 high a3 edge h100 80 gb hbm3 3 35 tbps nvlink full mesh 900 gbps large models with massive data tables for ml training inference hpc bert dlrm a2 ultra a100 80gb 80 gb hbm2e 1 9 tbps nvlink full mesh 600 gbps large models with massive data tables for ml training inference hpc bert dlrm a2 standard a100 40gb 40 gb hbm2 1 6 tbps nvlink full mesh 600 gbps ml training inference hpc g4 rtx pro 6000 96 gb gddr7 with ecc 1597 gbps n a ml inference training remote visualization workstations video transcoding hpc g2 l4 24 gb gddr6 300 gbps n a ml inference training remote visualization workstations video transcoding hpc n1 t4 16 gb gddr6 320 gbps n a ml inference training remote visualization workstations video transcoding n1 p4 8 gb gddr5 192 gbps n a remote visualization workstations ml inference and video transcoding n1 v100 16 gb hbm2 900 gbps nvlink ring 300 gbps ml training inference hpc n1 p100 16 gb hbm2 732 gbps n a ml training inference hpc remote visualization workstations to compare gpu pricing for the different gpu models and regions available on compute engine see gpu pricing tensor core and standard cuda core performance the following sections provide performance metrics for each gpu architecture separated into vector or standard cuda cores and tensor core performance tensor cores tensor performance refers to the throughput specialized tensor cores achieve these are dedicated hardware units often called matrix units designed specifically to accelerate the large matrix multiply accumulate operations that form the backbone of deep learning training and inference this type of performance is best for deep learning large language models llms and any workload that can be expressed as dense matrix operations tensor cores provide significantly higher throughput than cuda cores for the same data type vector or standard cuda cores vector performance refers to the throughput standard cuda cores achieve these are general purpose units that operate using a single instruction multiple threads simt model typically performing operations on individual data elements or vectors this type of performance is best for general compute graphics rendering and workloads that don t involve dense matrix math blackwell architecture the a4x max a4x a4 and g4 machine types run on nvidia s blackwell architecture tensor core nvidia s blackwell architecture used by these machine types introduces tensor core support for fp4 precision and expanded int4 capabilities for breakthrough performance in large model inference in the following table n a indicates that the metric is not applicable or not available for this gpu model machine type gpu model fp64 tflops tf32 tflops 1 mixed fp16 32 tflops 1 2 int8 tflops 1 fp8 tflops 1 fp4 tflops 1 a4x max gb300 1 3 1 250 2 500 5 000 5 000 15 000 a4x gb200 45 1 250 2 500 5 000 5 000 10 000 a4 b200 40 1 100 2 250 4 500 4 500 9 000 g4 rtx pro 6000 n a 233 9 467 8 935 6 935 6 1871 2 1 the blackwell architecture supports structural sparsity for tf32 fp16 32 int8 fp8 and fp4 precision metrics which can double computational throughput the performance values in this section assume dense matrix multiplication if you use structural sparsity performance is doubled 2 for mixed precision training gpus that run on blackwell architecture also support the bfloat16 data type standard cuda cores the machine types that use the blackwell architecture provide high performance fp64 and fp32 operations for demanding hpc and ai workloads for a4x max a4x and a4 fp16 operations are accelerated by tensor cores for g4 fp16 performance on standard cuda cores is included because graphics workloads such as rendering and visualization can benefit from the reduced memory usage and bandwidth requirements of fp16 precision even when not using tensor cores in the following table n a indicates that the metric is not applicable or not available for this gpu model machine type gpu model fp64 tflops fp32 tflops fp16 tflops a4x max gb300 1 39 1 45 n a a4x gb200 45 45 n a a4 b200 40 80 n a g4 rtx pro 6000 1 8 117 117 1 fp64 performance on gb300 is reduced to prioritize fp4 performance hopper ada lovelace and ampere architectures the a3 series uses the hopper architecture which introduced specialized engines for transformer models the a2 series uses the ampere architecture providing a balanced foundation for high performance training and inference the g2 series uses the ada lovelace architecture which provides versatile and energy efficient acceleration for ai inference video transcoding and graphics workloads tensor core the hopper ada lovelace and ampere architectures feature advanced tensor cores that accelerate tf32 fp16 fp8 and int8 data types providing high throughput for mixed precision training and inference in the following table n a indicates that the metric is not applicable or not available for this gpu model machine type gpu model fp64 tflops tf32 tflops 1 mixed fp16 32 tflops 1 2 int8 tops 1 fp8 tflops 1 a3 ultra h200 67 494 5 989 5 1 979 1 979 a3 mega high edge h100 67 494 5 989 5 1 979 1 979 a2 ultra a100 80gb 19 5 156 312 624 n a a2 standard a100 40gb 19 5 156 312 624 n a g2 l4 n a 60 121 242 5 121 1 the hopper ada lovelace and ampere architectures support structural sparsity for tf32 fp16 32 int8 int4 and fp8 precision metrics which can double computational throughput the performance values in this section assume dense matrix multiplication if you use structural sparsity performance is doubled 2 for mixed precision training nvidia h200 h100 a100 and l4 also support the bfloat16 data type standard cuda cores the machine types that use the hopper ada lovelace and ampere architectures provide high performance fp64 and fp32 operations for demanding hpc and ai workloads in the following t...
|
