Meta tags:

Headings (most frequently used words):

fields, syntax, template, example, how, it, works, vector, site, footer, event, strftime, specifiers, escaping, accessing, fallback, values, missing, about, components, setup, community, download,

Text of the page (most frequently used words):
metrics (32), logs (30), aws (25), the (21), #vector (19), log (13), gcp (12), option (12), datadog (11), #syntax (10), prometheus (10), model (10), event (9), this (9), data (8), architecture (8), configuration (7), and (7), metric (7), remap (7), can (7), application_id (7), for (6), template (6), websocket (6), splunk (6), hec (6), http (6), file (6), events (6), azure (6), kinesis (6), reference (6), from (6), source (6), docs (6), fields (6), toml (6), yaml (6), sinks (5), transforms (5), components (5), you (5), with (5), internal (5), field (5), chat (4), github (4), releases (4), api (4), pipeline (4), server (4), statsd (4), socket (4), sematext (4), redis (4), pulsar (4), remote (4), write (4), opentelemetry (4), nats (4), mqtt (4), formerly (4), kafka (4), influxdb (4), humio (4), greptimedb (4), pubsub (4), stackdriver (4), cloud (4), storage (4), monitoring (4), sqs (4), firehose (4), cloudwatch (4), amqp (4), route (4), windows (4), kubernetes (4), docker (4), agent (4), end (4), concurrency (4), linux (4), my_field (4), type (4), values (4), value (4), year (4), month (4), day (4), strftime (4), aws_s3 (4), date (4), key_prefix (4), rss (3), download (3), going (3), administration (3), deployment (3), installation (3), setup (3), sources (3), secrets (3), schema (3), updates (3), new (3), global (3), options (3), tag (3), vrl (3), examples (3), home (3), default (3), support (3), accessing (3), would (3), specifiers (3), versioning (2), security (2), meta (2), glossary (2), environment (2), variables (2), cli (2), unit (2), tests (2), tls (2), webhdfs (2), exporter (2), postgres (2), papertrail (2), relic (2), mezmo (2), logdna (2), loki (2), keep (2), honeycomb (2), chronicle (2), unstructured (2), elasticsearch (2), doris (2), traces (2), databricks (2), zerobus (2), databend (2), console (2), clickhouse (2), blackhole (2), monitor (2), ingestion (2), blob (2), axiom (2), sns (2), streams (2), appsignal (2), window (2), trace (2), throttle (2), cardinality (2), limit (2), sample (2), reduce (2), lua (2), incremental (2), absolute (2), filter (2), exclusive (2), delay (2), dedupe (2), ec2 (2), metadata (2), aggregate (2), syslog (2), stdin (2), static (2), scrape (2), pushgateway (2), postgresql (2), okta (2), nginx (2), mongodb (2), logstash (2), journald (2), client (2), host (2), heroku (2), logplex (2), fluent (2), descriptor (2), exec (2), eventstoredb (2), dnstap (2), demo (2), ecs (2), apache (2), expressions (2), errors (2), functions (2), language (2), pgo (2), optimization (2), validating (2), management (2), acknowledgements (2), guarantees (2), adaptive (2), buffering (2), runtime (2), sizing (2), capacity (2), planning (2), rollout (2), high (2), availability (2), hardening (2), architecting (2), unified (2), aggregator (2), architectures (2), production (2), topologies (2), roles (2), installer (2), archives (2), manual (2), ubuntu (2), rhel (2), raspbian (2), nixos (2), macos (2), debian (2), centos (2), arch (2), amazon (2), operating (2), systems (2), platforms (2), yum (2), rpm (2), pacman (2), nix (2), msi (2), homebrew (2), helm (2), dpkg (2), apt (2), package (2), managers (2), quickstart (2), concepts (2), introduction (2), missing (2), exists (2), inputs (2), set_defaults (2), fallback (2), open (2), documentation (2), path (2), expression (2), each (2), above (2), like (2), field_name (2), escape (2), example (2), derived (2), timestamp (2), parent (2), child (2), sink (2), dynamic (2), 2020 (2), that (2), allows (2), all_application_logs (2), bucket (2), backup (2), supports (2), observability (2), pipelines (2), blog (2), guides (2), sidebar, 2026, inc, all, rights, reserved, community, prod, cookies, privacy, team, contact, about, site, footer, next, previous, thank, joining, our, newsletter, sign, receive, emails, latest, content, close, slideover, panel, page, error, logged, drops, incremented, template_failed, error_type, component_errors_total, doesn, currently, add, functionality, interim, use, set, transform, issue, 1692, find, additional, how, works, treated, literally, specified, prefixing, character, escaping, name, changed, via, timestamp_key, addition, directly, offers, shortcut, injecting, individual, accessed, using, wrap, because, batches, grouped, its, produced, effectively, enables, partitioning, something, fundamental, storing, filesystems, equal, hello, world, message, 14t01, 223z, notice, direct, references, well, were, run, following, through, let, partition, accomplish, some, any, will, clearly, documented, such, description, navbar, dropdown, menu, search, toggle, dark, mode,

Text of the page (random words):
template syntax vector documentation docs guides components download blog support observability pipelines toggle dark mode search x 𝕏 github chat rss updates open navbar dropdown menu docs guides components download blog support observability pipelines x 𝕏 github chat rss updates docs home introduction concepts setup quickstart installation package managers apt dpkg helm homebrew msi nix pacman rpm yum platforms docker kubernetes operating systems amazon linux arch linux centos debian macos nixos raspbian rhel ubuntu windows manual from archives from source vector installer deployment roles topologies going to production reference architectures aggregator architecture agent architecture unified architecture architecting hardening high availability rollout sizing and capacity planning architecture data model log events metric events pipeline model runtime model buffering model concurrency model adaptive concurrency guarantees end to end acknowledgements administration management monitoring validating optimization pgo reference vector remap language functions errors examples expressions configuration sources amqp apache metrics aws ecs metrics aws kinesis firehose aws s3 aws sqs datadog agent demo logs dnstap docker logs eventstoredb metrics exec file file descriptor fluent gcp pubsub heroku logplex host metrics http client http server internal logs internal metrics journald kafka kubernetes logs logstash mongodb metrics mqtt nats nginx metrics okta opentelemetry postgresql metrics prometheus pushgateway prometheus remote write prometheus scrape pulsar redis socket splunk hec static metrics statsd stdin syslog vector websocket windows event log transforms remap with vrl aggregate aws ec2 metadata dedupe delay exclusive route filter incremental to absolute log to metric lua metric to log reduce route sample tag cardinality limit throttle trace to log window sinks amqp appsignal aws cloudwatch logs aws cloudwatch metrics aws kinesis data firehose logs aws kinesis streams logs aws s3 aws sns aws sqs axiom azure blob storage azure logs ingestion azure monitor logs blackhole clickhouse console databend databricks zerobus datadog events datadog logs datadog metrics datadog traces doris elasticsearch file gcp chronicle unstructured gcp cloud monitoring formerly stackdriver gcp cloud storage gcp stackdriver gcp pubsub greptimedb logs greptimedb metrics honeycomb http humio logs humio metrics influxdb logs influxdb metrics kafka keep loki mezmo formerly logdna mqtt nats new relic opentelemetry papertrail postgres prometheus exporter prometheus remote write pulsar redis sematext logs sematext metrics socket splunk hec logs splunk hec metrics statsd vector webhdfs websocket websocket server global options pipeline components tls configuration api unit tests schema template syntax secrets cli environment variables api glossary meta security releases versioning vector docs home reference configuration template syntax template syntax vector supports a template syntax for some configuration options this allows for dynamic values derived from event data any option that supports this syntax will be clearly documented as such in the description example let s partition data on aws s3 by application_id and date we can accomplish this with the key_prefix option in the aws_s3 sink yaml toml sinks backup type aws_s3 bucket all_application_logs key_prefix application_id application_id date f sinks backup type aws_s3 bucket all_application_logs key_prefix application_id application_id date f notice that vector allows direct field references as well as strftime specifiers if we were to run the following log event through vector timestamp 2020 02 14t01 22 23 223z application_id 1 message hello world the value of the above key_prefix option would equal application_id 1 date 2020 02 14 because the aws_s3 sink batches data each event would be grouped by its produced value this effectively enables dynamic partitioning something fundamental to storing log data in filesystems syntax event fields individual log event fields can be accessed using to wrap a vrl path expression yaml toml option parent child option parent child strftime specifiers in addition to directly accessing fields vector offers a shortcut for injecting strftime specifiers yaml toml option year y month m day d option year y month m day d the value is derived from the timestamp field and the name of this field can be changed via the global timestamp_key option escaping you can escape this syntax by prefixing the character with a for example you can escape the event field syntax like this yaml toml option field_name option field_name and strftime specified like so yaml toml option year y month m day d option year y month m day d each of the values above would be treated literally how it works accessing fields you can find additional examples for accessing fields in the path expression reference documentation fallback values vector doesn t currently support fallback values issue 1692 is open to add this functionality in the interim you can use the remap transform to set a default value yaml toml transforms set_defaults type remap inputs my source id source if exists my_field my_field default transforms set_defaults type remap inputs my source id source if exists my_field my_field default missing fields if a field is missing an error is logged and vector drops the event the component_errors_total internal metric is incremented with an error_type tag of template_failed on this page close docs slideover panel docs home introduction concepts setup quickstart installation package managers apt dpkg helm homebrew msi nix pacman rpm yum platforms docker kubernetes operating systems amazon linux arch linux centos debian macos nixos raspbian rhel ubuntu windows manual from archives from source vector installer deployment roles topologies going to production reference architectures aggregator architecture agent architecture unified architecture architecting hardening high availability rollout sizing and capacity planning architecture data model log events metric events pipeline model runtime model buffering model concurrency model adaptive concurrency guarantees end to end acknowledgements administration management monitoring validating optimization pgo reference vector remap language functions errors examples expressions configuration sources amqp apache metrics aws ecs metrics aws kinesis firehose aws s3 aws sqs datadog agent demo logs dnstap docker logs eventstoredb metrics exec file file descriptor fluent gcp pubsub heroku logplex host metrics http client http server internal logs internal metrics journald kafka kubernetes logs logstash mongodb metrics mqtt nats nginx metrics okta opentelemetry postgresql metrics prometheus pushgateway prometheus remote write prometheus scrape pulsar redis socket splunk hec static metrics statsd stdin syslog vector websocket windows event log transforms remap with vrl aggregate aws ec2 metadata dedupe delay exclusive route filter incremental to absolute log to metric lua metric to log reduce route sample tag cardinality limit throttle trace to log window sinks amqp appsignal aws cloudwatch logs aws cloudwatch metrics aws kinesis data firehose logs aws kinesis streams logs aws s3 aws sns aws sqs axiom azure blob storage azure logs ingestion azure monitor logs blackhole clickhouse console databend databricks zerobus datadog events datadog logs datadog metrics datadog traces doris elasticsearch file gcp chronicle unstructured gcp cloud monitoring formerly stackdriver gcp cloud storage gcp stackdriver gcp pubsub greptimedb logs greptimedb metrics honeycomb http humio logs humio metrics influxdb logs influxdb metrics kafka keep loki mezmo formerly logdna mqtt nats new relic opentelemetry papertrail postgres prometheus exporter prometheus remote write pulsar redis sematext logs sematext metrics socket splunk hec logs splunk hec metrics statsd vector webhdfs websocket websocket server global options pipeline components tls configuration api unit tests schema template syntax secrets cli environment variables api glossary meta security releases versioning sign up to receive emails on the latest vector content and new releases thank you for joining our updates newsletter previous schema next secrets vector site footer about vector contact us the team privacy cookies components sources transforms sinks setup installation deployment configuration administration going to prod community github x chat download releases x 𝕏 github chat rss 2026 datadog inc all rights reserved sidebar