Meta tags:
description=Read observability data from subjects on the NATS messaging system;
keywords=nats,component,source;

Headings (most frequently used words):

decoding, framing, auth, tls, jetstream, chunked_gelf, length_delimited, lossy, nkey, protobuf, max_length, user_password, avro, gelf, vrl, character_delimited, batch_config, nats, metrics, credentials_file, token, influxdb, json, native_json, syslog, max_frame_length, newline_delimited, octet_counting, deprecated, configuration, outputs, output, types, telemetry, how, it, works, vector, site, footer, example, configurations, connection_name, queue, subject, subject_key_field, subscriber_capacity, url, component_id, traces, logs, context, rs, state, transport, layer, security, about, components, setup, community, download, strategy, codec, signal_types, method, consumer, stream, alpn_protocols, ca_file, crt_file, enabled, key_file, key_pass, server_name, verify_certificate, verify_hostname, warning, record, component_discarded_events_total, component_errors_total, component_received_bytes_total, component_received_event_bytes_total, component_received_events_count, component_received_events_total, component_sent_event_bytes_total, component_sent_events_total, source_buffer_max_byte_size, source_buffer_max_event_size, source_buffer_max_size_bytes, source_buffer_max_size_events, source_buffer_utilization, source_buffer_utilization_level, source_buffer_utilization_mean, source_lag_time_seconds, path, seed, value, password, user, schema, strip_schema_id_prefix, validation, desc_file, message_type, use_json_names, source, timezone, delimiter, decompression, pending_messages_limit, timeout_secs, length_field_is_big_endian, length_field_length, length_field_offset, batch, max_bytes, fields,

Text of the page (most frequently used words):
the (421), optional (123), vector (103), component (73), this (62), nats (55), from (41), data (38), string (38), for (35), #decoding (35), when (34), which (33), literal (31), type (29), output (29), required (28), bytes (27), are (27), metrics (26), and (25), file (25), logs (25), default (24), originated (24), object (23), framing (23), source (22), events (22), name (22), that (22), not (22), tls (21), options (21), message (21), relevant (21), length (21), can (19), hostname (19), host (19), buffer (19), gelf (19), with (18), system (18), maximum (18), process (17), component_id (17), specific (17), number (17), subject (17), examples (17), instance (16), pid (16), running (16), component_type (16), kind (16), component_kind (16), certificate (16), messages (16), byte (16), raw (15), decoder (15), codec (15), configuration (14), used (13), true (13), json (13), format (12), uint (12), protobuf (12), auth (12), aws (12), mode (11), use (11), server (11), set (11), bool (11), path (11), sequences (11), decodes (11), sources (10), uri (10), traces (10), enum (10), invalid (10), utf (10), log (9), more (9), types (9), url (9), method (9), character (9), vrl (9), avro (9), event (8), time (8), outputs (8), demo (8), must (8), key (8), jetstream (8), influxdb (8), token (8), nkey (8), authentication (8), you (7), send (7), metric (7), counter (7), foo (7), memory (7), delimited (7), frames (7), length_delimited (7), chunked_gelf (7), option (7), instead (7), syslog (7), connection_name (7), strategy (7), datadog (6), all (6), components (6), each (6), gauge (6), batch (6), enabled (6), stream (6), connections (6), pem (6), whether (6), order (6), determines (6), pull (6), field (6), limit (6), user_password (6), my_source_id (6), gcp (6), github (5), uses (5), specified (5), local (5), input (5), how (5), level (5), connection (5), either (5), origins (5), note (5), internal (5), see (5), remote (5), only (5), client (5), example (5), value (5), does (5), character_delimited (5), description (5), decompression (5), replaced (5), fffd (5), replacement (5), replace (5), failing (5), lossy (5), otlp (5), strict (5), schema (5), credentials_file (5), prometheus (5), model (5), chat (4), new (4), openssl (4), defaults (4), histogram (4), utilization (4), hold (4), has (4), deprecated (4), total (4), emitted (4), sanitized (4), pod (4), pod_name (4), pathname (4), peer_path (4), peer_addr (4), container (4), container_name (4), than (4), error (4), errors (4), false (4), http (4), fields (4), will (4), may (4), unless (4), outgoing (4), verification (4), valid (4), also (4), absolute (4), incoming (4), information (4), documentation (4), subscriber_capacity (4), subject_key_field (4), octet_counting (4), max_length (4), processing (4), malformed (4), such (4), user (4), unbounded (4), lead (4), usage (4), newline_delimited (4), prefixed (4), chunks (4), signal (4), names (4), native_json (4), native (4), validation (4), spec (4), specification (4), style (4), reference (4), kafka (4), password (4), docs (4), architecture (4), rss (3), releases (3), download (3), going (3), sinks (3), transforms (3), mqtt (3), updates (3), protocols (3), due (3), behavior (3), state (3), seconds (3), using (3), accepted (3), like (3), other (3), dropped (3), record (3), trace (3), localhost (3), key_file (3), der (3), pkcs (3), additionally (3), provided (3), inline (3), additional (3), supported (3), configures (3), 65536 (3), consumer (3), size (3), reached (3), batch_config (3), there (3), being (3), continue (3), include (3), delimiter (3), frame (3), graylog (3), automatically (3), program (3), remap (3), relaxed (3), opentelemetry (3), protocol (3), experimental (3), apache (3), username (3), credentials (3), observability (3), websocket (3), splunk (3), hec (3), azure (3), kinesis (3), administration (2), deployment (2), installation (2), setup (2), nginx (2), receive (2), via (2), environment (2), security (2), stateless (2), its (2), sink (2), context (2), between (2), timestamp (2), mean (2), source_buffer_max_size_events (2), source_buffer_max_size_bytes (2), been (2), favor (2), tagged (2), cumulatively (2), useful (2), performance (2), issues (2), batches (2), passed (2), stage (2), within (2), filter (2), line (2), individual (2), namespacing (2), 4222 (2), takes (2), form (2), port (2), connect (2), understand (2), risks (2), verifying (2), presented (2), enables (2), certificates (2), issuer (2), host_certificate (2), private (2), identify (2), crt_file (2), crt (2), 509 (2), alpn (2), they (2), defined (2), subscriber (2), buffers (2), before (2), capacity (2), underlying (2), u003e (2), east (2), queue (2), request (2), fulfilled (2), max_bytes (2), 200 (2), single (2), follow (2), explicit (2), per (2), settings (2), octet (2), counting (2), risk (2), controlled (2), consider (2), setting (2), reasonably (2), large (2), safety (2), net (2), prevents (2), enforced (2), resource (2), buffered (2), potentially (2), exhaustion (2), extreme (2), cases (2), trailing (2), newline (2), indicating (2), varint_length_delimited (2), according (2), big (2), endian (2), chunked (2), 388608e (2), max_frame_length (2), header (2), timeout (2), received (2), pending (2), payload (2), auto (2), zlib (2), gzip (2), based (2), handles (2), encoded (2), where (2), timezone (2), zone (2), result (2), language (2), deserializer (2), parse (2), systems (2), proto (2), package (2), descriptor (2), strictly (2), rfc (2), decoded (2), sdk (2), loki (2), some (2), following (2), conceptually (2), equivalent (2), seed (2), open (2), acknowledgements (2), aggregator (2), home (2), api (2), pipeline (2), statsd (2), socket (2), sematext (2), redis (2), pulsar (2), write (2), formerly (2), humio (2), greptimedb (2), pubsub (2), stackdriver (2), cloud (2), storage (2), monitoring (2), sqs (2), firehose (2), cloudwatch (2), amqp (2), route (2), windows (2), kubernetes (2), docker (2), agent (2), end (2), concurrency (2), linux (2), pipelines (2), support (2), blog (2), guides (2), sidebar, 2026, inc, rights, reserved, community, prod, cookies, privacy, team, contact, about, site, footer, next, previous, thank, joining, our, newsletter, sign, emails, latest, content, page, maturity, enable, adjust, location, variable, openssl_conf, usr, ssl, cnf, transport, layer, meaning, consistent, across, under, hood, augments, helpful, keys, works, difference, recorded, was, ingested, expressed, fractional, source_lag_time_seconds, smoothed, over, exponentially, weighted, moving, average, ewma, source_buffer_utilization_mean, current, source_buffer_utilization_level, source_buffer_utilization, source_buffer_max_event_size, source_buffer_max_byte_size, component_sent_events_total, component_sent_event_bytes_total, component_received_events_total, separate, batching, mostly, low, debugging, small, topology, component_received_events_count, component_received_event_bytes_total, component_received_bytes_total, occurred, error_type, encountered, component_errors_total, were, discarded, intentionally, transform, intentional, component_discarded_events_total, link, telemetry, source_type, 20308, 401, get, disintermediate, 0400, 2020, oct, 246, 150, 126, shown, below, different, details, warning, downstream, 5222, 6222, 127, 4242, present, common, entry, alternative, extension, verify_hostname, validity, expired, issued, trusted, operates, hierarchical, manner, checking, leaf, but, until, reaches, root, create, requires, have, initiate, requests, validates, upstream, verify_certificate, www, com, indication, sni, server_name, password1, key_pass_env_var, effect, passphrase, unlock, encrypted, key_pass, identity, require, archive, certificate_authority, ca_file, declare, during, negotiation, peer, prioritized, sets, list, alpn_protocols, async_nats, many, group, join, bind, durable, these, crate, streambuilder, async, pulled, expires, after, varint, compatible, encoding, unsigned, bit, integer, chosen, through, boundaries, split, segments, length_field_offset, representing, length_field_length, little, length_field_is_big_endian, fully, drops, timed, out, float, timeout_secs, incomplete, starts, dropping, ensuring, bounded, grow, matches, pending_messages_limit, into, account, excluded, calculation, concatenation, chunk, payloads, composed, multiple, applies, whole, longer, decompress, none, detect, magic, delimits, ascii_char, separated, way, marks, begins, ends, est5edt, america, new_york, apply, conversions, contain, any, indicate, database, execute, final, contents, target, compilation, abort, trying, duplicate, removed, while, preserving, tries, signals, allows, optimize, know, expected, avoid, attempting, first, attempt, parsing, priority, signal_types, working, needs, converted, interfacing, naming, conventions, their, job_description, jobdescription, camelcase, snake_case, use_json_names, serializing, message_type, buf, images, work, protoc, desc, desc_file, closely, follows, treat, violations, allowing, accept, don, skips, checks, passes, them, old, includes, structured, 5424, 3164, three, detects, forward, implementation, means, relax, enforcement, codecs, maintained, much, actual, receiver, adheres, exception, characters, allowed, reason, datum, strip, prefix, described, confluent, strip_schema_id_prefix, timemillis, fixed, duration, decimal, date, definition, variants, apache_avro, encoder, decoders, determine, assigned, jwt, various, strategies, found, specifically, authenticate, public, nkeys, etc, creds, interacting, 65_536, toml, yaml, advanced, minimal, configurations, view, egress, delivery, best, effort, role, status, stable, read, subjects, messaging, versioning, meta, glossary, variables, cli, secrets, template, syntax, unit, tests, global, webhdfs, exporter, postgres, papertrail, relic, mezmo, logdna, keep, honeycomb, chronicle, unstructured, elasticsearch, doris, databricks, zerobus, databend, console, clickhouse, blackhole, monitor, ingestion, blob, axiom, sns, streams, appsignal, window, throttle, tag, cardinality, sample, reduce, lua, incremental, exclusive, delay, dedupe, ec2, metadata, aggregate, stdin, static, scrape, pushgateway, postgresql, okta, mongodb, logstash, journald, heroku, logplex, fluent, exec, eventstoredb, dnstap, ecs, expressions, functions, pgo, optimization, validating, management, guarantees, adaptive, buffering, runtime, sizing, planning, rollout, high, availability, hardening, architecting, unified, architectures, production, topologies, roles, installer, archives, manual, ubuntu, rhel, raspbian, nixos, macos, debian, centos, arch, amazon, operating, platforms, yum, rpm, pacman, nix, msi, homebrew, helm, dpkg, apt, managers, quickstart, concepts, introduction, navbar, dropdown, menu, search, toggle, dark,

Text of the page (random words):
coding native_json lossy optional bool determines whether to replace invalid utf 8 sequences instead of failing when true invalid utf 8 sequences are replaced with the u fffd replacement character default true decoding protobuf optional object protobuf specific decoding options relevant when codec protobuf decoding protobuf desc_file optional string literal the path to the protobuf descriptor set file this file is the output of protoc i include path o desc output path proto for more information see how buf images work decoding protobuf message_type optional string literal the name of the message type to use for serializing examples package message decoding protobuf use_json_names optional bool use json field names camelcase instead of protobuf field names snake_case when enabled the deserializer will output fields using their json names as defined in the proto file for example jobdescription instead of job_description this is useful when working with data that needs to be converted to json or when interfacing with systems that use json naming conventions default false decoding signal_types optional string signal types to attempt parsing in priority order the deserializer tries to parse signals in the specified order this allows you to optimize performance when you know the expected signal types for example if you only receive traces set this to traces to avoid attempting to parse as logs or metrics first if not specified defaults to trying all types in this order logs metrics traces duplicate signal types are automatically removed while preserving order relevant when codec otlp default logs metrics traces decoding syslog optional object syslog specific decoding options relevant when codec syslog decoding syslog lossy optional bool determines whether to replace invalid utf 8 sequences instead of failing when true invalid utf 8 sequences are replaced with the u fffd replacement character default true decoding vrl required object vrl specific decoding options relevant when codec vrl decoding vrl source required string literal the vector remap language vrl program to execute for each event the final contents of the target are used as the decoding result compilation errors or use of abort in the program result in a decoding error decoding vrl timezone optional string literal the name of the timezone to apply to timestamp conversions that do not contain an explicit time zone the time zone name may be any name in the tz database or local to indicate system local time if not set local is used examples local america new_york est5edt framing optional object framing configuration framing handles how events are separated when encoded in a raw byte form where each event is a frame that must be prefixed or delimited in a way that marks where an event begins and ends within the byte stream framing character_delimited required object options for the character delimited decoder relevant when method character_delimited framing character_delimited delimiter required ascii_char the character that delimits byte sequences framing character_delimited max_length optional uint the maximum length of the byte buffer this length does not include the trailing delimiter by default no maximum length is enforced if events are malformed this can lead to additional resource usage as events continue to be buffered in memory and can potentially lead to memory exhaustion in extreme cases if there is a risk of processing malformed data such as logs with user controlled input consider setting the maximum length to a reasonably large value as a safety net this prevents processing from being unbounded framing chunked_gelf optional object options for the chunked gelf decoder relevant when method chunked_gelf framing chunked_gelf decompression optional string literal enum decompression configuration for gelf messages enum options option description auto automatically detect the decompression method based on the magic bytes of the message gzip use gzip decompression none do not decompress the message zlib use zlib decompression default auto framing chunked_gelf max_length optional uint the maximum length of a single gelf message in bytes messages longer than this length are dropped if this option is not set the decoder does not limit the length of messages and the per message memory is unbounded note a message can be composed of multiple chunks and this limit applies to the whole message not to individual chunks this limit takes into account only the message payload gelf header bytes are excluded from the calculation the message payload is the concatenation of all chunk payloads framing chunked_gelf pending_messages_limit optional uint the maximum number of pending incomplete messages if this limit is reached the decoder starts dropping chunks of new messages ensuring the memory usage of the decoder s state is bounded if this option is not set the decoder does not limit the number of pending messages and the memory usage of its messages buffer can grow unbounded this matches graylog server s behavior framing chunked_gelf timeout_secs optional float the timeout in seconds for a message to be fully received if the timeout is reached the decoder drops all received chunks for the timed out message default 5 framing length_delimited required object options for the length delimited decoder relevant when method length_delimited framing length_delimited length_field_is_big_endian optional bool length field byte order little or big endian default true framing length_delimited length_field_length optional uint number of bytes representing the field length default 4 framing length_delimited length_field_offset optional uint number of bytes in the header before the length field framing length_delimited max_frame_length optional uint maximum frame length default 8 388608e 06 framing max_frame_length optional uint maximum frame length relevant when method varint_length_delimited default 8 388608e 06 framing method optional string literal enum the framing method enum options option description bytes byte frames are passed through as is according to the underlying i o boundaries for example split between messages or stream segments character_delimited byte frames which are delimited by a chosen character chunked_gelf byte frames which are chunked gelf messages length_delimited byte frames which are prefixed by an unsigned big endian 32 bit integer indicating the length newline_delimited byte frames which are delimited by a newline character octet_counting byte frames according to the octet counting format varint_length_delimited byte frames which are prefixed by a varint indicating the length this is compatible with protobuf s length delimited encoding default bytes framing newline_delimited optional object options for the newline delimited decoder relevant when method newline_delimited framing newline_delimited max_length optional uint the maximum length of the byte buffer this length does not include the trailing delimiter by default no maximum length is enforced if events are malformed this can lead to additional resource usage as events continue to be buffered in memory and can potentially lead to memory exhaustion in extreme cases if there is a risk of processing malformed data such as logs with user controlled input consider setting the maximum length to a reasonably large value as a safety net this prevents processing from being unbounded framing octet_counting optional object options for the octet counting decoder relevant when method octet_counting framing octet_counting max_length optional uint the maximum length of the byte buffer jetstream optional object configuration for nats jetstream jetstream batch_config optional object batch settings for a jetstream pull consumer by default messages are pulled in batches of up to 200 each pull request expires after 30 seconds if not fulfilled there is no explicit maximum byte size per batch unless specified note these defaults follow the async nats crate s streambuilder jetstream batch_config batch optional uint the maximum number of messages to pull in a single batch default 200 jetstream batch_config max_bytes optional uint the maximum total byte size for a batch the pull request will be fulfilled when either size or max_bytes is reached jetstream consumer required string literal the name of the durable consumer to pull from jetstream stream required string literal the name of the stream to bind to queue optional string literal the nats queue group to join subject required string literal the nats subject to pull messages from examples foo time us east time east time u003e u003e subject_key_field optional string literal the nats subject key default subject subscriber_capacity optional uint the buffer capacity of the underlying nats subscriber this value determines how many messages the nats subscriber buffers before incoming messages are dropped see the async_nats documentation for more information default 65536 tls optional object configures the tls options for incoming outgoing connections tls alpn_protocols optional string sets the list of supported alpn protocols declare the supported alpn protocols which are used during negotiation with a peer they are prioritized in the order that they are defined tls ca_file optional string literal absolute path to an additional ca certificate file the certificate must be in the der or pem x 509 format additionally the certificate can be provided as an inline string in pem format examples path to certificate_authority crt tls crt_file optional string literal absolute path to a certificate file used to identify this server the certificate must be in der pem x 509 or pkcs 12 format additionally the certificate can be provided as an inline string in pem format if this is set and is not a pkcs 12 archive key_file must also be set examples path to host_certificate crt tls enabled optional bool whether to require tls for incoming or outgoing connections when enabled and used for incoming connections an identity certificate is also required see tls crt_file for more information tls key_file optional string literal absolute path to a private key file used to identify this server the key must be in der or pem pkcs 8 format additionally the key can be provided as an inline string in pem format examples path to host_certificate key tls key_pass optional string literal passphrase used to unlock the encrypted key file this has no effect unless key_file is set examples key_pass_env_var password1 tls server_name optional string literal server name to use when using server name indication sni only relevant for outgoing connections examples www example com tls verify_certificate optional bool enables certificate verification for components that create a server this requires that the client connections have a valid client certificate for components that initiate requests this validates that the upstream has a valid certificate if enabled certificates must not be expired and must be issued by a trusted issuer this verification operates in a hierarchical manner checking that the leaf certificate the certificate presented by the client server is not only valid but that the issuer of that certificate is also valid and so on until the verification process reaches a root certificate do not set this to false unless you understand the risks of not verifying the validity of certificates tls verify_hostname optional bool enables hostname verification if enabled the hostname used to connect to the remote host must be present in the tls certificate presented by the remote host either as the common name or as an entry in the subject alternative name extension only relevant for outgoing connections do not set this to false unless you understand the risks of not verifying the remote hostname url required string literal the nats url to connect to the url takes the form of nats server port if the port is not specified it defaults to 4222 examples nats demo nats io nats 127 0 0 1 4242 nats localhost 4222 nats localhost 5222 nats localhost 6222 outputs component_id default output stream of the component use this component s id as an input to downstream transforms and sinks output types metrics metric events that may be emitted by this source traces trace events that may be emitted by this source logs warning the fields shown below will be different if log namespacing is enabled see log namespacing for more details record an individual nats record fields message required string literal the raw line from the nats message examples 53 126 150 246 01 oct 2020 11 25 58 0400 get disintermediate http 2 0 401 20308 source_type required string literal the name of the source type examples nats subject required string literal the subject from the nats message examples nats subject telemetry metrics link component_discarded_events_total counter the number of events dropped by this component component_id the vector component id component_kind the vector component kind component_type the vector component type host optional the hostname of the system vector is running on intentional true if the events were discarded intentionally like a filter transform or false if due to an error pid optional the process id of the vector instance component_errors_total counter the total number of errors encountered by this component component_id the vector component id component_kind the vector component kind component_type the vector component type error_type the type of the error host optional the hostname of the system vector is running on pid optional the process id of the vector instance stage the stage within the component at which the error occurred component_received_bytes_total counter the number of raw bytes accepted by this component from source origins component_id the vector component id component_kind the vector component kind component_type the vector component type container_name optional the name of the container from which the data originated file optional the file from which the data originated host optional the hostname of the system vector is running on mode optional the connection mode used by the component peer_addr optional the ip from which the data originated peer_path optional the pathname from which the data originated pid optional the process id of the vector instance pod_name optional the name of the pod from which the data originated uri optional the sanitized uri from which the data originated component_received_event_bytes_total counter the number of event bytes accepted by this component either from tagged origins like file and uri or cumulatively from other origins component_id the vector component id component_kind the vector component kind component_type the vector component type container_name optional the name of the container from which the data originated file optional the file from which the data originated host optional the hostname of the system vector is running on mode optional the connection mode u...