Meta tags:
description=This file is used to define YAML models. For more information on our SQL models, see the SQL models documentation.;

Headings (most frequently used words):

connector, properties, additional, when, is, or, named, of, path, region, uri, extract, glob, batch_size, common, for, output, athena, azure, bigquery, duckdb, gcs, local_file, redshift, s3, type, name, refs, dev, prod, output_location, workgroup, format, models, yaml, clickhouse, examples, refresh, sql, pre_exec, post_exec, create_secrets_from_connectors, retry, timeout, incremental, change_mode, state, partitions, tests, materialize, partitions_watermark, partitions_concurrency, stage, columns, engine_full, engine, order_by, partition_by, primary_key, sample_by, ttl, table_settings, query_settings, distributed_settings, distributed_sharding_key, dictionary_source_user, dictionary_source_password, account, project_id, invalidate_on_change, database, cluster_identifier, role_arn, endpoint,

Text of the page (most frequently used words):
the (155), string (137), connector (72), for (67), object (52), glob (46), sql (45), path (37), when (36), rill (31), query (31), properties (30), from (29), model (29), table (28), data (26), #output (24), yaml (23), use (23), partition (23), and (22), required (22), duckdb (20), source (20), models (20), additional (19), named (19), name (19), type (19), partitions (19), files (19), iso (18), option (18), are (17), 8601 (17), format (16), integer (16), project (16), used (16), analysis (16), clickhouse (15), file (15), array (15), this (15), optional (15), time (15), with (15), can (14), settings (14), engine (13), incremental (13), end (13), boolean (13), list (13), only (13), start (13), select (12), not (12), resource (12), expression (12), that (12), period (12), api (12), pattern (12), directory (11), uri (10), gcs (10), true (10), maximum (10), matched (10), queries (10), run (10), refers (10), region (9), redshift (9), state (9), view (9), range (9), size (9), timestamp (9), custom (9), columns (8), metrics (8), defined (8), number (8), results (8), dictionary (8), clause (8), include (8), exists (8), postgres (8), test (8), each (8), comparison (8), level (8), ttl (7), create_secrets_from_connectors (7), post_exec (7), pre_exec (7), refresh (7), staging (7), key (7), value (7), user (7), olap (7), assert (7), match (7), batch_size (6), extract (6), workgroup (6), bigquery (6), azure (6), athena (6), materialize (6), where (6), objects (6), default (6), based (6), using (6), applicable (6), rows (6), prompt (6), resolver (6), configuration (6), uses (6), hive (6), directories (6), bound (6), store (6), database (5), tests (5), retry (5), into (5), days (5), parquet (5), page (5), listing (5), matching (5), aws (5), will (5), should (5), before (5), after (5), main (5), whether (5), returns (5), explore (5), how (5), you (5), defines (5), specifies (5), advanced (5), raw (5), against (5), output_location (4), local_file (4), prod (4), dev (4), refs (4), common (4), partition_by (4), stage (4), day (4), all (4), public (4), cron (4), processed (4), overrides (4), any (4), environment (4), inferred (4), password (4), distributed (4), order (4), definition (4), names (4), my_s3_connector (4), allows (4), postgres_db (4), ensure (4), idempotent (4), statements (4), row (4), define (4), passes (4), cannot (4), combined (4), condition (4), measures (4), dimensions (4), dashboard (4), context (4), offset (4), p7d (4), iso_offset (4), duration (4), iso_duration (4), last (4), month (4), provided (4), insights (4), available (4), union (4), metrics_sql (4), status (4), want (4), lower (4), filtering (4), paths (4), than (4), considered (4), options (4), oneof (4), targets (4), executes (4), attempts (4), schedule (4), url (4), endpoint (3), account (3), dictionary_source_password (3), dictionary_source_user (3), distributed_sharding_key (3), distributed_settings (3), query_settings (3), table_settings (3), sample_by (3), primary_key (3), order_by (3), engine_full (3), timeout (3), previous (3), snowflake (3), temporary (3), events (3), current (3), generate (3), append (3), incremental_strategy (3), rilldata (3), analytics (3), github (3), partitioned (3), max_date (3), batch (3), 100mb (3), page_size (3), listed (3), max_objects_listed (3), max_objects_matched (3), total (3), bytes (3), max_total_size (3), related (3), pairs (3), extraction (3), access (3), location (3), changes (3), production (3), development (3), specified (3), create (3), specific (3), full (3), column (3), executing (3), their (3), set (3), over (3), guide (3), error (3), partitioning (3), mode (3), simple (3), your (3), delay (3), execution (3), complex (3), syntax (3), policy (2), service (2), examples (2), role_arn (2), cluster_identifier (2), invalidate_on_change (2), project_id (2), partitions_concurrency (2), partitions_watermark (2), change_mode (2), ingest (2), event_time (2), read_parquet (2), commits_ (2), null (2), 2025 (2), max (2), date (2), identifier (2), csv (2), json (2), references (2), usually (2), filename (2), but (2), manually (2), accessing (2), sharding (2), insert (2), sample (2), primary (2), mergetree (2), keys (2), etc (2), types (2), also (2), indexes (2), unspecified (2), detected (2), my_other_s3_connector (2), which (2), secrets (2), created (2), applies (2), cloud (2), storage (2), credentials (2), connectors (2), detach (2), attach (2), dbname (2), host (2), localhost (2), port (2), 5432 (2), merge (2), strategy (2), updates (2), my_bucket (2), zero (2), applied (2), violate (2), must (2), analyze (2), constrain (2), compare (2), week (2), comparison_time_range (2), p1d (2), yesterday (2), p1m (2), time_range (2), generating (2), automated (2), reports (2), definitions (2), whose (2), single (2), result (2), invokes (2), multiple (2), resolvers (2), entry (2), indicates (2), trigger (2), where_error (2), resource_status (2), transform (2), resolved (2), referenced (2), transform_sql (2), includes (2), rollup_files (2), controls (2), grouped (2), returned (2), example (2), deprecated (2), instead (2), directly (2), matches (2), groups (2), extracts (2), style (2), values (2), sets (2), nth (2), lexicographically (2), sorted (2), successfully (2), point (2), included (2), upper (2), exclusive (2), less (2), inclusive (2), greater (2), equal (2), scheme (2), specifying (2), patternwith (2), arguments (2), pass (2), args (2), calls (2), compute (2), running (2), existing (2), manual (2), new (2), logic (2), without (2), processing (2), if_error_matches (2), exponential_backoff (2), transformations (2), cli (2), reference (2), parameters (2), 2026, inc, contributing, community, terms, privacy, next, produce, final, requires, bucket, temp, since, vice, versa, date_trunc, belonging, don, reload, previously, ingested, every, interval, now, timestamptz, 2024, 1st, jan, today, deployment, arn, iam, role, assume, cluster, serverless, stored, invalidated, arbitrary, connect, uniquely, identify, unique_key, partition_overwrite, my_staging_table, case, input, does, support, direct, write, concurrent, read, same, customizable, check, has, been, updated, materialized, my_model, fail, count, row_count_check, no_null_ids, revenue, assert_positive_revenue, defaults, representing, failures, satisfy, unique, quality, have, either, num, explicitly, configure, specifications, reset, drop, recreate, automatically, require, apply, patch, switch, historical, modeling, wait, ingestion, connection, refused, bad, gateway, overcommittracker, 10s, messages, exponential, backoff, between, needed, setting, explicit, run_in_dev, dependency, ref_update, disables, deleting, disable, zone, interpret, utc, america, los_angeles, time_zone, follow, update, underlying, give, fine, grained, control, transformed, making, them, ideal, workloads, pipelines, formats, destinations, intermediate, tables, pre, post, hooks, process, changed, efficiently, optimize, performance, strategies, both, regular, specification, described, while, perfect, provide, capabilities, scenarios, tip, more, information, our, see, documentation, home, release, notes, whoami, version, validate, usergroup, upgrade, uninstall, token, org, logout, login, init, env, docs, deploy, chat, billing, extensions, component, theme, report, alert, canvas, search, blog, contact, rest, developers, skip, content,

Text of the page (random words):
ssion string rill time expression for comparison period context object context to constrain the ai analysis explore string name of the explore dashboard to analyze dimensions array of string list of dimensions to include in analysis measures array of string list of measures to include in analysis state sql select max date as max_date partitions oneof refers to the how your data is partitioned cannot be used with state optional option 1 object executes a raw sql query against the project s data models sql string raw sql query to run against existing models in the project required connector string specifies the connector to use when running sql or glob queries option 2 object executes a sql query that targets a defined metrics view metrics_sql string sql query that targets a metrics view in the project required option 3 object calls a custom api defined in the project to compute data api string name of a custom api defined in the project required args object arguments to pass to the custom api option 4 object uses a file matching pattern glob to query data from a connector glob oneof simple path glob pattern or path glob patternwith advanced options required option 1 string glob pattern used to match files or directories in the object store option 2 object configuration for specifying a file path glob pattern with advanced options connector string specifies the object store connector to use e g s3 gcs if not provided it is inferred from the scheme of the path path string glob pattern used to match files or directories in the object store required start string defines the lower bound inclusive for partition filtering only partitions with paths greater than or equal to this value are considered end string defines the upper bound exclusive for partition filtering only partitions with paths less than this value are considered last integer sets a lower bound based on the nth partition from the end of the lexicographically sorted successfully processed partitions only partitions after this point are included partition string controls how matched files are grouped file default each matched path is returned as a row use the glob pattern to match files or directories at the level you want for example file level or directory level directory this mode is deprecated instead use file with a glob that directly matches the directory level you want hive groups files by directory and extracts hive style partition values from the path as columns rollup_files boolean if true includes a files array listing all files in each partition only applicable when using directory or hive partitioning transform_sql string optional duckdb sql query used to transform the results the resolved data is available as a table referenced using table option 5 object uses the status of a resource as data resource_status object based on resource status required where_error boolean indicates whether the condition should trigger when the resource is in an error state option 6 object invokes multiple resolvers and returns the union of their results each entry in the list is a resolver definition e g sql glob metrics_sql api union array of object list of resolver definitions whose results are combined into a single result set required option 7 object uses ai to generate insights and analysis from metrics data only available for reports ai object ai resolver configuration for generating automated insights required prompt string custom prompt to guide the ai analysis if not provided a default analysis prompt is used time_range object time range for the analysis period iso_duration string iso 8601 duration e g p7d for 7 days p1m for 1 month iso_offset string iso 8601 offset from current time e g p1d to start from yesterday start string start timestamp in iso 8601 format end string end timestamp in iso 8601 format expression string rill time expression e g last 7 days this month comparison_time_range object optional comparison time range for period over period analysis iso_duration string iso 8601 duration for comparison period iso_offset string iso 8601 offset for comparison period e g p7d to compare with previous week start string start timestamp in iso 8601 format end string end timestamp in iso 8601 format expression string rill time expression for comparison period context object context to constrain the ai analysis explore string name of the explore dashboard to analyze dimensions array of string list of dimensions to include in analysis measures array of string list of measures to include in analysis partitions glob gcs my_bucket y m d parquet partitions connector duckdb sql select range as num from range 0 10 tests array of object define data quality tests for the model each test must have a name and either an assert expression or a sql query an assert test passes when no rows violate the condition a sql test passes when the query returns zero rows name string a unique name for the test required assert string a sql boolean expression applied to each row of the model the test passes if no rows violate the condition i e all rows satisfy assert cannot be combined with sql sql string a sql query that returns rows representing test failures the test passes if the query returns zero rows cannot be combined with assert connector string the connector to use when executing the test query defaults to the model s connector tests name assert_positive_revenue assert revenue 0 name no_null_ids assert id is not null tests name row_count_check sql select fail where select count from my_model 0 materialize boolean models will be materialized in olap partitions_watermark string refers to a customizable timestamp that can be set to check if an object has been updated optional partitions_concurrency integer refers to the number of concurrent partitions that can be read at the same time optional stage object in the case of staging models where an input source does not support direct write to the output and a staging table is required connector string refers to the connector type for the staging table required path string refers to the path to the staging table stage connector s3 path s3 my_bucket my_staging_table output object to define the properties of output table string name of the output table if not specified the model name is used materialize boolean whether to materialize the model as a table or view connector string refers to the connector type for the output table can be clickhouse or duckdb and their named connector incremental_strategy string strategy to use for incremental updates can be append merge or partition_overwrite unique_key array of string list of columns that uniquely identify a row for merge strategy partition_by string column or expression to partition the table by pre_exec string sql query to run on the output olap engine duckdb or clickhouse before the main query optional ensure pre_exec queries are idempotent use if not exists statements when applicable pre_exec attach if not exists dbname postgres host localhost port 5432 user postgres password postgres as postgres_db type postgres post_exec string sql query to run on the output olap engine duckdb or clickhouse after the main query optional ensure post_exec queries are idempotent use if exists statements when applicable post_exec detach database if exists postgres_db create_secrets_from_connectors string array list of connector names for which temporary secrets should be created on the output olap engine before executing the sql only applies when the output connector is duckdb this allows duckdb based models to access cloud storage s3 gcs azure using credentials from named connectors create_secrets_from_connectors my_s3_connector create_secrets_from_connectors my_s3_connector my_other_s3_connector additional properties for output when connector is clickhouse type string type to materialize the model into can be table view or dictionary columns string column names and types can also include indexes if unspecified detected from the query engine_full string full engine definition in sql format can include partition keys order ttl etc engine string table engine to use default is mergetree order_by string order by clause partition_by string partition by clause primary_key string primary key clause sample_by string sample by clause ttl string ttl settings for the table or columns table_settings string table specific settings query_settings string settings used in insert create table as select queries distributed_settings string settings for distributed table distributed_sharding_key string sharding key for distributed table dictionary_source_user string user for accessing the source dictionary table used if type is dictionary dictionary_source_password string password for the dictionary source user common properties name string name is usually inferred from the filename but can be specified manually refs array of string list of resource references dev object overrides any properties in development environment prod object overrides any properties in production environment additional properties for output when connector is clickhouse type string type to materialize the model into can be table view or dictionary columns string column names and types can also include indexes if unspecified detected from the query engine_full string full engine definition in sql format can include partition keys order ttl etc engine string table engine to use default is mergetree order_by string order by clause partition_by string partition by clause primary_key string primary key clause sample_by string sample by clause ttl string ttl settings for the table or columns table_settings string table specific settings query_settings string settings used in insert create table as select queries distributed_settings string settings for distributed table distributed_sharding_key string sharding key for distributed table dictionary_source_user string user for accessing the source dictionary table used if type is dictionary dictionary_source_password string password for the dictionary source user common properties name string name is usually inferred from the filename but can be specified manually refs array of string list of resource references dev object overrides any properties in development environment prod object overrides any properties in production environment additional properties when connector is athena or named connector for athena output_location string output location for query results in s3 workgroup string aws athena workgroup to use for queries region string aws region to connect to athena and the output location additional properties when connector is azure or named connector of azure path string path to the source account string account identifier uri string source uri extract object arbitrary key value pairs for extraction settings glob object settings related to glob file matching max_total_size integer maximum total size in bytes matched by glob max_objects_matched integer maximum number of objects matched by glob max_objects_listed integer maximum number of objects listed in glob page_size integer page size for glob listing batch_size string size of a batch e g 100mb additional properties when connector is bigquery or named connector of bigquery project_id string id of the bigquery project additional properties when connector is duckdb or named connector of duckdb path string path to the data source format string format of the data source e g csv json parquet additional properties when connector is gcs or named connector of gcs path string path to the source uri string source uri extract object key value pairs for extraction settings glob object settings related to glob file matching max_total_size integer maximum total size in bytes matched by glob max_objects_matched integer maximum number of objects matched by glob max_objects_listed integer maximum number of objects listed in glob page_size integer page size for glob listing batch_size string size of a batch e g 100mb additional properties when connector is local_file or named connector of local_file path string path to the data source format string format of the data source e g csv json parquet invalidate_on_change boolean when true the model will be invalidated and re processed if the source file changes additional properties when connector is redshift or named connector of redshift output_location string s3 location where query results are stored workgroup string redshift serverless workgroup to use database string name of the redshift database cluster_identifier string identifier of the redshift cluster role_arn string arn of the iam role to assume for redshift access region string aws region of the redshift deployment additional properties when connector is s3 or named connector of s3 region string aws region endpoint string aws endpoint path string path to the source uri string source uri extract object key value pairs for extraction settings glob object settings related to glob file matching max_total_size integer maximum total size in bytes matched by glob max_objects_matched integer maximum number of objects matched by glob max_objects_listed integer maximum number of objects listed in glob page_size integer page size for glob listing batch_size string size of a batch e g 100mb examples incremental model type model incremental true connector bigquery state sql select max date as max_date sql select from events n if incremental n where event_time state max_date n end n output connector duckdb partitioned model type model partitions glob connector gcs path gs rilldata public github analytics clickhouse 2025 commits_ parquet sql select from read_parquet partition uri output connector duckdb incremental_strategy append partitioned incremental model type model incremental true refresh cron 0 8 partitions glob path gs rilldata public github analytics clickhouse 2025 partition directory sql select n from read_parquet gs rilldata public partition path commits_ parquet n where partition path is not null n output connector duckdb incremental_strategy append staging model type model connector snowflake use duckdb to generate a range of days from 1st jan to today partitions connector duckdb sql select range as day from range timestamptz 2024 01 01 now interval 1 day don t reload previously ingested partitions on every refresh incremental true query snowflake for all events belonging to the current partition sql select from events where date_trunc day event_time partition day since clickhouse can t ingest from snowflake or vice versa we use s3 as a temporary staging connector stage connector s3 path s3 bucket temp data produce the final output into clickhouse requires a clickhouse yaml connector defined output connector clickhouse previous source yaml next metrics view yaml properties type refresh connector sql pre_exec post_exec create_secrets_from_connectors retry timeout incremental change_mode state partitions tests materialize partitions_watermark partit...