Meta tags:
description=How Rybbit identifies and filters bot traffic from your analytics;
author=Rybbit Team;
keywords=web analytics,privacy analytics,open source analytics,Google Analytics alternative,website tracking,self-hosted analytics;

Headings (most frequently used words):

detection, bot, and, what, signals, enabling, happens, to, detected, bots, layers, how, decisions, are, made, server, side, tracking, proxies, cdn, setups, does, not, guarantee, user, agent, patterns, header, heuristics, client, asn, network, rate, anomaly, on, this, page,

Text of the page (most frequently used words):
bot (39), the (28), and (26), #detection (18), traffic (15), rybbit (15), layer (15), browser (14), user (13), not (12), signals (12), agent (12), for (12), request (11), are (9), asn (9), from (9), layers (8), can (8), your (8), proxy (8), requests (8), this (7), site (7), side (6), tracking (6), analytics (6), visitor (6), api (6), many (6), what (5), server (5), events (5), normal (5), headers (5), when (5), one (5), too (5), hosting (5), settings (5), does (4), client (4), patterns (4), detected (4), bots (4), filter (4), may (4), but (4), you (4), automation (4), blocking (4), matches (4), event (4), that (4), missing (4), block (4), script (4), guides (4), cdn (3), how (3), rate (3), anomaly (3), network (3), header (3), heuristics (3), happens (3), track (3), monitoring (3), like (3), users (3), through (3), uses (3), multiple (3), still (3), than (3), all (3), troubleshooting (3), also (3), same (3), every (3), records (3), agents (3), such (3), evidence (3), asns (3), include (3), framework (3), headless (3), usage (3), self (3), documentation (3), guarantee (2), proxies (2), setups (2), decisions (2), made (2), enabling (2), page (2), visits (2), goals (2), address (2), some (2), look (2), legitimate (2), browse (2), unusual (2), paths (2), counts (2), rather (2), measurement (2), examples (2), guide (2), sent (2), location (2), provider (2), original (2), where (2), real (2), another (2), keys (2), key (2), treated (2), use (2), because (2), match (2), per (2), each (2), first (2), runs (2), final (2), decision (2), crawlers (2), scores (2), cloud (2), while (2), other (2), supporting (2), generic (2), known (2), metadata (2), identify (2), own (2), lists (2), chrome (2), apis (2), suspicious (2), window (2), dimensions (2), seo (2), crawler (2), search (2), before (2), matched (2), filters (2), data (2), dashboard (2), funnels (2), open (2), identifies (2), feature (2), 687 (2), github (2), conversions, custom, exclude, country, path, hostname, sophisticated, browsers, constrained, environments, reduce, both, misses, false, positives, should, interpret, operational, signal, exact, improves, quality, perfect, see, instead, geolocated, inherit, preserve, possible, accept, language, referer, forwarded, cloudflare, workers, aws, cloudfront, nginx, caddy, reverse, forward, expose, javascript, only, with, valid, trusted, ingestion, bypass, backend, generated, coming, authorization, bearer, add, more, total, number, pattern, marked, least, resulting, includes, boolean, fields, stop, matching, then, makes, designed, catch, fast, floods, replayed, have, obvious, fingerprint, large, volumes, high, wide, volume, hostnames, different, distinct, visited, quickly, tracks, short, rolling, windows, watches, bursty, crawl, shaped, behavior, rate_anomaly, avoids, filtering, who, vpn, corporate, gateway, party, preserving, context, exists, they, recorded, alone, enough, datacenter, scanner, internet, providers, trigger, directly, curated, there, two, kinds, resolved, bot_asn, combines, these, into, weighted, score, strong, their, weaker, contribute, empty, plugin, swiftshader, renderer, globals, outer, default, viewport, sizes, 1024x768, 800x600, zero, impossible, lightweight, collected, client_signals, useful, send, full, set, normally, sends, style, stale, versions, looking, inconsistent, claims, fetch, looks, consistency, header_heuristics, http, clients, tools, engine, checks, against, tool, social, preview, ua_pattern, making, stores, compact, record, inspect, was, filtered, route, device, which, count, toward, billable, charged, visit, added, keeps, totals, reports, journeys, session, metrics, focused, human, setting, off, based, enable, configured, reaches, enabled, incoming, checked, several, any, excluded, tables, stored, separately, inspection, copy, markdown, definitions, import, inviting, billing, teams, organization, account, migration, managing, installation, quick, start, embeds, hiding, sdks, integration, tagging, autocapture, hosted, architecture, comparison, introduction, setup, reference,

Text of the page (random words):
bot detection rybbit search k github rybbit io rybbit 12 4k 687 documentation api reference documentation guides setup and feature documentation github rybbit io rybbit 12 4k 687 introduction comparison architecture self hosted vs cloud usage tracking script troubleshooting track events autocapture identify users tagging integration guides proxy guide sdks hiding your own traffic filter traffic bot detection goals funnels embeds feature guides self hosting quick start managing your installation troubleshooting self hosting guides migration to v1 0 settings account settings organization settings teams billing inviting users site settings data import other definitions bot detection bot detection how rybbit identifies and filters bot traffic from your analytics copy markdown open rybbit can filter bot traffic before it reaches your normal analytics when block bot traffic is enabled for a site each incoming tracking request is checked by several detection layers if any layer identifies the request as bot traffic the request is excluded from the normal analytics tables and stored separately for bot traffic inspection enabling bot detection bot blocking is configured per site open your site in the rybbit dashboard go to site settings enable block bot traffic when this setting is off rybbit does not block requests based on bot detection what happens to detected bots detected bot requests are not added to your normal analytics data this keeps dashboard totals reports journeys funnels session lists and usage metrics focused on human traffic detected bot visits also do not count toward billable analytics usage if bot blocking filters a request you are not charged for that bot visit rybbit still stores a compact bot event record so you can inspect what was filtered bot event records include route device location asn and which detection layers matched detection layers rybbit runs all detection layers before making a final decision a request can match multiple layers and the bot event records every layer that matched user agent patterns the ua_pattern layer checks the request user agent against known bot crawler ai agent seo tool monitoring social preview framework and headless browser patterns examples include search engine crawlers headless browser user agents ai crawler and agent user agents seo and monitoring tools script or framework http clients header heuristics the header_heuristics layer scores request headers for browser consistency it looks for signals such as missing browser headers suspicious fetch metadata inconsistent browser claims headless or automation looking headers stale or unusual chrome versions script framework style requests that do not look like normal browser traffic this layer is useful because many bots use a browser like user agent but do not send the full set of headers a real browser normally sends client signals the client_signals layer uses lightweight browser side signals collected by the tracking script signals include automation apis zero or impossible window dimensions default automation viewport sizes such as 800x600 and 1024x768 suspicious outer window dimensions missing browser apis missing chrome globals swiftshader renderer signals empty plugin lists rybbit combines these into a weighted score strong signals can identify bot traffic on their own while weaker signals contribute supporting evidence asn and network signals the bot_asn layer uses asn metadata from the resolved ip address there are two kinds of asn matches curated bot provider asns known ai scanner and internet measurement providers can trigger bot detection directly generic hosting datacenter asns hosting asns are treated as supporting evidence they are recorded when another bot layer also matches but generic hosting asn alone is not enough to block a request this avoids filtering every legitimate visitor who happens to browse through a cloud cdn vpn corporate gateway or first party proxy while still preserving asn context when other bot evidence exists rate and anomaly detection the rate_anomaly layer watches for bursty or crawl shaped behavior it tracks short rolling windows for patterns such as too many events from the same ip and user agent too many events from the same ip too many distinct paths visited quickly too many different user agents from one ip too many hostnames from one ip high site wide volume from one user agent large volumes of requests missing client side bot scores this layer is designed to catch fast crawlers floods and replayed tracking requests that may not have obvious user agent or browser fingerprint signals how decisions are made rybbit does not stop at the first matching layer it runs every layer records all matches and then makes one final decision a request is marked as bot traffic when at least one blocking layer matches the resulting bot event includes boolean fields for each layer user agent pattern header heuristics client signals bot asn rate anomaly because multiple layers can match the same request per layer bot counts can add up to more than the total number of bot requests server side tracking requests sent to api track with authorization bearer api key and a valid api key are treated as trusted server side ingestion and bypass bot blocking use this for backend generated events where the request is coming from your server rather than a visitor s browser do not expose api keys in browser javascript api keys are only for server side requests proxies and cdn setups if you proxy rybbit through cloudflare workers aws cloudfront nginx caddy or another reverse proxy forward the original visitor ip x forwarded for visitor ip x real ip visitor ip also preserve the original user agent referer and accept language headers where possible if the proxy ip is sent instead of the visitor ip traffic may be geolocated to the proxy location and may inherit the proxy provider s asn see the proxy troubleshooting guide for examples what bot detection does not guarantee bot detection improves analytics quality but no bot filter is perfect some sophisticated bots can look like normal browsers some legitimate users may browse through unusual network paths or constrained browser environments rybbit uses multiple layers to reduce both misses and false positives but you should still interpret bot counts as an operational signal rather than an exact measurement of all automation on your site filter traffic exclude traffic by ip address country path hostname or user agent goals track conversions by monitoring page visits and custom events on this page enabling bot detection what happens to detected bots detection layers user agent patterns header heuristics client signals asn and network signals rate and anomaly detection how decisions are made server side tracking proxies and cdn setups what bot detection does not guarantee