Custom Observability Destinations

Send a copy of your usage logs to an observability endpoint or datastore you control, alongside Edgee’s own pipeline.

Available on hosted gateways only. ☁️ Self-hosted deployments configure their own OTLP usage-export target directly. See Kubernetes & Helm.

Supported protocols

Protocol	Transport	Status
OTLP	gRPC or HTTP	✅
S3	Batched NDJSON	✅
ClickHouse	Native protocol, batched `INSERT ... FORMAT JSONEachRow`	✅
BigQuery	Streaming inserts	✅
Snowflake	SQL REST API, batched INSERT	✅

Setup

From Org settings → Integrations (Infrastructure group), add one or more destinations. Pick a destination type, then fill in its fields.

OTLP

Field	Purpose
Name	Label shown in the destinations table
Protocol	gRPC or HTTP
URL	Your endpoint
Headers	Custom key/value pairs, e.g. a bearer token for auth
Active	Toggles forwarding on or off without deleting the destination

S3

Field	Purpose
Name	Label shown in the destinations table
Bucket	Destination bucket name
Region	Bucket region
Endpoint	Only needed for an S3-compatible provider other than AWS (e.g. MinIO, R2, Wasabi)
Key prefix	Optional path prefix applied to every object written
Access key ID / Secret access key	Static credentials with write access to the bucket
Active	Toggles forwarding on or off without deleting the destination

ClickHouse

Field	Purpose
Name	Label shown in the destinations table
Host	Your ClickHouse server hostname
Port	Optional; defaults to 9440 (TLS) or 9000 (no TLS) if left blank
Database	Destination database name
Table	Destination table name (must already exist)
Username / Password	Credentials with insert access to the table
Secure	Toggles TLS for the connection (on by default)
Active	Toggles forwarding on or off without deleting the destination

The table must already exist before you add the destination: Edgee never creates or migrates schema on your instance. Every row is sent via INSERT ... FORMAT JSONEachRow, so the table needs a column for each field Edgee sends, with matching names and compatible types.

Full table definition

CREATE TABLE IF NOT EXISTS your_database.usage_logs
(
    timestamp DateTime64(9),
    request_id String DEFAULT '',
    provider String,
    model String,
    input_tokens UInt32,
    output_tokens UInt32,
    cached_input_tokens UInt32,
    cache_creation_input_tokens UInt32 DEFAULT 0,
    reasoning_output_tokens UInt32,
    input_cost UInt64,
    output_cost UInt64,
    cached_input_cost UInt64,
    cache_creation_input_cost UInt64 DEFAULT 0,
    reasoning_output_cost UInt64,
    edgee_tools_cost UInt64 DEFAULT 0,
    total_cost UInt64,
    duration_ms UInt128,
    key_uuid String,
    organization_uuid String,
    billable Bool DEFAULT true,
    has_edgee_tools Bool DEFAULT false,
    edgee_tools_count UInt32 DEFAULT 0,
    tags Array(String) DEFAULT [],
    dc_name String DEFAULT '',
    ttft_ms UInt64 DEFAULT 0,
    network_latency_ms UInt64 DEFAULT 0,
    error Bool DEFAULT false,
    error_type String DEFAULT '',
    error_message String DEFAULT '',
    streaming Bool DEFAULT true,
    compression_enabled Bool DEFAULT false,
    brevity_enabled Bool DEFAULT false,
    brevity_rate Float64 DEFAULT 0,
    compression_time_ms Nullable(UInt64),
    compression_rate Nullable(Float64),
    uncompressed_input_tokens Nullable(UInt32),
    compressed_input_tokens Nullable(UInt32),
    compression_error Nullable(String),
    compression_fell_back_to_original Bool DEFAULT false,
    token_cost_savings UInt64 DEFAULT 0,
    debug_enabled Bool DEFAULT false,
    session_id String DEFAULT '',
    compression_tool_stats JSON,
    uncompressed_tools_tokens Nullable(UInt32),
    compressed_tools_tokens Nullable(UInt32),
    user_id String DEFAULT '',
    squad_id String DEFAULT '',
    mcp_surface_stats JSON,
    optimization_stats JSON,
    is_byok Bool DEFAULT false,
    is_api_billing Bool DEFAULT false,
    is_subscription_billing Bool DEFAULT false,
    is_fallback Bool DEFAULT false,
    is_reroute Bool DEFAULT false,
    is_plan_fallback Bool DEFAULT false
)
ENGINE = MergeTree()
PARTITION BY toYYYYMMDD(timestamp)
ORDER BY (organization_uuid, key_uuid, timestamp, provider, model)
COMMENT 'Edgee Usage Logs v1';

This is the exact schema Edgee’s own internal ClickHouse pipeline uses, so you can copy it as-is. Adapt the ENGINE/PARTITION BY/ORDER BY clauses to your own retention and query patterns as long as every column above stays present. Usage logs are batched in memory per destination and flushed every 60 seconds, or sooner if the buffer grows past 8 MB. Each flush is one write, not one per log record. For S3, that write is one NDJSON object with one usage-log row per line; for ClickHouse, one INSERT ... FORMAT JSONEachRow into your configured table. S3 object keys are laid out as <prefix>/org=<your org id>/date=<yyyy-mm-dd>/hour=<HH>/<timestamp>.jsonl, the partitioning layout most S3-based query engines (Athena, BigQuery external tables, DuckDB) expect out of the box.

BigQuery

Field	Purpose
Name	Label shown in the destinations table
Project ID	GCP project that owns the destination dataset
Dataset ID	BigQuery dataset the usage rows are written into
Table ID	Optional; defaults to `usage_logs` when left blank
Service account key	A GCP service-account key (JSON) with the BigQuery Data Editor role on the target dataset
Active	Toggles forwarding on or off without deleting the destination

Usage rows are buffered in memory per destination and written through BigQuery streaming inserts in batches, every 60 seconds or sooner once 5,000 rows accumulate. Each row carries the same fields Edgee’s own ClickHouse pipeline stores. The destination table is created automatically on first delivery if it does not exist yet, provided the service account has the BigQuery Table Data Editor role; a pre-created table works too.

Snowflake

Field	Purpose
Name	Label shown in the destinations table
Account	Snowflake account identifier, e.g. `myorg-myaccount`
Token	A Programmatic Access Token issued for a user with write access
Database	Optional; falls back to the token user’s default database
Schema	Optional; falls back to the token user’s default schema
Table	Optional; defaults to `EDGEE_USAGE_LOGS_V1`
Warehouse	Optional; falls back to the token user’s default warehouse
Active	Toggles forwarding on or off without deleting the destination

Before saving, generate a Programmatic Access Token for the user this destination writes as (ALTER USER <user> ADD PROGRAMMATIC ACCESS TOKEN <name>; the secret is shown once, so capture it immediately), and attach a network policy to that user. Snowflake refuses PAT authentication without one. A “Test Connection” that fails with Network policy is required is failing on this check. An allow-all policy satisfies the requirement:

CREATE NETWORK POLICY edgee_pat ALLOWED_IP_LIST = ('0.0.0.0/0');
ALTER USER <user> SET NETWORK_POLICY = edgee_pat;

Narrow ALLOWED_IP_LIST once you know which source IPs the traffic should come from. ALLOWED_IP_LIST takes IPv4 addresses only. If your account enforces IPv6, add the IPv6 counterpart with SET ALLOWED_IP6_LIST = ('::/0'). Both statements need a role holding the network-policy privileges, typically ACCOUNTADMIN. Rows are delivered over Snowflake’s SQL REST API rather than a driver connection, so there is no username field: the token resolves its own user. If the target table doesn’t already exist, Edgee creates it automatically the first time it flushes, with this shape: one typed column per usage-log field, the same fields the OTLP and S3 destinations carry.

Snowflake DDL: the table Edgee auto-creates

CREATE TABLE IF NOT EXISTS EDGEE_USAGE_LOGS_V1 (
  timestamp TIMESTAMP_NTZ,
  request_id STRING,
  provider STRING,
  model STRING,
  streaming BOOLEAN,
  input_tokens NUMBER,
  output_tokens NUMBER,
  cached_input_tokens NUMBER,
  cache_creation_input_tokens NUMBER,
  reasoning_output_tokens NUMBER,
  input_cost NUMBER,
  output_cost NUMBER,
  cached_input_cost NUMBER,
  cache_creation_input_cost NUMBER,
  reasoning_output_cost NUMBER,
  edgee_tools_cost NUMBER,
  total_cost NUMBER,
  key_uuid STRING,
  organization_uuid STRING,
  billable BOOLEAN,
  is_byok BOOLEAN,
  is_api_billing BOOLEAN,
  is_subscription_billing BOOLEAN,
  is_fallback BOOLEAN,
  is_reroute BOOLEAN,
  is_plan_fallback BOOLEAN,
  has_edgee_tools BOOLEAN,
  edgee_tools_count NUMBER,
  tags VARIANT,
  dc_name STRING,
  network_latency_ms NUMBER,
  ttft_ms NUMBER,
  duration_ms NUMBER,
  debug_enabled BOOLEAN,
  compression_enabled BOOLEAN,
  brevity_enabled BOOLEAN,
  brevity_rate FLOAT,
  compression_time_ms NUMBER,
  compression_rate FLOAT,
  uncompressed_input_tokens NUMBER,
  compressed_input_tokens NUMBER,
  compression_error STRING,
  compression_fell_back_to_original BOOLEAN,
  token_cost_savings NUMBER,
  compression_tool_stats VARIANT,
  uncompressed_tools_tokens NUMBER,
  compressed_tools_tokens NUMBER,
  mcp_surface_stats VARIANT,
  optimization_stats VARIANT,
  error BOOLEAN,
  error_type STRING,
  error_message STRING,
  session_id STRING,
  user_id STRING,
  squad_id STRING
);

Four columns (tags, compression_tool_stats, mcp_surface_stats, optimization_stats) are VARIANT rather than a scalar type, since each holds a list or a nested object, the standard Snowflake choice for that shape. Query them with Snowflake’s dot/bracket notation or FLATTEN, e.g. SELECT tags[0]::string FROM EDGEE_USAGE_LOGS_V1. Every other column is a plain scalar you can SELECT directly, e.g. SELECT model, total_cost FROM EDGEE_USAGE_LOGS_V1. The _V1 suffix is a schema version. It applies only to the auto-created default table, not to the name you pass in the Table field. If a future change needs a different column shape, it ships as EDGEE_USAGE_LOGS_V2 rather than altering _V1 in place, so existing rows and any views or pipes built on top of this table keep working unchanged. If you’d rather pre-create the table yourself (e.g. under a role that doesn’t grant CREATE TABLE), run the statement above against your own table name and pass it in the Table field. Usage logs are batched in memory per destination and flushed as one multi-row INSERT every 60 seconds, or sooner once 1,000 rows are buffered, delivered in chunks of 50 rows per statement. Larger chunks would put too many bound parameters in a single statement, since every usage-log field is its own column. You can configure more than one destination. Each active one gets its own copy of the logs.

Test before saving

Run two checks against the form before saving:

Test Connection: checks reachability only (a TCP/TLS handshake for OTLP, a bucket HEAD for S3, a ping for ClickHouse, a dataset lookup for BigQuery, a SELECT 1 probe for Snowflake).
Send Log: sends one real log record through the full path, in the destination’s protocol, so you can confirm your endpoint or bucket actually receives it.

Security

Secret values (OTLP header values, an S3 destination’s secret access key, a ClickHouse destination’s password, a BigQuery destination’s service account key, a Snowflake destination’s Programmatic Access Token) are encrypted at rest and masked (********) on every read, the same handling Edgee uses for BYOK provider keys. Plaintext values are never returned once saved. Editing a destination requires re-entering its secret, since saving fully replaces the stored config.

Delivery semantics

Usage logs keep reaching Edgee’s internal pipeline unchanged, regardless of what you configure here.
Forwarding is additive and asynchronous: a slow or failing destination never blocks or delays internal delivery, and never affects your other destinations.
The feature is entirely opt-in: no destination configured means no behavior change and no extra calls.

Introduction

Quickstart

Features

Self-Hosting

Integrations

Custom Observability Destinations

Supported protocols

Setup

OTLP

S3

ClickHouse

BigQuery

Snowflake

Test before saving

Security

Delivery semantics

​Supported protocols

​Setup

​OTLP

​S3

​ClickHouse

​BigQuery

​Snowflake

​Test before saving

​Security

​Delivery semantics

Supported protocols

Setup

OTLP

S3

ClickHouse

BigQuery

Snowflake

Test before saving

Security

Delivery semantics