AI data engineering platform
Keep your warehouse fast, cheap and correct.
For ClickHouse, Cloud or self-hosted.
Book a free audit- Lower warehouse bill
- faster dashboards
- More agents running in the cluster
How Underpin works
Three AI agents. One platform underneath.
The agents are what you see: they find problems, propose fixes and answer questions. Underneath, the Underpin platform does the heavy work. It measures, tests and keeps the record each agent relies on.
Why now
AI agents ask far more questions than people do.
Each question costs money and adds load to the warehouse. An agent also repeats whatever the data says, so one wrong number ends up in many answers. A warehouse that served a team of analysts well can turn slow, expensive or unreliable once agents start using it.
How we start
Start with a free audit.
-
1
Free audit
You see where your money and time go, and what to fix first.
-
2
Fixes
Underpin proposes each fix. You approve them one by one.
-
3
Ongoing care
Underpin watches the warehouse and warns you before the bill or the speed drifts.
Under the hood
The Underpin platform, in detail.
Tune
queries and modelsEvery change is built next to the current version and compared on output, memory and running time before it goes live.
| model | rows | memory, old | memory, new | time, old | time, new | output |
|---|---|---|---|---|---|---|
| sessions_daily | 1,826 | 18.4 GiB | 3.1 GiB | 142 s | 38 s | identical |
| active_users_weekly | 261 | 9.7 GiB | 2.2 GiB | 64 s | 21 s | identical |
| retention_cohorts | 4,380 | 22.6 GiB | 6.8 GiB | 211 s | 74 s | identical |
| orders_by_country_daily | 9,412 | 4.1 GiB | 1.3 GiB | 19 s | 7 s | identical |
Ledger
cost and monitoringOn ClickHouse you pay for the capacity you run, so the next agent question has no price of its own. Ledger keeps that capacity small. On self-hosted clusters it also plans storage table by table, because S3 bills per request and ClickHouse makes a lot of requests.
Measured continuously
- real timeCPU, memory and query load on every node.
- per userWhat each person, dashboard and agent runs, and what it costs the cluster.
- per tableStorage on each disk, parts and merges, and how far back it is still written.
- real timeS3 requests by table and by operation.
- dailyCost per table, reconciled against the AWS bill.
Alerts include
- alertThe S3 write rate doubles against its baseline.
- alertA new table breaks a storage rule.
- alertDaily cost drifts from the AWS bill.
- alertA table outgrows its share of SSD.
- alertQueries run close to the memory limit.
- alertS3 starts throttling requests.
…and more, tuned to your cluster.
Where S3 requests come from
Where each kind of table keeps its data
Old history moves to Iceberg tables on S3. Without zero-copy replication, each ClickHouse replica keeps its own copy on S3. Iceberg stores the data once, however many replicas read it. ClickHouse reads Iceberg directly, and a view unions recent data in ClickHouse with the history in Iceberg, so queries, dashboards and agents see one table.
Atlas
knowledge baseAtlas is the data knowledge base. Connectors take snapshots of the warehouse system tables and query log, dbt artifacts, Airflow, the BI tool and Kafka, and Atlas merges them into one graph by table identity, with lineage and usage per reader. On that graph it keeps owners, metric definitions, data health (test and run results, freshness) and the prompts and context the agents read. It reads metadata only, never the data itself, and runs as one small service on a DuckDB file or the ClickHouse you already have. Agents read it read-only through the HTTP API (an MCP server is planned) and query the warehouse as their own logged users.
- About
-
One row per day and country: orders, revenue and refunds.
- Metrics
-
- revenue = sum(amount) − refunds, net of tax
- orders = count of paid orders
- Lineage
-
upstream
- stg_orders
- stg_refunds
downstream- revenue_weekly
- dashboard: Revenue overview
- role: agent_analyst
- Reads, 7 days
- Health
- fresh · updated 06:10 · 4 of 4 tests passing last 14 runs: 13 ok, 1 failed
Use case
A consumer app’s warehouse, two months in.
ClickHouse Cloud to self-hosted on AWS, plus every heavy dbt model and dashboard tuned. Weekly figures: the last week before the tuning against the most recent full week.
$6.5k
ClickHouse Cloud to self-hosted on AWS. No savings plan.
0
While weekly model runs grew from 23k to 35k.
333GiB-h −81%
Memory × time across every model run.
0
Tiles that ran out of memory.
2.2s
Across 11,000+ tile queries a week.
6.7s
From opening a dashboard to its first screen loaded.
Free audit
Three free audits open this month.
You choose how much access to give.
-
No access
You export usage summaries and we look at those.
-
Read-only access
Underpin looks at the warehouse directly and cannot change anything.
Send a message to book one.