返回

Optimizing your runs for lower compute, fresher data, and faster iteration with dbt State

RSS dbt Blog 原文 · 未翻译 发布 2026-10-27 05:53

有效阅读仅保存在当前浏览器,用于「为你」推荐;暂停或清除记录。

DataHot 速览

dbt State turns freshness rules into model-level code, and it took dbt Labs' own project from nineteen scheduled jobs down to two.

原文

Speaking at dbt Summit, Reuben McCreanor, product manager at dbt Labs for dbt State, and Iaroslav Zeigerman, principal software engineer at Fivetran and a lead developer of dbt State (and co-founder of Tobiko, the team behind SQLMesh), explained that dbt was effectively stateless for most of its history: it had no memory of previous runs and no view of what had actually been applied to the warehouse. dbt State exists to fix one thing: every run should only build what actually changed.

That idea shows up in two places. In production, dbt State moves freshness logic out of job schedules and into model-level code. Teams can set a lag_tolerance (how stale a model is allowed to get) and define what counts as "new data" for a given source, down to a custom SQL condition, at the folder, subfolder, or individual model level.

Reuben pointed to dbt Labs' own internal project as an example: what used to be 19 separate jobs, often running on clashing schedules, is now handled by 2. In development, auto-cloning from production and auto-deferral mean an analytics engineer (or an AI agent) can build and test a single model without first rebuilding everything upstream of it, and without hand-managing manifest.json files.

Fanatics is one of the customers already seeing this in production. As Alvin Chai, senior analytics engineer at Fanatics, put it: "Declaring SLAs at the model and letting it decide what actually needs to run has already delivered 25-30% model reuse and roughly 15% in Snowflake savings on our pilot project."

Iaroslav ran a live demo on the Jaffle Shop project to show what this looks like day to day: a second run of dbt build skipping models with no upstream changes, a state:modified selector that no longer needs a manifest file at all, a zero-copy clone that pulls an already-built table from another environment instead of recomputing it, and a dbt explain command that gives a plain-language (and a verbose) answer to "why did this model build, or not build." He also showed a rebase scenario: pulling in a teammate's already-run model from their dev environment, with no re-execution required.

Looking ahead, the team is working on lag-tolerance tuning recommendations based on warehouse query history, an experimental column-aware mode that would skip a model rebuild even when upstream code changed, if the specific columns a model depends on didn't, and a compare-changes view in the platform UI.

In Q&A, they clarified a few boundaries: dbt State isn't a replacement for source freshness alerting (state is about execution efficiency, not monitoring, and alerting is still on the roadmap); views are still rebuilt whenever their underlying query changes; and snapshots are exempt from state entirely. Pricing is usage-based, measured in Daily Active Target Tables, meaning teams pay only for the first reuse of a given table each day, and views don't count toward billing at all.

dbt State is available today: pip install it if you're running dbt 1.7 through 1.12, it's built into dbt v2, and it's a single checkbox if you're on the dbt platform. Read more on the dbt State product page.

Watch every keynote and breakout from dbt Summit 2026 on demand. Watch dbt Summit 2026 on demand →

这篇内容对你有用吗?

反馈只用于改善内容筛选,不等同于收藏

分享这条资讯
分享海报
保存图片
iOS 也可以长按图片保存