Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New snapshot_meta_column_names config for dbt snapshots #6211

Merged
merged 43 commits into from
Oct 3, 2024
Merged
Changes from 1 commit
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
f6a5780
Getting started on docs for `snapshot_meta_column_names` config
dbeatty10 Oct 2, 2024
008d818
Merge branch 'add/new-snapshot-spec-main' into dbeatty10-patch-1
mirnawong1 Oct 2, 2024
c4ead5f
The metadata columns for snapshots can be customized via the `snapsho…
dbeatty10 Oct 2, 2024
53bd2ff
Add the new `snapshot_meta_column_names` config to the release notes
dbeatty10 Oct 2, 2024
a5208f8
Skeleton reference page for `snapshot_meta_column_names` config for s…
dbeatty10 Oct 2, 2024
fcd5ecd
Links to `snapshot_meta_column_names` page across project file, prope…
dbeatty10 Oct 2, 2024
3d3e09d
Add new page to `sidebars.js`
dbeatty10 Oct 2, 2024
98a66b4
Rename `snapshot_meta_column_names` to `snapshot_meta_column_names.md`
dbeatty10 Oct 2, 2024
97aa3a2
Add hyperlinks for property YAML file `schema.yml` example
dbeatty10 Oct 2, 2024
aceaaf9
Uniform newlines with the code examples for project and property YAML…
dbeatty10 Oct 2, 2024
5090922
Link to the reference page for the `snapshot_meta_column_names` config
dbeatty10 Oct 2, 2024
e21dad5
Remove the `<Version>` tag so that it is always visible
dbeatty10 Oct 2, 2024
74991e6
Merge branch 'add/new-snapshot-spec-main' into dbeatty10-patch-1
dbeatty10 Oct 2, 2024
0fe2acd
Rough draft for `snapshot_meta_column_names` config
dbeatty10 Oct 2, 2024
31b4cfc
Remove the reference to the `dbt_valid_to_current` config until it is…
dbeatty10 Oct 2, 2024
911d83a
Remove extraneous content
dbeatty10 Oct 2, 2024
6781603
Add a filename for the example
dbeatty10 Oct 2, 2024
dc9746f
Align with other snapshot configs that lead with Jinja and project fi…
dbeatty10 Oct 2, 2024
faf60df
Add an example of the table output
dbeatty10 Oct 2, 2024
fa38b7b
Update release-notes.md
dbeatty10 Oct 2, 2024
2bb58f5
Add a property file / `schema.yml` example to the top
dbeatty10 Oct 2, 2024
28d84e5
Fix SQL file docs for Jinja config
dbeatty10 Oct 2, 2024
5e67484
Available in v1.9 or with versionless dbt Cloud
dbeatty10 Oct 2, 2024
4cc1efc
Link to SCD type 2 wikipedia page
dbeatty10 Oct 2, 2024
f28e48c
Fix hyperlink
dbeatty10 Oct 2, 2024
28e51d3
Link to page for building dbt snapshots
dbeatty10 Oct 2, 2024
f9dc63e
Add `datatype` and `id` for this docs page
dbeatty10 Oct 2, 2024
a55ab99
Add `default_value` for this page
dbeatty10 Oct 2, 2024
fb73bc3
Merge branch 'add/new-snapshot-spec-main' into dbeatty10-patch-1
dbeatty10 Oct 2, 2024
3e36fc3
Link to the metadata fields specifically
dbeatty10 Oct 2, 2024
cc8175e
Link to dbt Cloud Versionless
dbeatty10 Oct 2, 2024
2ac9a9f
Starting in 1.9
dbeatty10 Oct 2, 2024
f9ba325
Starting in v1.9
dbeatty10 Oct 2, 2024
3409771
Update snapshot_meta_column_names.md
dbeatty10 Oct 2, 2024
36c1547
Separate release note
dbeatty10 Oct 2, 2024
e47e381
Version entire page
dbeatty10 Oct 2, 2024
0b2c49b
Improve wording and fix misspellings
dbeatty10 Oct 2, 2024
464ce51
Convert from a note to a warning
dbeatty10 Oct 2, 2024
5467e1f
Update wording in release notes
dbeatty10 Oct 2, 2024
54e9f06
Merge branch 'add/new-snapshot-spec-main' into dbeatty10-patch-1
dbeatty10 Oct 2, 2024
af80431
Merge branch 'current' into dbeatty10-patch-1
dbeatty10 Oct 3, 2024
a9f25e3
Remove extraneous newline
dbeatty10 Oct 3, 2024
54153b4
Merge branch 'current' into dbeatty10-patch-1
dbeatty10 Oct 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'add/new-snapshot-spec-main' into dbeatty10-patch-1
dbeatty10 authored Oct 2, 2024
commit 54e9f06b865b53130f0b3f0f05849d207f1b67de
2 changes: 2 additions & 0 deletions website/docs/docs/dbt-versions/release-notes.md
Original file line number Diff line number Diff line change
@@ -26,6 +26,8 @@ Release notes are grouped by month for both multi-tenant and virtual private clo
- Users on dbt 1.8 and earlier: No action is needed; existing snapshots will continue to work as before. However, we recommend upgrading to Versionless to take advantage of the new snapshot features.
- **Enhancement**: In dbt Cloud Versionless, snapshots defined in SQL files can now use `config` defined in `schema.yml` YAML files. This update resolves the previous limitation that required snapshot properties to be defined exclusively in `dbt_project.yml` and/or a `config()` block within the SQL file. This enhancement will be included in the upcoming dbt Core v1.9 release.
- **New**: In dbt Cloud Versionless, the `snapshot_meta_column_names` config allows for customizing the snapshot metadata columns. This feature allows an organization to align these automatically-generated column names with their conventions, and will be included in the upcoming dbt Core 1.9 release.
- **Enhancement**: In May 2024, dbt Cloud versionless began inferring a model's `primary_key` based on configured data tests and/or constraints within `manifest.json`. The inferred `primary_key` is visible in dbt Explorer and utilized by the dbt Cloud [compare changes](/docs/deploy/run-visibility#compare-tab) feature. This will also be released in dbt Core 1.9.
Read about the [order dbt infers columns can be used as primary key of a model](https://github.com/dbt-labs/dbt-core/blob/7940ad5c7858ff11ef100260a372f2f06a86e71f/core/dbt/contracts/graph/nodes.py#L534-L541).
- **New:** dbt Explorer now includes trust signal icons, which is currently available as a [Preview](/docs/dbt-versions/product-lifecycles#dbt-cloud). Trust signals offer a quick, at-a-glance view of data health when browsing your dbt models in Explorer. These icons indicate whether a model is **Healthy**, **Caution**, **Degraded**, or **Unknown**. For accurate health data, ensure the resource is up-to-date and has had a recent job run. Refer to [Trust signals](/docs/collaborate/explore-projects#trust-signals-for-resources) for more information.
- **New:** Auto exposures are now available in Preview in dbt Cloud. Auto-exposures helps users understand how their models are used in downstream analytics tools to inform investments and reduce incidents. It imports and auto-generates exposures based on Tableau dashboards, with user-defined curation. To learn more, refer to [Auto exposures](/docs/collaborate/auto-exposures).


Unchanged files with check annotations Beta

import React from 'react';
import Layout from '@theme/Layout';
import CodeBlock from '@theme/CodeBlock';

Check warning on line 4 in website/src/pages/styles.js

GitHub Actions / eslint-check

'CodeBlock' is defined but never used
import Changelog from '@site/src/components/changelog';
import CloudCore from '@site/src/components/cloudcore';
import Collapsible from '@site/src/components/collapsible';
version: dbtVersion,
EOLDate,
isPrerelease,
latestStableRelease,

Check warning on line 43 in website/src/theme/DocRoot/Layout/Main/index.js

GitHub Actions / eslint-check

'latestStableRelease' is assigned a value but never used
} = useContext(VersionContext);
const {
pageAvailable,

Check warning on line 47 in website/src/theme/DocRoot/Layout/Main/index.js

GitHub Actions / eslint-check

'pageAvailable' is assigned a value but never used
firstAvailableVersion,
lastAvailableVersion

Check warning on line 49 in website/src/theme/DocRoot/Layout/Main/index.js

GitHub Actions / eslint-check

'lastAvailableVersion' is assigned a value but never used
} = pageVersionCheck(dbtVersion, versionedPages, currentDocRoute);
const hasFirstAvailableVersion =

Check warning on line 52 in website/src/theme/DocRoot/Layout/Main/index.js

GitHub Actions / eslint-check

'hasFirstAvailableVersion' is assigned a value but never used
firstAvailableVersion && firstAvailableVersion !== "0";
// Check whether this version is a isPrerelease, and show banner if so
You are viewing a condensed version of this merge commit. You can view the full changes here.