Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Add Quickstart Model Counts to README #183

Merged
merged 4 commits into from
Jan 14, 2025
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
# dbt_zendesk version.version

## Documentation
- Added Quickstart model counts to README. ([#183](https://github.com/fivetran/dbt_zendesk/pull/183))
- Corrected references to connectors and connections in the README. ([#183](https://github.com/fivetran/dbt_zendesk/pull/183))

# dbt_zendesk v0.19.3
[PR #185](https://github.com/fivetran/dbt_zendesk/pull/185) includes the following updates:

Expand Down Expand Up @@ -27,7 +33,6 @@
## Documentation Updates
- [Updated README](https://github.com/fivetran/dbt_zendesk?tab=readme-ov-file#step-4-enabledisable-models-for-non-existent-sources) with instructions on how to disable `brand` and `organization` sources.


# dbt_zendesk v0.19.1
[PR #180](https://github.com/fivetran/dbt_zendesk/pull/180) includes the following update:

Expand Down
33 changes: 18 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,14 +41,16 @@ The following table provides a detailed list of final tables materialized within

Many of the above reports are now configurable for [visualization via Streamlit](https://github.com/fivetran/streamlit_zendesk). Check out some [sample reports here](https://fivetran-zendesk.streamlit.app/).

### Materialized Models
Each Quickstart transformation job run materializes 74 models if all components of this data model are enabled. This count includes all staging, intermediate, and final models materialized as `view`, `table`, or `incremental`.
<!--section-end-->

## How do I use the dbt package?

### Step 1: Prerequisites
To use this dbt package, you must have the following:

- At least one Fivetran zendesk connector syncing data into your destination.
- At least one Fivetran zendesk connection syncing data into your destination.
- A **BigQuery**, **Snowflake**, **Redshift**, **PostgreSQL**, or **Databricks** destination.

#### Databricks Dispatch Configuration
Expand All @@ -71,7 +73,7 @@ packages:
> **Note**: Do not include the Zendesk Support source package. The Zendesk Support transform package already has a dependency on the source in its own `packages.yml` file.

### Step 3: Define database and schema variables
#### Option A: Single connector
#### Option A: Single connection
By default, this package runs using your destination and the `zendesk` schema. If this is not where your zendesk data is (for example, if your zendesk schema is named `zendesk_fivetran`), update the following variables in your root `dbt_project.yml` file accordingly:

```yml
Expand All @@ -80,10 +82,10 @@ vars:
zendesk_schema: your_schema_name
```

> **Note**: When running the package with a single source connector, the `source_relation` column in each model will be populated with an empty string.
> **Note**: When running the package with a single source connection, the `source_relation` column in each model will be populated with an empty string.

#### Option B: Union multiple connectors
If you have multiple Zendesk connectors in Fivetran and would like to use this package on all of them simultaneously, we have provided functionality to do so. For each source table, the package will union all of the data together and pass the unioned table into the transformations. The `source_relation` column in each model indicates the origin of each record.
#### Option B: Union multiple connections
If you have multiple Zendesk connections in Fivetran and would like to use this package on all of them simultaneously, we have provided functionality to do so. For each source table, the package will union all of the data together and pass the unioned table into the transformations. The `source_relation` column in each model indicates the origin of each record.

To use this functionality, you will need to set the `zendesk_sources` variable in your root `dbt_project.yml` file:

Expand All @@ -92,21 +94,21 @@ To use this functionality, you will need to set the `zendesk_sources` variable i

vars:
zendesk_sources:
- database: connector_1_destination_name # Required
schema: connector_1_schema_name # Rquired
name: connector_1_source_name # Required only if following the step in the following subsection
- database: connection_1_destination_name # Required
schema: connection_1_schema_name # Rquired
name: connection_1_source_name # Required only if following the step in the following subsection

- database: connector_2_destination_name
schema: connector_2_schema_name
name: connector_2_source_name
- database: connection_2_destination_name
schema: connection_2_schema_name
name: connection_2_source_name
```

##### Recommended: Incorporate unioned sources into DAG
> *If you are running the package through [Fivetran Transformations for dbt Core™](https://fivetran.com/docs/transformations/dbt#transformationsfordbtcore), the below step is necessary in order to synchronize model runs with your Zendesk connectors. Alternatively, you may choose to run the package through Fivetran [Quickstart](https://fivetran.com/docs/transformations/quickstart), which would create separate sets of models for each Zendesk source rather than one set of unioned models.*
> *If you are running the package through [Fivetran Transformations for dbt Core™](https://fivetran.com/docs/transformations/dbt#transformationsfordbtcore), the below step is necessary in order to synchronize model runs with your Zendesk connections. Alternatively, you may choose to run the package through Fivetran [Quickstart](https://fivetran.com/docs/transformations/quickstart), which would create separate sets of models for each Zendesk source rather than one set of unioned models.*

By default, this package defines one single-connector source, called `zendesk`, which will be disabled if you are unioning multiple connectors. This means that your DAG will not include your Zendesk sources, though the package will run successfully.
By default, this package defines one single-connection source, called `zendesk`, which will be disabled if you are unioning multiple connections. This means that your DAG will not include your Zendesk sources, though the package will run successfully.

To properly incorporate all of your Zendesk connectors into your project's DAG:
To properly incorporate all of your Zendesk connections into your project's DAG:
1. Define each of your sources in a `.yml` file in your project. Utilize the following template for the `source`-level configurations, and, **most importantly**, copy and paste the table and column-level definitions from the package's `src_zendesk.yml` [file](https://github.com/fivetran/dbt_zendesk_source/blob/main/models/src_zendesk.yml#L15-L351).

```yml
Expand Down Expand Up @@ -137,8 +139,9 @@ vars:

### Step 4: Enable/Disable models for non-existent sources

> _This step is optional if you are unioning multiple connectors together in the previous step. The `union_data` macro will create empty staging models for sources that are not found in any of your Zendesk schemas/databases. However, you can still leverage the below variables if you would like to avoid this behavior._
> _This step is optional if you are unioning multiple connections together in the previous step. The `union_data` macro will create empty staging models for sources that are not found in any of your Zendesk schemas/databases. However, you can still leverage the below variables if you would like to avoid this behavior._
This package takes into consideration that not every Zendesk Support account utilizes the `schedule`, `schedule_holiday`, `ticket_schedule`, `daylight_time`, `time_zone`, `audit_log`, `domain_name`, `user_tag`, `brand`,`organization`, `organization_tag`, or `ticket_form_history` features, and allows you to disable the corresponding functionality. By default, all variables' values are assumed to be `true`, except for `using_schedule_histories`. Add variables for only the tables you want to enable/disable:

```yml
vars:
using_schedule_histories: True #Enable if you are using audit_logs for schedule histories
Expand Down