From f832390158257973e7a171315688a305c096edf0 Mon Sep 17 00:00:00 2001 From: tjperry07 Date: Mon, 16 Sep 2024 10:35:02 -0400 Subject: [PATCH 1/7] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index b59e56e..54cf2f6 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ - Produces modeled tables that leverage Google Ads data from [Fivetran's connector](https://fivetran.com/docs/connectors/applications/google-ads) in the format described by [this ERD](https://fivetran.com/docs/applications/google-ads#schemainformation). - Enables you to better understand the performance of your ads across varying grains: - Providing an account, campaign, ad group, keyword, ad, and utm level reports. -- Generates a comprehensive data dictionary of your source and modeled Jira data available in your Coalesce Organization Generated Documentation tab. [Environment or Generated Documentation](https://docs.coalesce.io/docs/generated-documentation). +- Generates a comprehensive data dictionary of your source and modeled Jira data available in your Coalesce Organization [Environment or Generated Documentation](https://docs.coalesce.io/docs/generated-documentation). From c02102fa18caea6071a6490d6de88cbb9cb2e88e Mon Sep 17 00:00:00 2001 From: tjperry07 Date: Mon, 16 Sep 2024 10:36:08 -0400 Subject: [PATCH 2/7] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 54cf2f6..b55b5df 100644 --- a/README.md +++ b/README.md @@ -22,7 +22,7 @@ You can also refer to the table below for a detailed view of all models material | **Model** | **Description** | | ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- | | **GOOGLE_ADS__ACCOUNT_REPORT** | Each record in this table represents the daily performance at the account level. | -|**GOOGLE_ADS__CAMPAIGN_REPORT** | Each record in this table represents the daily performance of a campaign at the campaign/advertising_channel/advertising_channel_subtype level. | +|**GOOGLE_ADS__CAMPAIGN_REPORT** | Each record in this table represents the daily performance of a campaign at the campaign/ advertising_channel/ advertising_channel_subtype level. | | **GOOGLE_ADS__AD_GROUP_REPORT** | Each record in this table represents the daily performance at the ad group level.| | **GOOGLE_ADS__KEYWORD_REPORT** |Each record in this table represents the daily performance at the ad group level for keywords.| | **GOOGLE_ADS__AD_REPORT** |Each record in this table represents the daily performance at the ad level.| From 7116b848e94e021337dc3b1a99debf239b262baf Mon Sep 17 00:00:00 2001 From: tjperry07 Date: Mon, 16 Sep 2024 10:51:42 -0400 Subject: [PATCH 3/7] Update README.md --- README.md | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index b55b5df..7b607ad 100644 --- a/README.md +++ b/README.md @@ -56,27 +56,28 @@ After the import is complete you will see two branches available: Either branch can be selected when setting up a Workspace which will be described below. ## Step 3: Set up a Project / Workspace in Coalesce -After the Git repo has been imported follow the Coalesce [documentation](https://docs.coalesce.io/docs/projects#create-a-new-project) to create a new project. Initially, choose the option `Skip and Create` in the window for `Setup Version Control`. We will connect to the Git repository after creating a Workspace. -Once the Project has been created select `Create Workspace`. Enter a name and meaninful desription based on the Git branch you want to start from, either Dynamic Table or full load based. +1. After the Git repo has been imported follow the Coalesce [documentation](https://docs.coalesce.io/docs/projects#create-a-new-project) to create a new project. Initially, choose the option `Skip and Create` in the window for `Setup Version Control`. We will connect to the Git repository after creating a Workspace. -At this point we are going to set up version control. Select `Project Settings` and in the [Git Repository](https://docs.coalesce.io/docs/changing-a-git-repository-in-coalesce) section enter the URL of the repository you imported into your Git account as the Git Repository URL. +2. Once the Project has been created select `Create Workspace`. Enter a name and meaninful desription based on the Git branch you want to start from, either Dynamic Table or full load based. -Save the `Project Settings`. +3. At this point we are going to set up version control. Select `Project Settings` and in the [Git Repository](https://docs.coalesce.io/docs/changing-a-git-repository-in-coalesce) section enter the URL of the repository you imported into your Git account as the Git Repository URL. -If you have enabled security for your Git repo, [Configure Git Account](https://docs.coalesce.io/docs/set-up-your-git-integration#add-through-the-project-dashboard). +4. Save the `Project Settings`. -After configuring the Git repo select `Launch` to launch the Workspace so we can attach it to a Git branch. +5. If you have enabled security for your Git repo, [Configure Git Account](https://docs.coalesce.io/docs/set-up-your-git-integration#add-through-the-project-dashboard). -A Workspace can be attached to a branch by either selecting the `Git` modal or selecting `git branch` from the Workspace warning message `"Finish setting up version control for this workspace and avoid losing any work. Attach this workspace to a git branch"`. +6. After configuring the Git repo select `Launch` to launch the Workspace so we can attach it to a Git branch. -After the `Attach Workspace to Branch` opens select the desired branch - **google_ads_dynamic_model** or **google_ads_incremental** to attach and `Attach` it. +7. A Workspace can be attached to a branch by either selecting the `Git` modal or selecting `git branch` from the Workspace warning message `"Finish setting up version control for this workspace and avoid losing any work. Attach this workspace to a git branch"`. -Click on the `Git` modal, navigate to the `Branches` tab and select the [Branch Action](https://docs.coalesce.io/docs/git-branches#branch-actions) `Force Checkout` to populate the workspace with the latest Git commit. +8. After the `Attach Workspace to Branch` opens select the desired branch - **google_ads_dynamic_model** or **google_ads_incremental** to attach and `Attach` it. -This will overwrite any uncommitted work in the Workspace, which is what we want, so you will be required to confirm the Force Checkout by typing **FORCE** in the screen. +9. Click on the `Git` modal, navigate to the `Branches` tab and select the [Branch Action](https://docs.coalesce.io/docs/git-branches#branch-actions) `Force Checkout` to populate the workspace with the latest Git commit. -At this point the DAG objects should appear in your Workspace with errors. Some workspace configuration is required to fix these errors. +10. This will overwrite any uncommitted work in the Workspace, which is what we want, so you will be required to confirm the Force Checkout by typing **FORCE** in the screen. + +11. At this point the DAG objects should appear in your Workspace with errors. Some workspace configuration is required to fix these errors. ## Step 3: Workspace Configuration 🛠️ From 6eae388d4a4bf900dd08717b690ba0cdf9c485b0 Mon Sep 17 00:00:00 2001 From: tjperry07 Date: Mon, 16 Sep 2024 10:52:25 -0400 Subject: [PATCH 4/7] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 7b607ad..d7c326c 100644 --- a/README.md +++ b/README.md @@ -97,7 +97,7 @@ The pipeline equires four Storage Locations be created. #### Environments Environments must be configured in order to deploy pipeline to higher level environments (QA, UAT, Pre-Prod, Prod, etc.) based on how you are managing your environments. -### ⚙️Workspace Settings +### Workspace Settings The only difference between the `google_ads_dynamic_model ` and `google_ads_incremental` versions of the Google Ads pipeline is that the google_ads_dynamic_model version requires some parameters to be created and set. Other than that the configuration is the same between them. - **Settings** - Configure the Snowflake account that Coalesce will be utilizing From f3f38c15084b6736d149c3c6913f7c9ccb4aab04 Mon Sep 17 00:00:00 2001 From: tjperry07 Date: Mon, 16 Sep 2024 10:55:14 -0400 Subject: [PATCH 5/7] Update README.md --- README.md | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index d7c326c..fc9e8e5 100644 --- a/README.md +++ b/README.md @@ -98,16 +98,14 @@ The pipeline equires four Storage Locations be created. Environments must be configured in order to deploy pipeline to higher level environments (QA, UAT, Pre-Prod, Prod, etc.) based on how you are managing your environments. ### Workspace Settings -The only difference between the `google_ads_dynamic_model ` and `google_ads_incremental` versions of the Google Ads pipeline is that the google_ads_dynamic_model version requires some parameters to be created and set. Other than that the configuration is the same between them. +The only difference between the `google_ads_dynamic_model ` and `google_ads_incremental` versions of the Google Ads pipeline is that the `google_ads_dynamic_model` version requires some parameters to be created and set. Other than that the configuration is the same between them. - **Settings** - Configure the Snowflake account that Coalesce will be utilizing - **User Credentials / OAuth Settings** - Enter the credentials required to connect to Snowflake - **Storage Mappings** - This can be configured to use one database / schema for all Storage Locations or up to four database / schema mappings, one for each Storage Location, depending on whether or not you want to seperate Source, Staging, Intermediate and Target objects. -- **Parameters** - The Dynamic Tables in the Google Ads pipeline require two Parameters to function. - - The first, `targetDynamicTableWarehouse` is the standard Dynamic Table Parameter described in the Dynamic Table Package documentation. - - The second one, `GoogleAdsPipelineWarehouse` is specific to this pipeline. It allows you to set a warehouse for the entire pipeline using a parameter instead of configuring individual nodes. +- **Parameters** - The Dynamic Tables in the Google Ads pipeline require two Parameters to function. + - `targetDynamicTableWarehouse` is the standard Dynamic Table Parameter described in the Dynamic Table Package documentation. + - `GoogleAdsPipelineWarehouse` is specific to this pipeline. It allows you to set a warehouse for the entire pipeline using a parameter instead of configuring individual nodes. This parameter can be used or individual nodes can have their configs updated to use different warehouses. @@ -131,7 +129,7 @@ The target lag of the pipeline can be changed by changing the `Lag Specification Alternatively, individual nodes could be changed from Downstream to a Lag Specification. ### Missing Sources -If there are sources not available in your specific case nodes related to those areas can be deleted from the pipeline. This will require modification of any downstream objects that rely on sources, but should be quick to figure out utilizing Coalesce object and column level lineage capabilities. +If there are sources not available in your specific case nodes related to those areas can be deleted from the pipeline. This will require modification of any downstream objects that rely on sources, but should be quick to figure out utilizing Coalesce object and column level lineage capabilities. ### Executing Pipeline @@ -148,7 +146,7 @@ Dynamic Tables are DDL only Snowflake objects, meaning there is no `Run` compone The pipeline is comprised of 10 sources and 28 tables. ### Missing Sources -If there are sources not available in your specific case nodes related to those areas can be deleted from the pipeline. This will require modification of any downstream objects that rely on sources, but should be quick to figure out utilizing Coalesce object and column level lineage capabilities. +If there are sources not available in your specific case nodes related to those areas can be deleted from the pipeline. This will require modification of any downstream objects that rely on sources, but should be quick to figure out utilizing Coalesce object and column level lineage capabilities. ### Executing Pipeline @@ -156,7 +154,6 @@ After the Workspace has been configured commit any changes into Git. If the onl At this point you can create the pipeline. The easiest way to do this is select `Create All` and then you can execute with `Run All` from the Graph action menu. - From here you can deploy to higher level environments, assuming you have created environments, utilizing the standard Coalesce deployment mechanisms. Once successfully deployed, execute the Job **GOOGLE_ADS_REPORTS** against an environment. From cd19557a9e846e568d1190318b23dc246624db22 Mon Sep 17 00:00:00 2001 From: tjperry07 Date: Mon, 16 Sep 2024 10:55:35 -0400 Subject: [PATCH 6/7] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index fc9e8e5..5caef20 100644 --- a/README.md +++ b/README.md @@ -79,7 +79,7 @@ Either branch can be selected when setting up a Workspace which will be describe 11. At this point the DAG objects should appear in your Workspace with errors. Some workspace configuration is required to fix these errors. -## Step 3: Workspace Configuration 🛠️ +## Step 4: Workspace Configuration In this section you will configure the following Workspace settings: - **Build Settings** - Configure [Storage Locations](https://docs.coalesce.io/docs/storage-locations-and-storage-mappings) From d2e37cf7c95c898377cf746540c17afd22b03250 Mon Sep 17 00:00:00 2001 From: tjperry07 Date: Mon, 16 Sep 2024 12:23:09 -0400 Subject: [PATCH 7/7] Update README.md --- README.md | 20 ++++---------------- 1 file changed, 4 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index 5caef20..1e772d1 100644 --- a/README.md +++ b/README.md @@ -42,13 +42,11 @@ To use this Coalesce pipeline, you must have the following: - [Dynamic Tables](https://docs.coalesce.io/page/package_coalesce_dynamic-tables) (version 1.1.3+) ## Step 2: Import the coalesceio Google Ads repository -In Github select create a new repository and select `Import a repository`. -The URL for the Google Ads repository is [Google Ads ](https://github.com/coalesceio/Google_Ads.git). This is a Public repository so no credentials are required. - -Select an owner and give your repository a name. - -After the import is complete you will see two branches available: +1. In Github select create a new repository and select `Import a repository`. +2. The URL for the Google Ads repository is [Google Ads ](https://github.com/coalesceio/Google_Ads.git). This is a Public repository so no credentials are required. +3. Select an owner and give your repository a name. +4. After the import is complete you will see two branches available: - **google_ads_dynamic_model** - This is an entire pipeline of Dynamic Tables to manage the Google Ads pipeline. - **google_ads_incremental** - This is a pipeline built with an incremental node that loads the incremental data. @@ -58,25 +56,15 @@ Either branch can be selected when setting up a Workspace which will be describe ## Step 3: Set up a Project / Workspace in Coalesce 1. After the Git repo has been imported follow the Coalesce [documentation](https://docs.coalesce.io/docs/projects#create-a-new-project) to create a new project. Initially, choose the option `Skip and Create` in the window for `Setup Version Control`. We will connect to the Git repository after creating a Workspace. - 2. Once the Project has been created select `Create Workspace`. Enter a name and meaninful desription based on the Git branch you want to start from, either Dynamic Table or full load based. - 3. At this point we are going to set up version control. Select `Project Settings` and in the [Git Repository](https://docs.coalesce.io/docs/changing-a-git-repository-in-coalesce) section enter the URL of the repository you imported into your Git account as the Git Repository URL. - 4. Save the `Project Settings`. - 5. If you have enabled security for your Git repo, [Configure Git Account](https://docs.coalesce.io/docs/set-up-your-git-integration#add-through-the-project-dashboard). - 6. After configuring the Git repo select `Launch` to launch the Workspace so we can attach it to a Git branch. - 7. A Workspace can be attached to a branch by either selecting the `Git` modal or selecting `git branch` from the Workspace warning message `"Finish setting up version control for this workspace and avoid losing any work. Attach this workspace to a git branch"`. - 8. After the `Attach Workspace to Branch` opens select the desired branch - **google_ads_dynamic_model** or **google_ads_incremental** to attach and `Attach` it. - 9. Click on the `Git` modal, navigate to the `Branches` tab and select the [Branch Action](https://docs.coalesce.io/docs/git-branches#branch-actions) `Force Checkout` to populate the workspace with the latest Git commit. - 10. This will overwrite any uncommitted work in the Workspace, which is what we want, so you will be required to confirm the Force Checkout by typing **FORCE** in the screen. - 11. At this point the DAG objects should appear in your Workspace with errors. Some workspace configuration is required to fix these errors. ## Step 4: Workspace Configuration