Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elias/readme update and small fix #9

Merged
merged 7 commits into from
May 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/examples.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ jobs:
env:
CATALOG: dbt_hol_dev
SCHEMA: public
DBT_TARGET: dev
with:
command: 'sdf compile'
workspace_dir: 'workspace/dbt_init_challenge'
Expand Down Expand Up @@ -103,6 +104,7 @@ jobs:
env:
CATALOG: dbt_hol_dev
SCHEMA: public
DBT_TARGET: dev
with:
command: 'sdf push'
# relative path to ${{ github.workspace }} which is automatically mounted by GitHub Actions
Expand Down
124 changes: 120 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,129 @@
# sdf-action
Github Action to run `sdf` CLI in workflows.
[![SDF + Snowflake / Redshift / DBT Tests](https://github.com/sdf-labs/sdf-action/actions/workflows/examples.yml/badge.svg)](https://github.com/sdf-labs/sdf-action/actions/workflows/examples.yml)


# Official SDF GitHub Action

This action is used to run SDF in CI/CD workflows with GitHub Actions. A common use case would be to run an impact analysis with `sdf compile` and `sdf check` on PR creation.

*Note: This GitHub action is still < v1, as such certain scenarios may result in unexpected behavior. Please follow the [contributing guidelines](./CONTRIBUTING.md) and create an issue in this repo if you find any bugs or issues.*

For an in-depth guide on how to use this GitHub, please see the GitHub CI/CD section of [our official docs](https://docs.sdf.com/integrations/cicd/ci_cd)

## Usage
Check out this [example workflow](./.github/workflows/examples.yml) to see how to use this action.

<!-- start usage -->
```yaml
- uses: sdf-labs/sdf-action@v0
with:
# The version of SDF to use in the GitHub action.
# This should match the version of SDF you use locally in development.
sdf_version: '0.2.21'

# The SDF command to run. The exit code of this command is used to determine
# success or failure of the action.
command: 'sdf compile'

# The relative path to the workspace starting from the root of your repository.
# The workspace directory is where the `workspace.sdf.yml` file is located.
workspace_dir: './my_sdf_workspace/'

# Whether or not this SDF project is being used with dbt. If so,
# the action runs `sdf dbt refresh` in the container before the
# specified SDF command.
is_dbt: 'false' # Defaults to false

### SDF Cloud Authentication

# NOTE: Both the access_key and secret_key parameters are required for accessing the SDF

# The access key used for headless authentication to the SDF Cloud
access_key: ${{ secrets.SDF_ACCESS_KEY }}

# The secret key used for headless authentication to the SDF Cloud
secret_key: ${{ secrets.SDF_SECRET_KEY }}

### Snowflake Integration

# All of the following are used to authenticate to Snowflake.
# All of these are required if using a Snowflake provider.
snowflake_account_id: ${{ secrets.SNOWFLAKE_ACCOUNT_ID }}
snowflake_username: ${{ secrets.SNOWFLAKE_USERNAME }}
snowflake_password: ${{ secrets.SNOWFLAKE_PASSWORD }}
snowflake_role: ${{ secrets.SNOWFLAKE_ROLE }}
snowflake_warehouse: ${{ secrets.SNOWFLAKE_WAREHOUSE }}

### AWS / Redshift Integration

# All of the following are required if using an S3, AWS Glue, or AWS Redshift
# provider.
aws_region: ${{ secrets.AWS_REGION }}
aws_access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws_secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
```
<!-- end usage -->

# Scenarios

** [Run Compile on PR](#run-compile-on-pr-creation-and-update)
** [Run Checks on PR with Snowflake Provider](#run-compile-on-pr-creation-and-update-with-a-snowflake-provider)
** [Push to SDF Cloud on Merge to Main](#push-to-sdf-cloud-on-merge-to-main-with-a-redshift-provider)


## Run Compile on PR Creation and Update

```yaml
on:
pull_request:
...
- uses: sdf-labs/sdf-action@v0
with:
sdf_version: '0.2.21'
command: 'sdf compile'
```

## Run Checks on PR Creation and Update with a Snowflake Provider

```yaml
on:
pull_request:
...
- uses: sdf-labs/sdf-action@v0
with:
sdf_version: '0.2.21'
command: 'sdf check'
snowflake_account_id: ${{ secrets.SNOWFLAKE_ACCOUNT_ID }}
snowflake_username: ${{ secrets.SNOWFLAKE_USERNAME }}
snowflake_password: ${{ secrets.SNOWFLAKE_PASSWORD }}
snowflake_role: ${{ secrets.SNOWFLAKE_ROLE }}
snowflake_warehouse: ${{ secrets.SNOWFLAKE_WAREHOUSE }}
```

## Push to SDF Cloud on Merge to Main with a Redshift Provider

```yaml
on:
push:
branches:
- main
paths:
- 'my_sdf_workspace/**'
...
- uses: sdf-labs/sdf-action@v0
with:
sdf_version: '0.2.21'
command: 'sdf push -y'
workspace_dir: './my_sdf_workspace/'
access_key: ${{ secrets.SDF_ACCESS_KEY }}
secret_key: ${{ secrets.SDF_SECRET_KEY }}
aws_region: ${{ secrets.AWS_REGION }}
aws_access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws_secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
```

## Parameters

* `snowflake_`* parameters are required if the `snowflake` table provider is used
* `snowflake_*` parameters are required if the `snowflake` table provider is used
* `aws_*` parameters are required if the `redshift` table provider is used
* `access_key` and `secret_key` are required if the `sdf push` is used in `command`

Expand All @@ -18,7 +135,6 @@ Check out this [example workflow](./.github/workflows/examples.yml) to see how t
| `access_key` | access key created from the [console](https://console.sdf.com/catalog/settings/general) to be used in `sdf push` | No | |
| `secret_key` | secret key created from the [console](https://console.sdf.com/catalog/settings/general) to be used in `sdf push` | No | |
| `is_dbt` | Set to a non-empty string if the workspace is dbt based | No | `""` | |
| `dbt_target` | The dbt target | No | `"dev"` |
| `snowflake_account_id` | The snowflake account | No | |
| `snowflake_username` | The snowflake username | No | |
| `snowflake_password` | The snowflake password | No | |
Expand Down
5 changes: 0 additions & 5 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,6 @@ inputs:
description: "Whether the workspace is dbt based or not"
required: false
default: ""
dbt_target:
description: "The dbt target"
required: false
default: "dev"

# required if snowflake table provider is used
snowflake_account_id:
Expand Down Expand Up @@ -79,7 +75,6 @@ runs:
SNOWFLAKE_PASSWORD: ${{ inputs.snowflake_password }}
SNOWFLAKE_ROLE: ${{ inputs.snowflake_role }}
SNOWFLAKE_WAREHOUSE: ${{ inputs.snowflake_warehouse }}
DBT_TARGET: ${{ inputs.dbt_target }}

AWS_ACCESS_KEY_ID: ${{ inputs.aws_access_key_id }}
AWS_SECRET_ACCESS_KEY: ${{ inputs.aws_secret_access_key }}
Expand Down