Skip to content

sdf action

Actions
Run sdf cli commands
v0.4.1
Latest
Star (16)

SDF + Snowflake / Redshift / DBT Tests

Official SDF GitHub Action

This action is used to run SDF in CI/CD workflows with GitHub Actions. A common use case would be to run an impact analysis with sdf compile and sdf check on PR creation.

Note: This GitHub action is still < v1, as such certain scenarios may result in unexpected behavior. Please follow the contributing guidelines and create an issue in this repo if you find any bugs or issues.

For an in-depth guide on how to use this GitHub, please see the GitHub CI/CD section of our official docs

Usage

Check out this example workflow to see how to use this action.

Note: make sure to use a dbt version that is at least 1.8.0 if you need to use this action for a dbt project

- uses: sdf-labs/sdf-action@v0
  with:
    # The version of SDF to use in the GitHub action.
    # it default to use the latest version
    # or you should specify a version that matches the version of SDF you use locally in development.
    sdf_version: 'latest'

    # The SDF command to run. The exit code of this command is used to determine
    # success or failure of the action.
    command: 'sdf compile'

    # The relative path to the workspace starting from the root of your repository.
    # The workspace directory is where the `workspace.sdf.yml` file is located.
    workspace_dir: './my_sdf_workspace/'

    # Whether or not this SDF project is being used with dbt. If so, 
    # the action runs `sdf dbt refresh` in the container before the 
    # specified SDF command.
    is_dbt: 'false' # Defaults to false

    ### SDF Cloud Authentication
   
    # NOTE: Both the access_key and secret_key parameters are required for accessing the SDF 

    # The access key used for headless authentication to the SDF Cloud
    access_key: ${{ secrets.SDF_ACCESS_KEY }}

    # The secret key used for headless authentication to the SDF Cloud
    secret_key: ${{ secrets.SDF_SECRET_KEY }}

    ### Snowflake Integration

    # Snowflake authentication configuration
    # Required parameters for Snowflake provider:
    snowflake_account_id: ${{ secrets.SNOWFLAKE_ACCOUNT_ID }}
    snowflake_username: ${{ secrets.SNOWFLAKE_USERNAME }}
    snowflake_role: ${{ secrets.SNOWFLAKE_ROLE }}
    snowflake_warehouse: ${{ secrets.SNOWFLAKE_WAREHOUSE }}

    # Authentication method 1: Password-based (choose either method 1 or 2)
    snowflake_password: ${{ secrets.SNOWFLAKE_PASSWORD }}

    # Authentication method 2: Key-pair based (choose either method 1 or 2)
    # Provide either private_key_path OR private_key_pem:
    snowflake_private_key_path: ${{ secrets.SNOWFLAKE_PRIVATE_KEY_PATH }}  # Path to key file
    # OR
    snowflake_private_key_pem: ${{ secrets.SNOWFLAKE_PRIVATE_KEY_PEM }}    # Direct PEM content
    #If you use private_key_pem paste the pem directly into github secrets as is with delimeters and new lines preserved.
    
    # Optional for key-pair authentication:
    snowflake_private_key_passphrase: ${{ secrets.SNOWFLAKE_PRIVATE_KEY_PASSPHRASE }}

    ### AWS / Redshift Integration

    # All of the following are required if using an S3, AWS Glue, or AWS Redshift
    # provider.
    aws_region: ${{ secrets.AWS_REGION }}
    aws_access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
    aws_secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

    ### BigQuery Integration

    # Authentication method 1: Individual credentials
    bigquery_project_id: ${{ secrets.BIGQUERY_PROJECT_ID }}
    bigquery_client_email: ${{ secrets.BIGQUERY_CLIENT_EMAIL }}
    bigquery_private_key: ${{ secrets.BIGQUERY_PRIVATE_KEY }}

    # OR

    # Authentication method 2: Credentials JSON file
    bigquery_credentials_json_path: ${{ secrets.BIGQUERY_CREDENTIALS_JSON_PATH }}

Scenarios

**Run Compile on PR ** Run Checks on PR with Snowflake Provider ** Push to SDF Cloud on Merge to Main

Run Compile on PR Creation and Update

on:
  pull_request:
...
    - uses: sdf-labs/sdf-action@v0
      with:
        sdf_version: 'latest'
        command: 'sdf compile'

Run Checks on PR Creation and Update with a Snowflake Provider

on:
  pull_request:
...
    - uses: sdf-labs/sdf-action@v0
      with:
        sdf_version: 'latest'
        command: 'sdf check'
        snowflake_account_id: ${{ secrets.SNOWFLAKE_ACCOUNT_ID }}
        snowflake_username: ${{ secrets.SNOWFLAKE_USERNAME }}
        snowflake_password: ${{ secrets.SNOWFLAKE_PASSWORD }}
        snowflake_role: ${{ secrets.SNOWFLAKE_ROLE }}
        snowflake_warehouse: ${{ secrets.SNOWFLAKE_WAREHOUSE }}

Push to SDF Cloud on Merge to Main with a Redshift Provider

on: 
  push:
    branches:
      - main
    paths:
      - 'my_sdf_workspace/**'
...
    - uses: sdf-labs/sdf-action@v0
      with:
        sdf_version: 'latest'
        command: 'sdf push -y'
        workspace_dir: './my_sdf_workspace/'
        access_key: ${{ secrets.SDF_ACCESS_KEY }}
        secret_key: ${{ secrets.SDF_SECRET_KEY }}
        aws_region: ${{ secrets.AWS_REGION }}
        aws_access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws_secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

Parameters

  • snowflake_* parameters are required if the snowflake table provider is used
  • aws_* parameters are required if the redshift table provider is used
  • access_key and secret_key are required if the sdf push is used in command
Parameter Description Required Default
sdf_version SDF CLI version No "latest"
command The sdf CLI command to run. No sdf compile
workspace_dir The directory of the workspace No "."
access_key access key created from the console to be used in sdf push No
secret_key secret key created from the console to be used in sdf push No
is_dbt Set to a non-empty string if the workspace is dbt based No ""
snowflake_account_id The snowflake account No
snowflake_username The snowflake username No
snowflake_password The snowflake password No
snowflake_role The snowflake role No
snowflake_warehouse The snowflake warehouse No
aws_region The aws region No
aws_access_key_id The aws access key id created from an IAM user No
aws_secret_access_key The aws secret created from an IAM user No

sdf action is not certified by GitHub. It is provided by a third-party and is governed by separate terms of service, privacy policy, and support documentation.

About

Run sdf cli commands
v0.4.1
Latest

sdf action is not certified by GitHub. It is provided by a third-party and is governed by separate terms of service, privacy policy, and support documentation.