Skip to content

snowplow-incubator/snowplow-bigquery-loader

Repository files navigation

Snowplow Bigquery Loader

Build Status Release License

Introduction

This project contains applications required to load Snowplow data into Bigquery with low latency.

Check out the example config files for how to configure your loader.

Azure

The Azure bigquery loader reads the stream of enriched events from Event Hubs.

Basic usage: `

docker run \
  -v /path/to/config.hocon:/var/config.hocon \
  snowplow/bigquery-loader-kafka:2.0.0 \
  --config /var/config.hocon \
  --iglu-config /var/iglu.json

GCP

The GCP bigquery loader reads the stream of enriched events from Pubsub.

docker run \
  -v /path/to/config.hocon:/var/config.hocon \
  snowplow/bigquery-loader-pubsub:2.0.0 \
  --config /var/config.hocon \
  --iglu-config /var/iglu.json

AWS

The AWS bigquery loader reads the stream of enriched events from Kinesis.

docker run \
  -v /path/to/config.hocon:/var/config.hocon \
  snowplow/bigquery-loader-kinesis:2.0.0 \
  --config /var/config.hocon \
  --iglu-config /var/iglu.json

Find out more

Technical Docs Setup Guide Roadmap & Contributing
i1 i2 i3
Technical Docs Setup Guide Roadmap

Copyright and License

Copyright (c) 2012-present Snowplow Analytics Ltd. All rights reserved.

Licensed under the Snowplow Limited Use License Agreement. (If you are uncertain how it applies to your use case, check our answers to [frequently asked questions][faq].)