This repository has been archived by the owner on Oct 17, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 20
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
0 parents
commit 819be2e
Showing
48 changed files
with
1,946 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,90 @@ | ||
name: Target MSSQL tests | ||
|
||
on: | ||
push: | ||
branches: | ||
- main | ||
pull_request: | ||
branches: | ||
- main | ||
|
||
jobs: | ||
# linting: | ||
# runs-on: ubuntu-latest | ||
# strategy: | ||
# matrix: | ||
# # Only lint using the primary version used for dev | ||
# python-version: [3.9] | ||
# steps: | ||
# - uses: actions/checkout@v2 | ||
# - name: Set up Python '${{ matrix.python-version }}' | ||
# uses: actions/setup-python@v2 | ||
# with: | ||
# python-version: '${{ matrix.python-version }}' | ||
# - name: Install Poetry | ||
# uses: snok/install-poetry@v1 | ||
# with: | ||
# version: 1.1.14 | ||
# - name: Install dependencies | ||
# run: | | ||
# poetry install | ||
# - name: Run lint command from tox.ini | ||
# run: | | ||
# poetry run tox -e lint | ||
pytest: | ||
runs-on: ubuntu-latest | ||
services: | ||
mssql: | ||
image: mcr.microsoft.com/mssql/server:2019-latest | ||
env: | ||
SA_PASSWORD: P@55w0rd | ||
ACCEPT_EULA: "Y" | ||
MSSQL_PID: "Express" | ||
ports: | ||
- 1433:1433 | ||
strategy: | ||
matrix: | ||
# python-version: ["3.8", "3.9", "3.10"] | ||
python-version: ["3.10"] | ||
steps: | ||
- uses: actions/checkout@v2 | ||
- name: Set up Python '${{ matrix.python-version }}' | ||
uses: actions/setup-python@v2 | ||
with: | ||
python-version: '${{ matrix.python-version }}' | ||
- name: Install Poetry | ||
uses: snok/install-poetry@v1 | ||
with: | ||
version: 1.1.11 | ||
- name: Install dependencies | ||
run: | | ||
poetry install | ||
- name: Test with pytest | ||
run: | | ||
poetry run pytest --capture=no | ||
# integration: | ||
# runs-on: ubuntu-latest | ||
# services: | ||
# mssql: | ||
# image: mcr.microsoft.com/mssql/server:2019-latest | ||
# env: | ||
# SA_PASSWORD: P@55w0rd | ||
# ACCEPT_EULA: "Y" | ||
# MSSQL_PID: "Express" | ||
# ports: | ||
# - 1433:1433 | ||
# env: | ||
# python-version: "3.8" | ||
# steps: | ||
# - uses: actions/checkout@v2 | ||
# - name: Set up Python ${{ matrix.python-version }} | ||
# uses: actions/setup-python@v2 | ||
# with: | ||
# python-version: ${{ matrix.python-version }} | ||
# - name: Install dependencies | ||
# run: | | ||
# python -m pip install --upgrade pip | ||
# pip install meltano | ||
# meltano install | ||
# - name: smoke-test-tap | ||
# run: meltano run tap-smoke-test target-postgres |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,143 @@ | ||
# Secrets and internal config files | ||
.secrets/ | ||
|
||
# Ignore meltano internal cache and sqlite systemdb | ||
|
||
.meltano/ | ||
|
||
# Byte-compiled / optimized / DLL files | ||
__pycache__/ | ||
*.py[cod] | ||
*$py.class | ||
|
||
# C extensions | ||
*.so | ||
|
||
# Distribution / packaging | ||
.Python | ||
build/ | ||
develop-eggs/ | ||
dist/ | ||
downloads/ | ||
eggs/ | ||
.eggs/ | ||
lib/ | ||
lib64/ | ||
parts/ | ||
sdist/ | ||
var/ | ||
wheels/ | ||
pip-wheel-metadata/ | ||
share/python-wheels/ | ||
*.egg-info/ | ||
.installed.cfg | ||
*.egg | ||
MANIFEST | ||
|
||
# PyInstaller | ||
# Usually these files are written by a python script from a template | ||
# before PyInstaller builds the exe, so as to inject date/other infos into it. | ||
*.manifest | ||
*.spec | ||
|
||
# Installer logs | ||
pip-log.txt | ||
pip-delete-this-directory.txt | ||
|
||
# Unit test / coverage reports | ||
htmlcov/ | ||
.tox/ | ||
.nox/ | ||
.coverage | ||
.coverage.* | ||
.cache | ||
nosetests.xml | ||
coverage.xml | ||
*.cover | ||
*.py,cover | ||
.hypothesis/ | ||
.pytest_cache/ | ||
|
||
# Translations | ||
*.mo | ||
*.pot | ||
|
||
# Django stuff: | ||
*.log | ||
local_settings.py | ||
db.sqlite3 | ||
db.sqlite3-journal | ||
|
||
# Flask stuff: | ||
instance/ | ||
.webassets-cache | ||
|
||
# Scrapy stuff: | ||
.scrapy | ||
|
||
# Sphinx documentation | ||
docs/_build/ | ||
|
||
# PyBuilder | ||
target/ | ||
|
||
# Jupyter Notebook | ||
.ipynb_checkpoints | ||
|
||
# IPython | ||
profile_default/ | ||
ipython_config.py | ||
|
||
# pyenv | ||
.python-version | ||
|
||
# pipenv | ||
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. | ||
# However, in case of collaboration, if having platform-specific dependencies or dependencies | ||
# having no cross-platform support, pipenv may install dependencies that don't work, or not | ||
# install all needed dependencies. | ||
#Pipfile.lock | ||
|
||
# PEP 582; used by e.g. github.com/David-OConnor/pyflow | ||
__pypackages__/ | ||
|
||
# Celery stuff | ||
celerybeat-schedule | ||
celerybeat.pid | ||
|
||
# SageMath parsed files | ||
*.sage.py | ||
|
||
# Environments | ||
.env | ||
.venv | ||
env/ | ||
venv/ | ||
ENV/ | ||
env.bak/ | ||
venv.bak/ | ||
|
||
# Spyder project settings | ||
.spyderproject | ||
.spyproject | ||
|
||
# Rope project settings | ||
.ropeproject | ||
|
||
# mkdocs documentation | ||
/site | ||
|
||
# mypy | ||
.mypy_cache/ | ||
.dmypy.json | ||
dmypy.json | ||
|
||
# Pyre type checker | ||
.pyre/ | ||
|
||
# dev | ||
._* | ||
.devcontainer/ | ||
plugins/ | ||
config.json | ||
poetry.lock |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,139 @@ | ||
# target-mssql | ||
|
||
`target-mssql` is a Singer target for mssql. | ||
|
||
Build with the [Meltano Target SDK](https://sdk.meltano.com). | ||
|
||
## Known limitations | ||
|
||
- Does not handle complex types such as objects and arrays | ||
- Does not sanitize column names, so might fail on strange column names | ||
- Does not handle large INTs | ||
- Does not handle encoded strings | ||
|
||
<!-- | ||
Developer TODO: Update the below as needed to correctly describe the install procedure. For instance, if you do not have a PyPi repo, or if you want users to directly install from your git repo, you can modify this step as appropriate. | ||
## Installation | ||
Install from PyPi: | ||
```bash | ||
pipx install target-mssql | ||
``` | ||
Install from GitHub: | ||
```bash | ||
pipx install git+https://github.com/ORG_NAME/target-mssql.git@main | ||
``` | ||
--> | ||
|
||
## Configuration | ||
|
||
### Accepted Config Options | ||
|
||
<!-- | ||
Developer TODO: Provide a list of config options accepted by the target. | ||
This section can be created by copy-pasting the CLI output from: | ||
``` | ||
target-mssql --about --format=markdown | ||
``` | ||
--> | ||
|
||
A full list of supported settings and capabilities for this | ||
target is available by running: | ||
|
||
```bash | ||
target-mssql --about | ||
``` | ||
|
||
### Configure using environment variables | ||
|
||
This Singer target will automatically import any environment variables within the working directory's | ||
`.env` if the `--config=ENV` is provided, such that config values will be considered if a matching | ||
environment variable is set either in the terminal context or in the `.env` file. | ||
|
||
### Source Authentication and Authorization | ||
|
||
<!-- | ||
Developer TODO: If your target requires special access on the destination system, or any special authentication requirements, provide those here. | ||
--> | ||
|
||
## Usage | ||
|
||
You can easily run `target-mssql` by itself or in a pipeline using [Meltano](https://meltano.com/). | ||
|
||
### Executing the Target Directly | ||
|
||
```bash | ||
target-mssql --version | ||
target-mssql --help | ||
# Test using the "Carbon Intensity" sample: | ||
tap-carbon-intensity | target-mssql --config /path/to/target-mssql-config.json | ||
``` | ||
|
||
## Developer Resources | ||
|
||
Follow these instructions to contribute to this project. | ||
|
||
### Initialize your Development Environment | ||
|
||
```bash | ||
pipx install poetry | ||
poetry install | ||
``` | ||
|
||
### Create and Run Tests | ||
|
||
Create tests within the `target_mssql/tests` subfolder and | ||
then run: | ||
|
||
```bash | ||
poetry run pytest | ||
``` | ||
|
||
You can also test the `target-mssql` CLI interface directly using `poetry run`: | ||
|
||
```bash | ||
poetry run target-mssql --help | ||
``` | ||
|
||
### Testing with [Meltano](https://meltano.com/) | ||
|
||
_**Note:** This target will work in any Singer environment and does not require Meltano. | ||
Examples here are for convenience and to streamline end-to-end orchestration scenarios._ | ||
|
||
<!-- | ||
Developer TODO: | ||
Your project comes with a custom `meltano.yml` project file already created. Open the `meltano.yml` and follow any "TODO" items listed in | ||
the file. | ||
--> | ||
|
||
Next, install Meltano (if you haven't already) and any needed plugins: | ||
|
||
```bash | ||
# Install meltano | ||
pipx install meltano | ||
# Initialize meltano within this directory | ||
cd target-mssql | ||
meltano install | ||
``` | ||
|
||
Now you can test and orchestrate using Meltano: | ||
|
||
```bash | ||
# Test invocation: | ||
meltano invoke target-mssql --version | ||
# OR run a test `elt` pipeline with the Carbon Intensity sample tap: | ||
meltano elt tap-carbon-intensity target-mssql | ||
``` | ||
|
||
### SDK Dev Guide | ||
|
||
See the [dev guide](https://sdk.meltano.com/en/latest/dev_guide.html) for more instructions on how to use the Meltano SDK to | ||
develop your own Singer taps and targets. |
Oops, something went wrong.