Skip to content

Commit

Permalink
[feat]: add notebook and update reqs
Browse files Browse the repository at this point in the history
  • Loading branch information
erwinpaillacan committed May 14, 2023
1 parent 5762a10 commit 4e71b78
Show file tree
Hide file tree
Showing 8 changed files with 77,437 additions and 10 deletions.
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -166,3 +166,8 @@ venv.bak/
*/great_expectations/uncommitted
*.ge_store_backend_id
uncommitted


# save data as example

!data/01_raw/companies.csv
16 changes: 10 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,20 @@ make install
conda activate kedro-ge-example
```

Download this data <https://kedro-org.github.io/kedro/companies.csv>
and save it in `data/01_raw`

Then you can execute:

```bash
kedro run
(kedro-ge-example) kedro run
```

The folder `data/08_reporting/great_expectations` you will find a great expectations report
The file `data/08_reporting/great_expectations/data_docs/local_site/index.html` will contain a html great expectations report.

In `conf/base/parameters/great_expectations_hook.yml` you can find some parameters of the hook.
The validation works with in memory datasets [you can try commenting the catalog entry of `preprocessed_companies`]
The validation also works with in memory datasets [you can try commenting the catalog entry of `preprocessed_companies`]


There is a sample notebook in `notebooks/great_expectations_starter.ipynb` to create expectations:

```bash
(kedro-ge-example) kedro jupyter notebook
```
Empty file removed conf/base/parameters.yml
Empty file.
Loading

0 comments on commit 4e71b78

Please sign in to comment.