GitHub - matewz/simple_etl_example: Little ETL example. Extracting Data, Store and Visualization

matewz / simple_etl_example Public

Notifications You must be signed in to change notification settings
Fork 0
Star 0

Little ETL example. Extracting Data, Store and Visualization

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
templates		templates
.gitignore		.gitignore
README		README
dataManagement.py		dataManagement.py
importData.py		importData.py
process_data.db		process_data.db
task_data.csv		task_data.csv
visualization.py		visualization.py

Repository files navigation

Simple App to load and store data from CSV file at SQLite database.

- Use application *importData.py* to transfers `task_data.csv` to a database
- Use *visualization.py* for a web application for connect and visualize this database

To load data from CSV just replace csv file from folder and execute.
`python importData.py`

After this step you can see data using:
`python importData.py`

And then go to browser and access address:
`http://127.0.0.1`

You can see detail of measurement click at number at `id column`.

Technology and Justification
Database: 
    - SQLite. I chose this because is very small, fast and easy to solve this demand.
    - For this project i wrote SQL's query like text to show my knowledge but it can work with SQLAlchemy too.

WEB: 
    - Flask and Jinja2. 
    - Small, realiable and fast to solve it.

I tried to create a auto solve app. So if don't exists a database it going to create database and the schema too.