[WIP] Added First ML Preprocessing Pipeline (#13)

* some updates.

* Some updates.

* Added checks for if bands do not download.

Co-authored-by: Lilli Freischem <[email protected]>
Co-authored-by: annajungbluth <[email protected]>

* Added comments.

* updates.

* updated gitignore.

Co-authored-by: annajungbluth <[email protected]>

* update notebooks.

* added level2 data product download for goes16
squashed bug for specifying domain.

Co-authored-by: annajungbluth <[email protected]>
Co-authored-by: Lilli Freischem <[email protected]>

* Started Pipeline.
Co-authored-by: Lilli Freischem <[email protected]>
Co-authored-by: annajungbluth <[email protected]>

* Working MODIS download script.

* parsed pipeline to various files.

* rearranged pipeline.

---------

Co-authored-by: Lilli Freischem <[email protected]>
Co-authored-by: annajungbluth <[email protected]>

environment.yaml

@@ -0,0 +1,57 @@
+name: rs_tools
+channels:
+  - conda-forge
+dependencies:
+  - python=3.11
+  # Standard Libraries
+  - numpy # Numerical Linear Algebra
+  - scipy # Scientific Computing
+  - scikit-image
+  - conda-forge::xarray
+  - conda-forge::rioxarray
+  - conda-forge::goes2go
+  - conda-forge::earthaccess
+  - conda-forge::pyinterp
+  - conda-forge::torchgeo
+  - pandas # Data structure
+  - scikit-learn # Machine Learning
+  - joblib # Parallelization
+  # PLOTTING LIBRARY
+  - matplotlib # standard plotting library
+  - seaborn # Stats viz library
+  # Storage
+  - conda-forge::netCDF4
+  - conda-forge::zarr
+  - conda-forge::h5netcdf
+  - conda-forge::gdal>=3.4.1,<=3.6.3
+  - h5py
+  # GUI
+  - conda-forge::ipywidgets
+  - ipykernel
+  - conda-forge::tqdm
+  - pip
+  - pip:
+      - toolz
+      - typer
+      - einops
+      # formatting
+      - black
+      - pylint
+      - isort
+      - flake8
+      - mypy
+      - pytest
+      - pre-commit
+      # Notebook stuff
+      - autoroot
+      - pyprojroot
+      - python-dotenv
+      # logging
+      - loguru
+      # plotting
+      - celluloid
+      - corner
+      - tabulate
+      # experiment
+      - hydra-core
+      - hydra-zen