Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: initialise specs from primitives #603

Merged
merged 38 commits into from
Sep 3, 2024
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
38cbedf
initialise_specs_from_primitives
MartinBernstorff Aug 30, 2024
b652623
update 5 files
MartinBernstorff Aug 30, 2024
e245b8a
update 5 files
MartinBernstorff Aug 30, 2024
32387fd
update outcome.py
MartinBernstorff Aug 30, 2024
431463d
update temporal.py
MartinBernstorff Aug 30, 2024
6cf5078
build(deps): update Dockerfile, 01_basic.ipynb and pyproject.toml
MartinBernstorff Aug 30, 2024
84c008a
update Dockerfile and aggregators.py
MartinBernstorff Aug 30, 2024
d55c2eb
build: update Dockerfile
MartinBernstorff Aug 30, 2024
9cc48d5
build: update Dockerfile
MartinBernstorff Aug 30, 2024
2dac3fd
docs: update 01_basic.ipynb
MartinBernstorff Aug 30, 2024
52a11aa
build(deps): update action.yml, 01_basic.ipynb, pyproject.toml and uv…
MartinBernstorff Aug 30, 2024
a63f636
build(deps): update pyproject.toml and uv.lock
MartinBernstorff Aug 30, 2024
d317a64
update devcontainer.json and post-start.sh
MartinBernstorff Aug 30, 2024
584d31c
update 7 files
MartinBernstorff Aug 30, 2024
3dddca6
update post-start.sh and Dockerfile
MartinBernstorff Aug 30, 2024
77ba032
update 5 files update 5 files update post-start.sh, Makefile and task…
MartinBernstorff Aug 30, 2024
02fc722
chore: update action.yml
MartinBernstorff Aug 30, 2024
8ad9ef7
update action.yml, static_type_checks.yml and Makefile
MartinBernstorff Aug 30, 2024
a2b5438
build: update Makefile
MartinBernstorff Aug 30, 2024
35deb50
build: update Dockerfile
MartinBernstorff Aug 30, 2024
aa780c1
docs: update 03_text.ipynb
MartinBernstorff Aug 30, 2024
bd5be7c
docs: update 03_text.ipynb
MartinBernstorff Aug 30, 2024
bd7c0fc
update settings.json, test_specs.py and timedelta.py
MartinBernstorff Sep 2, 2024
b8f2364
update 5 files
MartinBernstorff Sep 2, 2024
c76ed25
ci: update documentation.yml
MartinBernstorff Sep 2, 2024
76c2858
chore: update action.yml
MartinBernstorff Sep 2, 2024
d39a00d
update action.yml and static_type_checks.yml
MartinBernstorff Sep 2, 2024
415bf40
ci: update benchmark.yml
MartinBernstorff Sep 2, 2024
3e3eca0
ci: update benchmark.yml
MartinBernstorff Sep 2, 2024
1cc4e53
update benchmark.yml and Makefile
MartinBernstorff Sep 2, 2024
c7fd6ae
build: update Makefile
MartinBernstorff Sep 2, 2024
c2ad6a8
ci: update benchmark.yml
MartinBernstorff Sep 2, 2024
91fe6b6
ci: update benchmark.yml
MartinBernstorff Sep 2, 2024
34f29f9
ci: update benchmark.yml
MartinBernstorff Sep 2, 2024
453a578
build: update Makefile
MartinBernstorff Sep 2, 2024
57e50d8
tutorials: add `to_primitives` examples for all specs
HLasse Sep 2, 2024
0c7413f
chore: rename function
HLasse Sep 2, 2024
95608f3
Update src/timeseriesflattener/specs/timedelta.py
HLasse Sep 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 2 additions & 9 deletions .devcontainer/post-start.sh
Original file line number Diff line number Diff line change
@@ -1,10 +1,3 @@
# Setup lefthook
#!/bin/bash
git init
lefthook install

# Disable Graphite pager
gt user pager --disable

# Install dependencies
pip install -e ".[dev, test]"

uv sync --all-extras
170 changes: 170 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
# Byte-compiled / optimized / DLL files
.git
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock

# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
poetry.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
*venv*/
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/

# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/

# Testing
pytest-coverage.txt

# Mac
.DS_Store

#other
*test_load_data.py
wandb
poetry.lock
*.html

# PyCharm
/.idea/*
**/tmp/
**/.tmp/

*nbconvert*
.testmondata*
.benchmark_cache/*
22 changes: 10 additions & 12 deletions .github/actions/test/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,19 +11,17 @@ runs:
- name: Checkout (GitHub)
uses: actions/checkout@v4

- name: Replace python version in Dockerfile
- name: Set up uv
shell: bash
run: |
sed -i '/python:.*/c\FROM python:${{ inputs.python-version }}' Dockerfile
# Install a specific uv version using the installer
run: curl -LsSf https://astral.sh/uv/0.4.0/install.sh | sh

- name: Create github hosts file
- name: Set up Python
shell: bash
run: | # If this file is not created, the dev container fails because of non-existent mount
mkdir -p ~/.config/gh
touch ~/.config/gh/hosts.yml
run: uv python install ${{ inputs.python-version }}

- name: Pre-build dev container image
uses: devcontainers/[email protected]
with:
runCmd: |
inv test --pytest-args "src/" --pytest-args --pytest-args "--cov=src" --pytest-args "--cov-report term-missing" --pytest-args="--color=yes" --pytest-args="-n auto"
# Installing
- name: Install dependencies
shell: bash
run: |
make test
19 changes: 9 additions & 10 deletions .github/actions/test_tutorials/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,27 +8,26 @@ runs:
using: "composite"
steps:
### Setup prerequisites
- name: Set up uv
shell: bash
# Install a specific uv version using the installer
run: curl -LsSf https://astral.sh/uv/0.4.0/install.sh | sh

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: ${{ inputs.python-version }}
shell: bash
run: uv python install

# Installing
- name: Install dependencies
shell: bash
run: |
pip install invoke
inv install
make install-tutorials

### Run tests
- name: Install jupyter
shell: bash
run: |
pip install jupyter
- name: Compile notebooks and check for errors
shell: bash
run: |
inv test-tutorials
make test-tutorials
for notebook in tutorials/*.ipynb; do
if grep -q "\"cell_type\": \"error\"" $notebook; then
echo "Error cell found in $notebook"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/documentation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
shell: bash
run: |
python -m pip install --upgrade pip
pip install -e ".[docs,tutorials,text]"
pip install -e ".[docs,tutorials]"

- name: Build and Commit
uses: sphinx-notes/[email protected]
Expand Down
19 changes: 10 additions & 9 deletions .github/workflows/static_type_checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,15 @@ jobs:
steps:
- name: Checkout (GitHub)
uses: actions/checkout@v4

- name: Set up uv
shell: bash
run: curl -LsSf https://astral.sh/uv/0.4.0/install.sh | sh

- name: Create github hosts file
run: | # If this file is not created, the dev container fails because of non-existent mount
mkdir -p ~/.config/gh
touch ~/.config/gh/hosts.yml
- name: Set up Python
shell: bash
run: uv python install

- name: Pre-build dev container image
uses: devcontainers/[email protected]
with:
runCmd: |
inv types
- name: Run static type checks
shell: bash
run: make types
12 changes: 1 addition & 11 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,4 @@ repos:
]
types_or: [ python, pyi, jupyter ]
- id: ruff-format
types_or: [ python, pyi, jupyter ]

- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.3.0
hooks:
- id: check-yaml

- repo: https://github.com/repo-helper/pyproject-parser
rev: v0.7.0
hooks:
- id: check-pyproject
types_or: [ python, pyi, jupyter ]
1 change: 1 addition & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,5 @@
"python.testing.pytestEnabled": true,
"python.analysis.typeCheckingMode": "standard",
"python.analysis.diagnosticMode": "workspace",
"restructuredtext.pythonRecommendation.disabled": true,
}
15 changes: 5 additions & 10 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
FROM python:3.9
ENV UV_SYSTEM_PYTHON=true
COPY --from=ghcr.io/astral-sh/uv:0.4.0 /uv /bin/uv

RUN apt-get update && apt-get install -y curl

# NVM and NPM are required for Graphite
# NVM and NPM are required for pyright
# Install nvm
# Explicitly set HOME environment variable
ENV NVM_DIR=$HOME/.nvm
Expand All @@ -19,17 +21,10 @@ RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.5/install.sh | b
ENV NODE_PATH=$NVM_DIR/v$NODE_VERSION/lib/node_modules
ENV PATH=$NVM_DIR/versions/node/v$NODE_VERSION/bin:$PATH


# Install graphite (PR stacking), experimental, can be deleted without notice
RUN npm install -g @withgraphite/graphite-cli@stable

# Install lefthook (git hooks, e.g. pre-commit)
RUN curl -1sLf 'https://dl.cloudsmith.io/public/evilmartians/lefthook/setup.deb.sh' | bash
RUN apt install lefthook

# Set the working directory to /app
WORKDIR /app
VOLUME psycop-common

COPY . /app
RUN --mount=type=cache,target=/root/.cache/pip pip install .
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync
Loading
Loading