Skip to content

Commit

Permalink
Merge branch 'main' into concat
Browse files Browse the repository at this point in the history
  • Loading branch information
veni-vidi-vici-dormivi committed Nov 6, 2024
2 parents 9b71afa + 78c20cc commit ac81c1b
Show file tree
Hide file tree
Showing 20 changed files with 804 additions and 176 deletions.
32 changes: 29 additions & 3 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,33 +12,59 @@ concurrency:

jobs:
test:
name: py${{ matrix.python-version }}
name: py${{ matrix.python-version }} ${{ matrix.env }}

runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
python-version: ["3.9", "3.11", "3.12"]
python-version: ["3.10", "3.12", "3.13"]
os: [ "ubuntu-latest"]
env: [""]
include:
- env: "min_all_deps"
os: "ubuntu-latest"
python-version: "3.10"

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Fetch all history for all branches and tags.

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Set environment variables
run: |
if [[ "${{ matrix.env }}" == "" ]]; then
echo "PIP_ENV_FILE=environment.txt" >> $GITHUB_ENV
else
echo "PIP_ENV_FILE=${{ matrix.env }}.txt" >> $GITHUB_ENV
fi
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install pytest pytest-cov
python -m pip install -r "ci/requirements/${{ env.PIP_ENV_FILE }}"
python -m pip install -e .
- name: List dependencies
run: |
python -m pip list
- name: Test with pytest
run: python -m pytest
--cov=filefinder
--cov-report=xml
--junitxml=test-results/${{ runner.os }}-${{ matrix.python-version }}.xml

- name: Upload code coverage to Codecov
uses: codecov/codecov-action@v4
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
file: ./coverage.xml
flags: unittests
Expand All @@ -51,7 +77,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ["3.11"]
python-version: ["3.13"]

steps:
- uses: actions/checkout@v4
Expand Down
6 changes: 3 additions & 3 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# https://pre-commit.com/
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
rev: v5.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
Expand All @@ -20,11 +20,11 @@ repos:
- id: isort
# https://github.com/python/black#version-control-integration
- repo: https://github.com/psf/black
rev: 24.3.0
rev: 24.10.0
hooks:
- id: black
- id: black-jupyter
- repo: https://github.com/PyCQA/flake8
rev: 7.0.0
rev: 7.1.1
hooks:
- id: flake8
34 changes: 34 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,39 @@
# Changelog

## v0.4.0 - unreleased

- Added two methods to find _exactly_ one file or path (and raise an error otherwise):
`FileFinder.find_single_file` and `FileFinder.find_single_path`
([#101](https://github.com/mathause/filefinder/pull/101)).
- Raise an error if an unnamed placeholder (e.g., `"{}"`) is passed
([#110](https://github.com/mathause/filefinder/pull/110))
- The `FileFinder.find_files` arguments `on_parse_error` and `_allow_empty` can no
longer be passed by position ([#99](https://github.com/mathause/filefinder/pull/99)).
- `FileFinder` now raises an error if an invalid `"{placeholder}"` is used
([#99](https://github.com/mathause/filefinder/pull/99)).
- An empty `FileContainer` is returned instead of an empty list when no files/ paths are
found ([#114](https://github.com/mathause/filefinder/pull/114))

- Define and test the minimum supported versions of the dependencies ([#125](https://github.com/mathause/filefinder/pull/125)).

| Package | Old | New |
| ---------- | ------- | ------ |
| numpy | undefined | 1.24 |
| pandas | undefined | 2.0 |
| parse | undefined | 1.19 |

- Changes to `FileContainer`:

- Renamed the `"filename"` column to `"path"` and made it a `pd.Index`, thus removing
this column from the underlying `DataFrame` ([#113](https://github.com/mathause/filefinder/pull/113)).
- Added `meta` and `paths` properties to `FileContainer` which allow to iterate over them
([#121](https://github.com/mathause/filefinder/pull/121)).
- Deprecated `combine_by_key` ([#115](https://github.com/mathause/filefinder/pull/115)).
- Added the number of paths to the repr ([#116](https://github.com/mathause/filefinder/pull/116)).

- Explicitly test on python 3.13 ([#103](https://github.com/mathause/filefinder/pull/103)).
- Drop support for python 3.9 ([#102](https://github.com/mathause/filefinder/pull/102)).

## v0.3.0 - 27.03.2024

New release that adds handling for parsing errors. It also drops python 3.7 and 3.8 support.
Expand Down
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ file_pattern = "{category}_file_{number}"
ff = FileFinder(path_pattern, file_pattern)
```

## Create file and path names

Everything enclosed in curly brackets is a placeholder. Thus, you can create file and
path names like so:

Expand All @@ -27,6 +29,8 @@ ff.create_full_name(category="a", number=1)
>>> /root/a/a_file_1
```

## Find files on disk

However, the strength of filefinder is parsing file names on disk. Assuming you have the
following folder structure:

Expand Down Expand Up @@ -73,6 +77,17 @@ ff.find_files(category=["a1", "b2"], number=1)
>>> 2 /root/b2/b2_file_1 b2 1
```

Often we need to be sure to find _exactly one_ file or path. This can be achieved using

```python
ff.find_single_file(category="a1", number=1)
>>> <FileContainer>
>>> filename category number
>>> 0 /root/a1/a1_file_1 a1 1
```

If none or more than one file is found a `ValueError` is raised.

## Format syntax

You can pass format specifiers to allow more complex formats, see
Expand Down
3 changes: 3 additions & 0 deletions ci/requirements/environment.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
numpy
pandas
parse
3 changes: 3 additions & 0 deletions ci/requirements/min_all_deps.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
numpy==1.24
pandas==2.0
parse==1.19
16 changes: 13 additions & 3 deletions filefinder/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,22 @@

from importlib.metadata import version as _get_version

from . import _filefinder, cmip, utils
from ._filefinder import FileContainer, FileFinder
from filefinder import _filefinder, _utils, cmip, filters
from filefinder._filefinder import FileContainer, FileFinder

__all__ = [
"_filefinder",
"_utils",
"cmip",
"FileContainer",
"FileFinder",
"filters",
]

try:
__version__ = _get_version("filefinder")
except Exception:
del _get_version
except Exception: # pragma: no cover
# Local copy or not installed with setuptools.
# Disable minimum version checks on downstream libraries.
__version__ = "999"
Loading

0 comments on commit ac81c1b

Please sign in to comment.