Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add option to generate test coverage report without codecov #207

Open
wants to merge 24 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
822a450
add option to generate test coverage report without codecov
zacharyburnett Jun 9, 2024
f95b1fb
update step name
zacharyburnett Jun 10, 2024
2ff2795
update matrix script
zacharyburnett Jun 10, 2024
f9713a8
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 10, 2024
c4ba2c9
upload `.coverage.*` file to combine
zacharyburnett Jun 11, 2024
9786a39
list files
zacharyburnett Jun 21, 2024
19c88a4
use runner.os instead of matrix value
zacharyburnett Jun 21, 2024
bcfbd6b
combine and report coverage
zacharyburnett Jun 21, 2024
6740255
undo style formatting
zacharyburnett Jun 21, 2024
a6f284a
set run shell
zacharyburnett Jun 21, 2024
d626757
set job name as `report coverage` to be explicit
zacharyburnett Jun 21, 2024
64f33ca
use conditional ifs instead of continue-of-error
zacharyburnett Jun 24, 2024
80bec60
reorder ls and mv
zacharyburnett Jun 26, 2024
a44231b
find .coverage
zacharyburnett Jun 26, 2024
35381ff
use `find` to find `.coverage`
zacharyburnett Jun 26, 2024
5ec90ff
rename job to indicate that it is reporting overall coverage
zacharyburnett Jun 27, 2024
88ebba9
always run coverage report
zacharyburnett Jun 27, 2024
5146563
search in current directory for coverage file
zacharyburnett Dec 4, 2024
6a8519d
Test a different thing
Cadair Dec 4, 2024
dcb85f1
Apply suggestions from code review
Cadair Dec 4, 2024
e611989
Update .github/workflows/tox.yml
Cadair Dec 4, 2024
0b43061
Update .github/workflows/tox.yml
Cadair Dec 4, 2024
5236380
Update .github/workflows/tox.yml
Cadair Dec 4, 2024
6b908a1
Merge branch 'main' into tox/coverage_github_actions
Cadair Dec 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 44 additions & 2 deletions .github/workflows/tox.yml
Original file line number Diff line number Diff line change
Expand Up @@ -117,8 +117,6 @@ jobs:
envs:
name: Load tox environments
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.set-outputs.outputs.matrix }}
steps:
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
with:
Expand All @@ -142,6 +140,8 @@ jobs:
--runs-on "${{ inputs.runs-on }}" --default-python "${{ inputs.default_python }}" \
--timeout-minutes "${{ inputs.timeout-minutes }}"
shell: sh
outputs:
matrix: ${{ steps.set-outputs.outputs.matrix }}

tox:
name: ${{ matrix.name }}
Expand Down Expand Up @@ -238,3 +238,45 @@ jobs:
uses: codecov/codecov-action@015f24e6818733317a2da2edd6290ab26238649a # v5.0.7
with:
token: ${{ secrets.CODECOV_TOKEN }}

- name: Upload coverage data to GitHub
if: ${{ (success() || failure()) && contains(matrix.coverage, 'github') && matrix.pytest == 'true' }}
uses: actions/upload-artifact@v4
with:
name: .coverage.${{ github.sha }}-${{ runner.os }}-${{ runner.arch }}-${{ matrix.toxenv }}
path: "${{ github.workspace }}/**/.coverage"

report_overall_test_coverage:
needs: [ tox ]
if: always()
name: report overall test coverage
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
lfs: true
submodules: ${{ inputs.submodules }}
ref: ${{ inputs.checkout_ref }}
- uses: actions/download-artifact@v4
with:
pattern: .coverage.*
merge-multiple: true
- id: check_downloaded_files
run: |
[ "$(ls -A .coverage*)" ] && exit 0 || exit 1
continue-on-error: true
- if: steps.check_downloaded_files.outcome == 'success'
uses: actions/setup-python@v5
with:
python-version: "3.12"
- if: steps.check_downloaded_files.outcome == 'success'
name: generate coverage report
run: |
python -Im pip install --upgrade coverage[toml]
python -Im coverage combine
python -Im coverage report -i -m --format=markdown >> $GITHUB_STEP_SUMMARY
- if: steps.check_downloaded_files.outcome == 'success'
uses: actions/upload-artifact@v4
zacharyburnett marked this conversation as resolved.
Show resolved Hide resolved
with:
name: .coverage
path: .coverage
4 changes: 2 additions & 2 deletions docs/source/tox.rst
Original file line number Diff line number Diff line change
Expand Up @@ -159,8 +159,8 @@ This option has no effect if ``pytest`` is ``false``.
coverage
^^^^^^^^

A space separated list of coverage providers to upload to. Currently
only ``codecov`` is supported. Default is to not upload coverage
A space separated list of coverage providers to upload to, either
``codecov`` or ``github``. Default is to not upload coverage
reports.

See also, ``CODECOV_TOKEN`` secret.
Expand Down
15 changes: 10 additions & 5 deletions tools/tox_matrix.py
Original file line number Diff line number Diff line change
Expand Up @@ -154,11 +154,16 @@ def get_matrix_item(env, global_libraries, global_string_parameters,
# set pytest_flag
item["pytest_flag"] = ""
sep = r"\\" if platform == "windows" else "/"
if item["pytest"] == "true" and "codecov" in item.get("coverage", ""):
item["pytest_flag"] += (
rf"--cov-report=xml:${{GITHUB_WORKSPACE}}{sep}coverage.xml ")
if item["pytest"] == "true" and item["pytest-results-summary"] == "true":
item["pytest_flag"] += rf"--junitxml ${{GITHUB_WORKSPACE}}{sep}results.xml "
if item["pytest"] == "true":
if "codecov" in item.get("coverage", ""):
item["pytest_flag"] += (
rf"--cov --cov-report=xml:${{GITHUB_WORKSPACE}}{sep}coverage.xml "
)
elif "github" in item.get("coverage", ""):
item["pytest_flag"] += "--cov "

if item["pytest-results-summary"] == "true":
item["pytest_flag"] += rf"--junitxml ${{GITHUB_WORKSPACE}}{sep}results.xml "

# set libraries
env_libraries = env.get("libraries")
Expand Down
Loading