Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add code coverage upload command #1520

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
74 changes: 74 additions & 0 deletions src/commands/coverage/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Code coverage upload command

Upload your code coverage report files.

## Usage

#### Commands

##### `upload`

This command uploads your code coverage reports to Datadog.

```bash
datadog-ci coverage upload [--dry-run] [--tags] [--flush] <path> <another_path>
```

For example:

```bash
datadog-ci coverage upload --tags key1:value1 --tags key2:value2 unit-tests/coverage-reports acceptance-tests/coverage-reports e2e-tests/coverage-report.xml
```

- The positional arguments are the directories or file paths in which the code coverage reports are located. If you pass a folder, the CLI will look for all `.xml` files in it.
- `--flush` (default: `false`): you may pass `--flush`, `--flush=1` or `--flush=true` to signal that you have uploaded all the coverage reports for the current commit/PR. This will trigger the coverage reports to be processed and the results to displayed in the UI.
- `--tags` is an array of key value pairs of the shape `key:value`. This will set global tags applied to all coverage reports.
- The resulting dictionary will be merged with whatever is in the `DD_TAGS` environment variable. If a `key` appears both in `--tags` and `DD_TAGS`, whatever value is in `DD_TAGS` will take precedence.
- `--measures` is an array of key numerical value pairs of the shape `key:123`. This will set global measures applied to all coverage reports.
- The resulting dictionary will be merged with whatever is in the `DD_MEASURES` environment variable. If a `key` appears both in `--measures` and `DD_MEASURES`, whatever value is in `DD_MEASURES` will take precedence.
- `--dry-run` (default: `false`): it will run the command without the final upload step. All other checks are performed.
- `--skip-git-metadata-upload` (default: `true`): if you want to upload git metadata, you may pass `--skip-git-metadata-upload=0` or `--skip-git-metadata-upload=false`.
- `--verbose` (default: `false`): it will add extra verbosity to the output of the command.

Either positional arguments or `--flush` (or both) must be provided.

#### Environment variables

Additionally, you might configure the `coverage` command with environment variables:

- `DD_API_KEY` (**required**): API key used to authenticate the requests.
- `DD_TAGS`: set global tags applied to all test spans. The format must be `key1:value1,key2:value2`.
- The resulting dictionary will be merged with whatever is in the `--tags` parameter. If a `key` appears both in `--tags` and `DD_TAGS`, whatever value is in `DD_TAGS` will take precedence.
- `DD_MEASURES`: set global numerical tags applied to all test spans. The format must be `key1:123,key2:321`.
- The resulting dictionary will be merged with whatever is in the `--measures` parameter. If a `key` appears both in `--measures` and `DD_MEASURES`, whatever value is in `DD_MEASURES` will take precedence.
- `DD_SITE`: choose your Datadog site, e.g. datadoghq.com or datadoghq.eu.
- `DD_SUBDOMAIN`: if you have a [custom sub-domain enabled](https://docs.datadoghq.com/account_management/multi_organization/#custom-sub-domains) for your organization, this value should be set with the subdomain so that the link to the Datadog Application that the library logs once the upload finishes is accurate.

### Optional dependencies

- [`git`](https://git-scm.com/downloads) is used for extracting repository metadata.

### End-to-end testing process

To verify this command works as expected, you can use `--dry-run`:

```bash
export DD_API_KEY='<API key>'

yarn launch coverage upload --dry-run ./src/commands/coverage/__tests__/fixtures/jacoco-report.xml
```

Successful output should look like this:

```bash
⚠️ DRY-RUN MODE ENABLED. WILL NOT UPLOAD COVERAGE REPORT
Starting upload.
Will upload ode coverage report file src/commands/coverage/__tests__/fixtures/jacoco-report.xml
[DRYRUN] Uploading code coverage report file in src/commands/coverage/__tests__/fixtures/jacoco-report.xml
✅ Uploaded 1 files in 0 seconds.
```


## Further reading

[//]: <> (TODO: Add link to the documentation page)
114 changes: 114 additions & 0 deletions src/commands/coverage/__tests__/fixtures/another-jacoco-report.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><!DOCTYPE report PUBLIC "-//JACOCO//DTD Report 1.1//EN"
"report.dtd">
<report name="module-a">
<sessioninfo id="COMP-QG19F24WGY-c9528a3d" start="1730108644010" dump="1730108649465"/>
<sessioninfo id="COMP-QG19F24WGY-250a784" start="1731948593153" dump="1731948596192"/>
<sessioninfo id="COMP-QG19F24WGY-c6883e55" start="1731948622885" dump="1731948625942"/>
<sessioninfo id="COMP-QG19F24WGY-c6447142" start="1731948645773" dump="1731948648790"/>
<package name="com/datadog/ci/test">
<class name="com/datadog/ci/test/Person" sourcefilename="Person.java">
<method name="&lt;init&gt;" desc="(Ljava/lang/String;)V" line="5">
<counter type="INSTRUCTION" missed="0" covered="6"/>
<counter type="LINE" missed="0" covered="1"/>
<counter type="COMPLEXITY" missed="0" covered="1"/>
<counter type="METHOD" missed="0" covered="1"/>
</method>
<method name="getName" desc="()Ljava/lang/String;" line="7">
<counter type="INSTRUCTION" missed="0" covered="3"/>
<counter type="LINE" missed="0" covered="1"/>
<counter type="COMPLEXITY" missed="0" covered="1"/>
<counter type="METHOD" missed="0" covered="1"/>
</method>
<method name="equals" desc="(Ljava/lang/Object;)Z" line="5">
<counter type="INSTRUCTION" missed="32" covered="0"/>
<counter type="BRANCH" missed="10" covered="0"/>
<counter type="LINE" missed="1" covered="0"/>
<counter type="COMPLEXITY" missed="6" covered="0"/>
<counter type="METHOD" missed="1" covered="0"/>
</method>
<method name="hashCode" desc="()I" line="5">
<counter type="INSTRUCTION" missed="20" covered="0"/>
<counter type="BRANCH" missed="2" covered="0"/>
<counter type="LINE" missed="1" covered="0"/>
<counter type="COMPLEXITY" missed="2" covered="0"/>
<counter type="METHOD" missed="1" covered="0"/>
</method>
<method name="toString" desc="()Ljava/lang/String;" line="5">
<counter type="INSTRUCTION" missed="12" covered="0"/>
<counter type="LINE" missed="1" covered="0"/>
<counter type="COMPLEXITY" missed="1" covered="0"/>
<counter type="METHOD" missed="1" covered="0"/>
</method>
<counter type="INSTRUCTION" missed="64" covered="9"/>
<counter type="BRANCH" missed="12" covered="0"/>
<counter type="LINE" missed="0" covered="2"/>
<counter type="COMPLEXITY" missed="9" covered="2"/>
<counter type="METHOD" missed="3" covered="2"/>
<counter type="CLASS" missed="0" covered="1"/>
</class>
<class name="com/datadog/ci/test/Calculator" sourcefilename="Calculator.java">
<method name="&lt;init&gt;" desc="()V" line="3">
<counter type="INSTRUCTION" missed="0" covered="3"/>
<counter type="LINE" missed="0" covered="1"/>
<counter type="COMPLEXITY" missed="0" covered="1"/>
<counter type="METHOD" missed="0" covered="1"/>
</method>
<method name="add" desc="(II)I" line="6">
<counter type="INSTRUCTION" missed="0" covered="4"/>
<counter type="LINE" missed="0" covered="1"/>
<counter type="COMPLEXITY" missed="0" covered="1"/>
<counter type="METHOD" missed="0" covered="1"/>
</method>
<method name="multiply" desc="(II)I" line="10">
<counter type="INSTRUCTION" missed="0" covered="4"/>
<counter type="LINE" missed="0" covered="1"/>
<counter type="COMPLEXITY" missed="0" covered="1"/>
<counter type="METHOD" missed="0" covered="1"/>
</method>
<method name="divide" desc="(II)I" line="14">
<counter type="INSTRUCTION" missed="0" covered="4"/>
<counter type="LINE" missed="0" covered="1"/>
<counter type="COMPLEXITY" missed="0" covered="1"/>
<counter type="METHOD" missed="0" covered="1"/>
</method>
<counter type="INSTRUCTION" missed="0" covered="15"/>
<counter type="LINE" missed="0" covered="4"/>
<counter type="COMPLEXITY" missed="0" covered="4"/>
<counter type="METHOD" missed="0" covered="4"/>
<counter type="CLASS" missed="0" covered="1"/>
</class>
<sourcefile name="Person.java">
<line nr="5" mi="70" ci="0" mb="12" cb="0"/>
<line nr="7" mi="0" ci="3" mb="0" cb="0"/>
<counter type="INSTRUCTION" missed="64" covered="9"/>
<counter type="BRANCH" missed="12" covered="0"/>
<counter type="LINE" missed="0" covered="2"/>
<counter type="COMPLEXITY" missed="9" covered="2"/>
<counter type="METHOD" missed="3" covered="2"/>
<counter type="CLASS" missed="0" covered="1"/>
</sourcefile>
<sourcefile name="Calculator.java">
<line nr="3" mi="0" ci="3" mb="0" cb="0"/>
<line nr="6" mi="0" ci="4" mb="0" cb="0"/>
<line nr="10" mi="0" ci="4" mb="0" cb="0"/>
<line nr="14" mi="0" ci="4" mb="0" cb="0"/>
<counter type="INSTRUCTION" missed="0" covered="15"/>
<counter type="LINE" missed="0" covered="4"/>
<counter type="COMPLEXITY" missed="0" covered="4"/>
<counter type="METHOD" missed="0" covered="4"/>
<counter type="CLASS" missed="0" covered="1"/>
</sourcefile>
<counter type="INSTRUCTION" missed="64" covered="24"/>
<counter type="BRANCH" missed="12" covered="0"/>
<counter type="LINE" missed="0" covered="6"/>
<counter type="COMPLEXITY" missed="9" covered="6"/>
<counter type="METHOD" missed="3" covered="6"/>
<counter type="CLASS" missed="0" covered="2"/>
</package>
<counter type="INSTRUCTION" missed="64" covered="24"/>
<counter type="BRANCH" missed="12" covered="0"/>
<counter type="LINE" missed="0" covered="6"/>
<counter type="COMPLEXITY" missed="9" covered="6"/>
<counter type="METHOD" missed="3" covered="6"/>
<counter type="CLASS" missed="0" covered="2"/>
</report>
113 changes: 113 additions & 0 deletions src/commands/coverage/__tests__/fixtures/invalid-jacoco-report.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><!DOCTYPE report PUBLIC "-//JACOCO//DTD Report 1.1//EN"
"report.dtd">
<report name="module-a">
<sessioninfo id="COMP-QG19F24WGY-c9528a3d" start="1730108644010" dump="1730108649465"/>
<sessioninfo id="COMP-QG19F24WGY-250a784" start="1731948593153" dump="1731948596192"/>
<sessioninfo id="COMP-QG19F24WGY-c6883e55" start="1731948622885" dump="1731948625942"/>
<sessioninfo id="COMP-QG19F24WGY-c6447142" start="1731948645773" dump="1731948648790"/>
<package name="com/datadog/ci/test">
<class name="com/datadog/ci/test/Person" sourcefilename="Person.java">
<method name="&lt;init&gt;" desc="(Ljava/lang/String;)V" line="5">
<counter type="INSTRUCTION" missed="0" covered="6"/>
<counter type="LINE" missed="0" covered="1"/>
<counter type="COMPLEXITY" missed="0" covered="1"/>
<counter type="METHOD" missed="0" covered="1"/>
</method>
<method name="getName" desc="()Ljava/lang/String;" line="7">
<counter type="INSTRUCTION" missed="0" covered="3"/>
<counter type="LINE" missed="0" covered="1"/>
<counter type="COMPLEXITY" missed="0" covered="1"/>
<counter type="METHOD" missed="0" covered="1"/>
</method>
<method name="equals" desc="(Ljava/lang/Object;)Z" line="5">
<counter type="INSTRUCTION" missed="32" covered="0"/>
<counter type="BRANCH" missed="10" covered="0"/>
<counter type="LINE" missed="1" covered="0"/>
<counter type="COMPLEXITY" missed="6" covered="0"/>
<counter type="METHOD" missed="1" covered="0"/>
</method>
<method name="hashCode" desc="()I" line="5">
<counter type="INSTRUCTION" missed="20" covered="0"/>
<counter type="BRANCH" missed="2" covered="0"/>
<counter type="LINE" missed="1" covered="0"/>
<counter type="COMPLEXITY" missed="2" covered="0"/>
<counter type="METHOD" missed="1" covered="0"/>
</method>
<method name="toString" desc="()Ljava/lang/String;" line="5">
<counter type="INSTRUCTION" missed="12" covered="0"/>
<counter type="LINE" missed="1" covered="0"/>
<counter type="COMPLEXITY" missed="1" covered="0"/>
<counter type="METHOD" missed="1" covered="0"/>
</method>
<counter type="INSTRUCTION" missed="64" covered="9"/>
<counter type="BRANCH" missed="12" covered="0"/>
<counter type="LINE" missed="0" covered="2"/>
<counter type="COMPLEXITY" missed="9" covered="2"/>
<counter type="METHOD" missed="3" covered="2"/>
<counter type="CLASS" missed="0" covered="1"/>
</class>
<class name="com/datadog/ci/test/Calculator" sourcefilename="Calculator.java">
<method name="&lt;init&gt;" desc="()V" line="3">
<counter type="INSTRUCTION" missed="0" covered="3"/>
<counter type="LINE" missed="0" covered="1"/>
<counter type="COMPLEXITY" missed="0" covered="1"/>
<counter type="METHOD" missed="0" covered="1"/>
</method>
<method name="add" desc="(II)I" line="6">
<counter type="INSTRUCTION" missed="0" covered="4"/>
<counter type="LINE" missed="0" covered="1"/>
<counter type="COMPLEXITY" missed="0" covered="1"/>
<counter type="METHOD" missed="0" covered="1"/>
</method>
<method name="multiply" desc="(II)I" line="10">
<counter type="INSTRUCTION" missed="0" covered="4"/>
<counter type="LINE" missed="0" covered="1"/>
<counter type="COMPLEXITY" missed="0" covered="1"/>
<counter type="METHOD" missed="0" covered="1"/>
</method>
<method name="divide" desc="(II)I" line="14">
<counter type="INSTRUCTION" missed="0" covered="4"/>
<counter type="LINE" missed="0" covered="1"/>
<counter type="COMPLEXITY" missed="0" covered="1"/>
<counter type="METHOD" missed="0" covered="1"/>
</method>
<counter type="INSTRUCTION" missed="0" covered="15"/>
<counter type="LINE" missed="0" covered="4"/>
<counter type="COMPLEXITY" missed="0" covered="4"/>
<counter type="METHOD" missed="0" covered="4"/>
<counter type="CLASS" missed="0" covered="1"/>
</class>
<sourcefile name="Person.java">
<line nr="5" mi="64" ci="6" mb="12" cb="0"/>
<line nr="7" mi="3" ci="0" mb="0" cb="0"/>
<counter type="INSTRUCTION" missed="64" covered="9"/>
<counter type="BRANCH" missed="12" covered="0"/>
<counter type="LINE" missed="0" covered="2"/>
<counter type="COMPLEXITY" missed="9" covered="2"/>
<counter type="METHOD" missed="3" covered="2"/>
<counter type="CLASS" missed="0" covered="1"/>
</sourcefile>
<sourcefile name="Calculator.java">
<line nr="3" mi="0" ci="3" mb="0" cb="0"/>
<line nr="6" mi="0" ci="4" mb="0" cb="0"/>
<line nr="10" mi="0" ci="4" mb="0" cb="0"/>
<line nr="14" mi="0" ci="4" mb="0" cb="0"/>
<counter type="INSTRUCTION" missed="0" covered="15"/>
<counter type="LINE" missed="0" covered="4"/>
<counter type="COMPLEXITY" missed="0" covered="4"/>
<counter type="METHOD" missed="0" covered="4"/>
<counter type="CLASS" missed="0" covered="1"/>
</sourcefile>
<counter type="INSTRUCTION" missed="64" covered="24"/>
<counter type="BRANCH" missed="12" covered="0"/>
<counter type="LINE" missed="0" covered="6"/>
<counter type="COMPLEXITY" missed="9" covered="6"/>
<counter type="METHOD" missed="3" covered="6"/>
<counter type="CLASS" missed="0" covered="2"/>
</package>
<counter type="INSTRUCTION" missed="64" covered="24"/>
<counter type="BRANCH" missed="12" covered="0"/>
<counter type="LINE" missed="0" covered="6"/>
<counter type="COMPLEXITY" missed="9" covered="6"/>
<counter type="METHOD" missed="3" covered="6"/>
<counter type="CLASS" missed="0" covered="2"/>
Loading
Loading