-
Notifications
You must be signed in to change notification settings - Fork 81
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Co-authored-by: Al Spinks <[email protected]> Co-authored-by: Andrew Baranec <[email protected]> Co-authored-by: Charles Wright <[email protected]> Co-authored-by: Chip Kent <[email protected]> Co-authored-by: Colin Alworth <[email protected]> Co-authored-by: Corey Kosak <[email protected]> Co-authored-by: Cristian Ferretti <[email protected]> Co-authored-by: Danila Ladner <[email protected]> Co-authored-by: Darin Petty <[email protected]> Co-authored-by: David Godinez <[email protected]> Co-authored-by: Davor Spasovski <[email protected]> Co-authored-by: Don McKenzie <[email protected]> Co-authored-by: James Nelson <[email protected]> Co-authored-by: Jianfeng Mao <[email protected]> Co-authored-by: Jo Janssens <[email protected]> Co-authored-by: Matthew Runyon <[email protected]> Co-authored-by: Mike Bender <[email protected]> Co-authored-by: Nate Bauernfeind <[email protected]> Co-authored-by: Pete Goddard <[email protected]> Co-authored-by: Robb Camera <[email protected]> Co-authored-by: Ryan Caudy <[email protected]> Co-authored-by: Vlad Babich <[email protected]>
- Loading branch information
1 parent
11ad079
commit c76e6e7
Showing
7,404 changed files
with
1,438,135 additions
and
0 deletions.
The diff you're trying to view is too large. We only load the first 3000 changed files.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
COMPOSE_PROJECT_NAME=core | ||
REPO=ghcr.io/ | ||
TAG=main | ||
PORT=10000 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
--- | ||
name: Bug report | ||
about: Create a report to help us improve | ||
labels: bug, triage | ||
|
||
--- | ||
|
||
<!-- Please read our Code of Conduct: https://github.com/deephaven/core/blob/main/CODE_OF_CONDUCT.md --> | ||
<!-- Please search existing issues to avoid creating duplicates. --> | ||
|
||
**Description** | ||
|
||
A clear and concise description of what the bug is. | ||
|
||
**Steps to reproduce** | ||
1. Go to '...' | ||
2. Click on '....' | ||
3. Scroll down to '....' | ||
|
||
**Expected results** | ||
|
||
A clear and concise description of what you expected to happen. | ||
|
||
**Actual results** | ||
|
||
A clear and concise description of what actually happened. | ||
|
||
**Additional details and attachments** | ||
|
||
If applicable, add any additional screenshots, logs, or other attachments to help explain your problem. | ||
|
||
**Versions** | ||
- Deephaven: ... | ||
- OS: ... | ||
- Browser: ... | ||
- Docker: ... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
--- | ||
name: Epic | ||
about: A large body of work that can be broken down into a number of feature requests | ||
labels: epic, triage | ||
|
||
--- | ||
|
||
<!-- Please read our Code of Conduct: https://github.com/deephaven/core/blob/main/CODE_OF_CONDUCT.md --> | ||
<!-- Please search existing issues to avoid creating duplicates. --> | ||
|
||
[Description of body of work] | ||
|
||
<!-- After creation of the epic, create new Feature request issues and mention this epic to have them link. --> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
--- | ||
name: Feature request | ||
about: Suggest an idea for this project | ||
labels: feature request, triage | ||
|
||
--- | ||
|
||
<!-- Please read our Code of Conduct: https://github.com/deephaven/core/blob/main/CODE_OF_CONDUCT.md --> | ||
<!-- Please search existing issues to avoid creating duplicates. --> | ||
|
||
<!-- Describe the feature you'd like. --> | ||
As a [user or stakeholder type], I want [some software feature] so that [some business value] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
# https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources | ||
org.gradle.parallel=true | ||
org.gradle.workers.max=2 | ||
org.gradle.jvmargs=-Xmx6g | ||
|
||
# Our CI JDKs should be pre-provisioned and invoked correctly, | ||
# we shouldn't rely on gradle for any of this logic. | ||
org.gradle.java.installations.auto-download=false | ||
org.gradle.java.installations.auto-detect=false |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
# GitHub Workflow Tools | ||
|
||
## Nightly Benchmark Metrics and Dashboarding | ||
|
||
Location: `.github/tools/metrics/benchmark_capture.py` | ||
|
||
The benchmark capture tool has been created to run after the nightly benchmark job in an effort to capture result data and publish it to GCP Monitoring for historical reference. This script consumes the CSV files that the benchmark gradle jobs publish to the `tmp/logs/` folder at the root of the project during the `Nightly Benchmark` workflow execution. | ||
|
||
### Benchmark File Format | ||
Due to constraints from the GCP API we are limited to 10 labels per metric. This is a limitation imposed by GCP and to accomodate this we have a check to only process the first 9 fields, the `Benchmark` field is skipped as it is used as the metric's base name. | ||
|
||
#### Baseline Fields | ||
- Benchmark: Name of the benchmark, used as the base of the metric name. | ||
- Score: Value used as the metric point. | ||
- Run & Iteration are not used in the dashboarding and the mean average of each metric combination is redered as a single data point. | ||
|
||
#### Dynamic Fields | ||
Any fields beyond the "Benchmark", "Score", "Run" and "Iteration" are added as labels to allow for the filtering and aggregation of the various permutations for all benchmark results. | ||
|
||
### Dashboard Creation | ||
|
||
For each CSV file processed by the tooling a dashboard will be generated to render the metrics. Each dashboard is limited to rendering a total of 40 metrics, this is a customizable value from the commandline but has been chosen as a default in line with the upper limit from GCP for metrics per dashboard. | ||
|
||
The dashboard title is generated from the filename basename and postfixed by the dashboard index, in the even there are greater than 40 metrics total then you will see multiple dashboards created to support this. | ||
|
||
### GitHub Actions Integration | ||
|
||
Location: `.github/workflows/nightly-benchmarks.yml` | ||
|
||
The `nightly-benchmarks.yml` file includes a `benchmarks` section which can be extended to expand the scope of benchmarks run and published, for each benchmark step defined the CSV files will be consumed in the later publish-metrics step as well as be archived to GitHub artifacts by the archive step. | ||
|
||
The following template can be used to create additional benchmark steps within the `benchmark` job definition. | ||
|
||
``` | ||
- name: Benchmark - [BENCHMARK_NAME] | ||
uses: burrunan/gradle-cache-action@v1 | ||
with: | ||
job-id: benchmark | ||
arguments: [GRADLE_COMMAND] | ||
gradle-version: wrapper | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
#!/bin/bash | ||
|
||
set -e | ||
|
||
######################################## | ||
## GitHub Runner Installation | ||
## Deephaven | ||
######################################## | ||
|
||
GH_RUNNER_VERSION=${GH_RUNNER_VERSION:="2.277.1"} | ||
GH_RUNNER_REPO_URL=${GH_RUNNER_REPO_URL:="https://github.com/deephaven/core"} | ||
GH_RUNNER_TOKEN=${GH_RUNNER_TOKEN:?""} | ||
GH_RUNNER_NAME=${GH_RUNNER_NAME:=$HOSTNAME} | ||
GH_RUNNER_LABELS=${GH_RUNNER_LABELS:="gce-runner,benchmark"} | ||
GH_RUNNER_ROOT=${GH_RUNNER_ROOT:="/github"} | ||
GH_RUNNER_TMP=${GH_RUNNER_TMP:="/github-tmp"} | ||
|
||
## Prepare Apt system packages | ||
apt update | ||
apt install --yes curl unzip docker.io | ||
|
||
## JDK Build Prereq | ||
apt install --yes openjdk-8-jdk-headless | ||
|
||
## Setup runner directories | ||
mkdir -p $GH_RUNNER_ROOT | ||
mkdir -p $GH_RUNNER_TMP | ||
|
||
## Pull runner binary | ||
curl -o actions.tar.gz --location "https://github.com/actions/runner/releases/download/v${GH_RUNNER_VERSION}/actions-runner-linux-x64-${GH_RUNNER_VERSION}.tar.gz" | ||
tar xzf actions.tar.gz --directory $GH_RUNNER_ROOT | ||
rm -f actions.tar.gz | ||
|
||
## Install dependencies | ||
/github/bin/installdependencies.sh | ||
|
||
## Configure runner | ||
RUNNER_ALLOW_RUNASROOT=1 $GH_RUNNER_ROOT/config.sh --unattended --replace --work $GH_RUNNER_TMP --url "$GH_RUNNER_REPO_URL" --token "$GH_RUNNER_TOKEN" --labels $GH_RUNNER_LABELS | ||
|
||
## Configure runner as service | ||
cd $GH_RUNNER_ROOT || exit | ||
./svc.sh install | ||
./svc.sh start |
Oops, something went wrong.