Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.github/workflows: Added ubi8-nightly.yml #53

Merged
merged 1 commit into from
Mar 28, 2024
Merged

.github/workflows: Added ubi8-nightly.yml #53

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .github/workflows/ubi8-nightly.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: github-UBI8-NIGHTLY

# Runs every night at midnight
on:
schedule:
- cron: '00 00 * * *'

permissions:
contents: none

# Cancels any in progress 'workflow' associated with this PR
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
ubi8-nightly:
name: ubi8-nightly
runs-on: [ubuntu-latest]
permissions:
packages: read
contents: read
container:
image: ghcr.io/sandialabs/opencsp:latest-ubi8
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

steps:
- name: checkout
uses: actions/checkout@v4
with:
path: OpenCSP

- name: pytest-cov
working-directory: OpenCSP/example
run: |
python3 -m pip install -r ../requirements.txt
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this create a new container every night? If that's the desired behavior, that seems good to me, because it will help us to catch dependency errors faster. If this doesn't create a new container every night, then maybe it should. Thoughts?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reasoning makes sense to me. I don't know the implications to runtime/using resources of this though. Those would be my only concerns but I'm not well versed in this.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this runs pip install in the container image.

A image would be built with something like: docker build ... or podman build .... For example, see https://github.com/sandialabs/OpenCSP/blob/main/.github/workflows/docker.yml.

Copy link
Collaborator

@bbean23 bbean23 Mar 28, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's more or less what I thought. Thanks!

Back to my original question, can we have a secondary container for the nightly tests that always updates to the latest packages? Really, I think the best set up would be:

  • one Ubi8 container for PR unit tests & example nightly tests
  • one Windows container for PR unit tests & example nightly tests
  • one Ubi8 container for "pip --update" unit+example nightly tests
  • one Windows container for "pip --update" unit+example nightly tests

All four containers should be created every time we merge to main. However, the second two containers should always be running with the latest version of all our dependencies, to try and catch dependency issues early.

What do you think @e10harvey? Is this a reasonable and achievable goal?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bbean23: In general, I think this is a good idea. That being said, I think updating everything in the image every night is asking for trouble. If you would like to triage the nightlies when these fail, then I am OK with it. Otherwise, I think it would be best to setup a second nightly that updates everything in a separate image and runs examples + unit tests. Even with that setup, I am not volunteering for keeping this running : )

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, a second nightly is exactly what I'm proposing. And I agree, you shouldn't be necessarily responsible for making it work. In fact, it would be helpful for me to learn how to set up a container like this. I'll make a ticket to do so.

export PYTHONPATH=$PWD/../
pytest --color=yes -rs -vv --cov=. --cov-report term --cov-config=.coveragerc
10 changes: 7 additions & 3 deletions example/scene_reconstruction/example_scene_reconstruction.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import opencsp.common.lib.tool.log_tools as lt


def example_scene_reconstruction(save_dir: str):
def scene_reconstruction(save_dir):
"""Example script that reconstructs the XYZ locations of Aruco markers in a scene."""
# Define input directory
dir_input = join(opencsp_code_dir(), 'app/scene_reconstruction/test/data/data_measurement')
Expand Down Expand Up @@ -45,12 +45,16 @@ def example_scene_reconstruction(save_dir: str):
fig.savefig(join(save_dir, fig.get_label() + '.png'))


if __name__ == '__main__':
def example_driver():
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this change made? My preference is to have setup code like this contained to within the main statement. Is that incompatible with unit tests somehow?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is incompatible with unit tests, then can we do this instead?

def set_up():
    # Define output directory
    save_path = join(dirname(__file__), 'data/output/scene_reconstruction')
    ft.create_directories_if_necessary(save_path)

    # Set up logger
    lt.logger(join(save_path, 'log.txt'), lt.log.INFO)

    return save_path

if __name__ == "__main__":
    save_path = set_up()
    scene_reconstruction(save_path)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see, it's because we're looking for methods that start with "example_". With that in mind, how do you feel about "example_main" instead of "example_driver"?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can refactor to example_main, if you prefer.

Copy link
Collaborator

@bbean23 bbean23 Mar 28, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a slight preference for example_main. Probably not worth another pr just for that, though. 🤷‍♂️

# Define output directory
save_path = join(dirname(__file__), 'data/output/scene_reconstruction')
ft.create_directories_if_necessary(save_path)

# Set up logger
lt.logger(join(save_path, 'log.txt'), lt.log.INFO)

example_scene_reconstruction(save_path)
scene_reconstruction(save_path)


if __name__ == '__main__':
example_driver()
10 changes: 7 additions & 3 deletions example/sofast_fringe/example_calibration_camera_pose.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
import opencsp.common.lib.tool.log_tools as lt


def example_run_camera_position_calibration(save_dir: str):
def run_camera_position_calibration(save_dir):
"""Calibrates the position of the Sofast camera. Saves the rvec/tvec that
define the relative pose of the camera/screen to a CSV file located
at ./data/output/camera_rvec_tvec.csv
Expand Down Expand Up @@ -47,12 +47,16 @@ def example_run_camera_position_calibration(save_dir: str):
cal.save_data_as_csv(join(save_dir, 'camera_rvec_tvec.csv'))


if __name__ == '__main__':
def example_driver():
# Define save dir
save_path = join(dirname(__file__), 'data/output/camera_pose')
ft.create_directories_if_necessary(save_path)

# Set up logger
lt.logger(join(save_path, 'log.txt'))

example_run_camera_position_calibration(save_path)
run_camera_position_calibration(save_path)


if __name__ == '__main__':
example_driver()
10 changes: 7 additions & 3 deletions example/sofast_fringe/example_calibration_screen_shape.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
import opencsp.common.lib.tool.log_tools as lt


def example_run_screen_shape_calibration(save_dir):
def run_screen_shape_calibration(save_dir):
"""Runs screen shape calibration. Saves data to ./data/output/screen_shape"""
# Load output data from Scene Reconstruction (Aruco marker xyz points)
file_pts_data = join(
Expand Down Expand Up @@ -66,12 +66,16 @@ def example_run_screen_shape_calibration(save_dir):
fig.savefig(file)


if __name__ == '__main__':
def example_driver():
# Define save directory
save_path = join(dirname(__file__), 'data/output/screen_shape')
ft.create_directories_if_necessary(save_path)

# Set up logger
lt.logger(join(save_path, 'log.txt'), lt.log.INFO)

example_run_screen_shape_calibration(save_path)
run_screen_shape_calibration(save_path)


if __name__ == '__main__':
example_driver()
10 changes: 7 additions & 3 deletions example/sofast_fringe/example_process_facet_ensemble.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
import opencsp.common.lib.tool.log_tools as lt


def example_driver(dir_save: str):
def example(dir_save: str):
"""Example SOFAST script

Performs processing of previously collected Sofast data of multi facet mirror ensemble.
Expand Down Expand Up @@ -109,12 +109,16 @@ def example_driver(dir_save: str):
sofast.save_to_hdf(f'{dir_save}/data_multifacet.h5')


if __name__ == '__main__':
def example_driver():
# Define save dir
save_path = join(dirname(__file__), 'data/output/facet_ensemble')
ft.create_directories_if_necessary(save_path)

# Set up logger
lt.logger(join(save_path, 'log.txt'), lt.log.INFO)

example_driver(save_path)
example(save_path)


if __name__ == '__main__':
example_driver()
10 changes: 7 additions & 3 deletions example/sofast_fringe/example_process_single_facet.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
import opencsp.common.lib.tool.log_tools as lt


def example_driver(dir_save: str):
def example(dir_save: str):
"""Example SOFAST script

Performs processing of previously collected Sofast data of single facet mirror.
Expand Down Expand Up @@ -85,12 +85,16 @@ def example_driver(dir_save: str):
sofast.save_to_hdf(f'{dir_save}/data_singlefacet.h5')


if __name__ == '__main__':
def example_driver():
# Define save dir
save_path = join(dirname(__file__), 'data/output/single_facet')
ft.create_directories_if_necessary(save_path)

# Set up logger
lt.logger(join(save_path, 'log.txt'), lt.log.INFO)

example_driver(save_path)
example(save_path)


if __name__ == '__main__':
example_driver()
10 changes: 7 additions & 3 deletions example/sofast_fringe/example_process_undefined_shape.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
import opencsp.common.lib.tool.log_tools as lt


def example_driver(dir_save):
def example(dir_save):
"""Example SOFAST script

Performs processing of previously collected Sofast data of single facet mirror.
Expand Down Expand Up @@ -83,12 +83,16 @@ def example_driver(dir_save):
sofast.save_to_hdf(f'{dir_save}/data_undefined.h5')


if __name__ == '__main__':
def example_driver():
# Define save dir
save_path = join(dirname(__file__), 'data/output/single_facet')
ft.create_directories_if_necessary(save_path)

# Set up logger
lt.logger(join(save_path, 'log.txt'), lt.log.INFO)

example_driver(save_path)
example(save_path)


if __name__ == '__main__':
example_driver()
2 changes: 1 addition & 1 deletion example/sofast_fringe/example_standard_mirror_plot_output.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

from scipy.spatial.transform import Rotation

import opencsp.app.sofast.lib.load_saved_data as lsd
import contrib.app.sofast.load_saved_data as lsd
import opencsp.common.lib.csp.standard_output as so
from opencsp.common.lib.csp.LightSourceSun import LightSourceSun
from opencsp.common.lib.geometry.Uxyz import Uxyz
Expand Down
2 changes: 1 addition & 1 deletion ...ple/solarfield/data/input/ExampleSolarFieldOutput/tsfo001_Heliostat_5E10_Face_West_3d.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Metadata:
Figure number: 1
Name: tsfo001_Heliostat 5E10, Face West
Title: Heliostat 5E10, Face West
Code tag: ExampleSolarFieldOutput.example_single_heliostat()
Code tag: ExampleSolarFieldOutput.single_heliostat()
View spec: 3d

Title:
Expand Down
2 changes: 1 addition & 1 deletion ...rfield/data/input/ExampleSolarFieldOutput/tsfo002_Heliostat_5E10_with_Highlighting_3d.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Metadata:
Figure number: 1
Name: tsfo002_Heliostat 5E10, with Highlighting
Title: Heliostat 5E10, with Highlighting
Code tag: ExampleSolarFieldOutput.example_annotated_heliostat()
Code tag: ExampleSolarFieldOutput.annotated_heliostat()
View spec: 3d

Title:
Expand Down
2 changes: 1 addition & 1 deletion ...ple/solarfield/data/input/ExampleSolarFieldOutput/tsfo003_Example_Poses_and_Styles_3d.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Metadata:
Figure number: 1
Name: tsfo003_Example Poses and Styles
Title: Example Poses and Styles
Code tag: ExampleSolarFieldOutput.example_multi_heliostat()
Code tag: ExampleSolarFieldOutput.multi_heliostat()
View spec: 3d

Title:
Expand Down
2 changes: 1 addition & 1 deletion example/solarfield/data/input/ExampleSolarFieldOutput/tsfo004_Heliostat_Names_xy.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Metadata:
Figure number: 1
Name: tsfo004_Heliostat Names
Title: Heliostat Names
Code tag: ExampleSolarFieldOutput.example_solar_field_h_names()
Code tag: ExampleSolarFieldOutput.solar_field_h_names()
View spec: xy

Title:
Expand Down
2 changes: 1 addition & 1 deletion example/solarfield/data/input/ExampleSolarFieldOutput/tsfo005_Heliostat_Centroids_3d.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Metadata:
Figure number: 1
Name: tsfo005_Heliostat Centroids
Title: Heliostat Centroids
Code tag: ExampleSolarFieldOutput.example_solar_field_h_centroids()
Code tag: ExampleSolarFieldOutput.solar_field_h_centroids()
View spec: 3d

Title:
Expand Down
2 changes: 1 addition & 1 deletion example/solarfield/data/input/ExampleSolarFieldOutput/tsfo006_Heliostat_Centroids_xy.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Metadata:
Figure number: 2
Name: tsfo006_Heliostat Centroids
Title: Heliostat Centroids
Code tag: ExampleSolarFieldOutput.example_solar_field_h_centroids()
Code tag: ExampleSolarFieldOutput.solar_field_h_centroids()
View spec: xy

Title:
Expand Down
2 changes: 1 addition & 1 deletion example/solarfield/data/input/ExampleSolarFieldOutput/tsfo007_Heliostat_Centroids_xz.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Metadata:
Figure number: 3
Name: tsfo007_Heliostat Centroids
Title: Heliostat Centroids
Code tag: ExampleSolarFieldOutput.example_solar_field_h_centroids()
Code tag: ExampleSolarFieldOutput.solar_field_h_centroids()
View spec: xz

Title:
Expand Down
2 changes: 1 addition & 1 deletion example/solarfield/data/input/ExampleSolarFieldOutput/tsfo008_Heliostat_Centroids_yz.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Metadata:
Figure number: 4
Name: tsfo008_Heliostat Centroids
Title: Heliostat Centroids
Code tag: ExampleSolarFieldOutput.example_solar_field_h_centroids()
Code tag: ExampleSolarFieldOutput.solar_field_h_centroids()
View spec: yz

Title:
Expand Down
2 changes: 1 addition & 1 deletion ...solarfield/data/input/ExampleSolarFieldOutput/tsfo009_Heliostat_Labelled_Centroids_xy.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Metadata:
Figure number: 1
Name: tsfo009_Heliostat Labelled Centroids
Title: Heliostat Labelled Centroids
Code tag: ExampleSolarFieldOutput.example_solar_field_h_centroids_names()
Code tag: ExampleSolarFieldOutput.solar_field_h_centroids_names()
View spec: xy

Title:
Expand Down
2 changes: 1 addition & 1 deletion example/solarfield/data/input/ExampleSolarFieldOutput/tsfo010_Heliostat_Outlines_3d.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Metadata:
Figure number: 1
Name: tsfo010_Heliostat Outlines
Title: Heliostat Outlines
Code tag: ExampleSolarFieldOutput.example_solar_field_h_outlines()
Code tag: ExampleSolarFieldOutput.solar_field_h_outlines()
View spec: 3d

Title:
Expand Down
2 changes: 1 addition & 1 deletion example/solarfield/data/input/ExampleSolarFieldOutput/tsfo011_Solar_Field_Situation_3d.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Metadata:
Figure number: 1
Name: tsfo011_Solar Field Situation
Title: Solar Field Situation
Code tag: ExampleSolarFieldOutput.example_annotated_solar_field()
Code tag: ExampleSolarFieldOutput.annotated_solar_field()
View spec: 3d

Title:
Expand Down
2 changes: 1 addition & 1 deletion example/solarfield/data/input/ExampleSolarFieldOutput/tsfo012_Selected_Heliostats_3d.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Metadata:
Figure number: 1
Name: tsfo012_Selected Heliostats
Title: Selected Heliostats
Code tag: ExampleSolarFieldOutput.example_solar_field_subset()
Code tag: ExampleSolarFieldOutput.solar_field_subset()
View spec: 3d

Title:
Expand Down
2 changes: 1 addition & 1 deletion example/solarfield/data/input/ExampleSolarFieldOutput/tsfo013_Heliostat_Vector_Field_3d.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Metadata:
Figure number: 1
Name: tsfo013_Heliostat Vector Field
Title: Heliostat Vector Field
Code tag: ExampleSolarFieldOutput.example_heliostat_vector_field()
Code tag: ExampleSolarFieldOutput.heliostat_vector_field()
View spec: 3d

Title:
Expand Down
2 changes: 1 addition & 1 deletion example/solarfield/data/input/ExampleSolarFieldOutput/tsfo014_Heliostat_Vector_Field_xy.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Metadata:
Figure number: 2
Name: tsfo014_Heliostat Vector Field
Title: Heliostat Vector Field
Code tag: ExampleSolarFieldOutput.example_heliostat_vector_field()
Code tag: ExampleSolarFieldOutput.heliostat_vector_field()
View spec: xy

Title:
Expand Down
3 changes: 1 addition & 2 deletions example/solarfield/data/input/ExampleSolarFieldOutput/tsfo015_Heliostat_Vector_Field_xz.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,8 @@ Metadata:
Figure number: 3
Name: tsfo015_Heliostat Vector Field
Title: Heliostat Vector Field
Code tag: TestSolarFieldOutput.test_heliostat_vector_field()
Code tag: ExampleSolarFieldOutput.heliostat_vector_field()
View spec: xz
Path: common\lib\test\data\output\TestSolarFieldOutput\tsfo015_Heliostat_Vector_Field_xz.png

Title:
Heliostat Vector Field
Expand Down
3 changes: 1 addition & 2 deletions example/solarfield/data/input/ExampleSolarFieldOutput/tsfo016_Heliostat_Vector_Field_yz.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,8 @@ Metadata:
Figure number: 4
Name: tsfo016_Heliostat Vector Field
Title: Heliostat Vector Field
Code tag: TestSolarFieldOutput.test_heliostat_vector_field()
Code tag: ExampleSolarFieldOutput.heliostat_vector_field()
View spec: yz
Path: common\lib\test\data\output\TestSolarFieldOutput\tsfo016_Heliostat_Vector_Field_yz.png

Title:
Heliostat Vector Field
Expand Down
2 changes: 1 addition & 1 deletion .../solarfield/data/input/ExampleSolarFieldOutput/tsfo017_Dense_Tracking_Vector_Field_xy.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Metadata:
Figure number: 1
Name: tsfo017_Dense Tracking Vector Field
Title: Dense Tracking Vector Field
Code tag: ExampleSolarFieldOutput.example_dense_vector_field()
Code tag: ExampleSolarFieldOutput.dense_vector_field()
View spec: xy

Title:
Expand Down
Loading