Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve design consistency in the Prefect Workflow #31

Open
3 of 14 tasks
krlberry opened this issue Feb 5, 2025 · 1 comment
Open
3 of 14 tasks

Improve design consistency in the Prefect Workflow #31

krlberry opened this issue Feb 5, 2025 · 1 comment
Assignees
Labels

Comments

@krlberry
Copy link
Contributor

krlberry commented Feb 5, 2025

Improve design consistency:

  • Update all flows to use a single context throughout the whole workflow
  • Merge more-or-less duplicated stages for import
    • calibration and target data import and prep stages call the same parent flow
    • Update the calibration pipeline stages to include the iterative solver loops used (Partially compeleted)
  • Settle on a consistent “fake data” format (or formats) that all stages of the pipeline will support
    • same type, dimension keys on every data object returned by a stage flow
    • data object returned from simulated archive query task in "data import and prep" stages passed via context all the way to imaging stages
  • Update all files to use the update fake data format:
    • stage_bandpass_solve.py
    • stage_calibrator_import_and_prep.py
    • stage_data_import_and_prep.py
    • stage_findcont.py
    • stage_image_calibrators.py
    • stage_image_cont_selfcal.py
    • stage_image_cube.py
    • stage_image_perspw_cont.py
    • stage_time_gain_solve.py
@krlberry
Copy link
Contributor Author

All flows were updated to use a consistent context.

I've updated stage_time_gain_solve and stage_bandpass_solve to use the common data format, and attempt to fetch this from the context, and use the interactive solve(), but stage_image_calibrators still needs to be updated in this manner as well .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants