Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/subseasonal mpmd plots fix #611

Merged
merged 12 commits into from
Nov 25, 2024
Merged

Feature/subseasonal mpmd plots fix #611

merged 12 commits into from
Nov 25, 2024

Conversation

ShannonShields-NOAA
Copy link
Contributor

Note to developers: You must use this PR template!

Description of Changes

Please include a summary of the changes and the related GitHub issue(s). Please also include relevant motivation and context.
This PR includes output directory changes for subseasonal plotting scripts to address the MPMD bugzilla 1547 (Issue #561).

Developer Questions and Checklist

  • Is this a high priorty PR? If so, why and is there a date it needs to be merged by?
  • Not exactly, but it is highly preferred to complete testing before Thanksgiving and before making changes for METplus RC1.
  • Do you have any planned upcoming annual leave/PTO?
  • No (I am ending my workday at 3pm on 11/22)
  • Are there any changes needed for when the jobs are supposed to run?
  • No
  • The code changes follow NCO's EE2 Standards.
  • Developer's name is removed throughout the code and have used ${USER} where necessary throughout the code.
  • References the feature branch for HOMEevs are removed from the code.
  • J-Job environment variables, COMIN and COMOUT directories, and output follow what has been defined for EVS.
  • Jobs over 15 minutes in runtime have restart capability.
  • If applicable, changes in the dev/drivers/scripts or dev/modulefiles have been made in the corresponding ecf/scripts and ecf/defs/evs-nco.def?
  • Jobs contain the approriate file checking and don't run METplus for any missing data.
  • Code is using METplus wrappers structure and not calling MET executables directly.
  • Log is free of any ERRORs or WARNINGs.

Testing Instructions

Please include testing instructions for the PR assignee. Include all relevant input datasets needed to run the tests.
Checkout feature branch, link FIXevs, set COMIN to EMC parallel stats directory, and set HOMEevs.
Please run all scripts under dev/drivers/scripts/plots/subseasonal.

@malloryprow malloryprow linked an issue Nov 22, 2024 that may be closed by this pull request
subseasonal: Address Bugzilla 1547 - MPMD processes share the same working directory #561
Closed
14 tasks
@malloryprow malloryprow added this to the EVS v2.0.0 milestone Nov 22, 2024
@malloryprow malloryprow self-assigned this Nov 22, 2024
@ShannonShields-NOAA
Copy link
Contributor Author

@malloryprow All commented out/old code has been removed. Thank you for catching that! Due to working extra time earlier this week, I only have half an hour left to my workday, and this testing will take some time in order to be thorough, so let's pick this back up next week on Monday.

@ShannonShields-NOAA
Copy link
Contributor Author

@malloryprow I have checked through the weekend prep and stat files for the EVS parallel and confirmed all is set (the disk quota exceedance over the weekend did not have a meaningful impact). So as long as today's prep and stats run fine (which I am monitoring), then we will be able to test this PR around noon/after the METplus telecon.

@malloryprow
Copy link
Contributor

Sounds good! Thanks for checking! I'm glad the stmp issues didn't cause a major impact to subseasonal.

@ShannonShields-NOAA
Copy link
Contributor Author

@malloryprow Prep and stats ran as they should for today's parallel jobs; we are set for this PR testing!

@malloryprow
Copy link
Contributor

malloryprow commented Nov 25, 2024 •
edited
Loading

Al jobs for testing have been submitted! COMOUT is /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr611/evs/v2.0/plots/subseasonal/atmos.20241123.

1. jevs_subseasonal_grid2grid_pres_lvls_plots_31days

Log File: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr611/EVS/dev/drivers/scripts/plots/subseasonal/jevs_subseasonal_grid2grid_pres_lvls_plots_31days.o207471711
DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_subseasonal_grid2grid_pres_lvls_plots_31days.207471711.dbqs01

2. jevs_subseasonal_grid2grid_pres_lvls_plots_90days

Log File: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr611/EVS/dev/drivers/scripts/plots/subseasonal/jevs_subseasonal_grid2grid_pres_lvls_plots_90days.o207471721
DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_subseasonal_grid2grid_pres_lvls_plots_31days.207471711.dbqs01

3. jevs_subseasonal_grid2grid_sea_ice_plots_31days

Log File: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr611/EVS/dev/drivers/scripts/plots/subseasonal/jevs_subseasonal_grid2grid_sea_ice_plots_31days.o207471820
DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_subseasonal_grid2grid_sea_ice_plots_31days.207471820.dbqs01

4. jevs_subseasonal_grid2grid_sea_ice_plots_90days

Log File: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr611/EVS/dev/drivers/scripts/plots/subseasonal/jevs_subseasonal_grid2grid_sea_ice_plots_90days.o207471834
DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_subseasonal_grid2grid_sea_ice_plots_90days.207471834.dbqs01

5. jevs_subseasonal_grid2grid_sst_plots_31days

Log File: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr611/EVS/dev/drivers/scripts/plots/subseasonal/jevs_subseasonal_grid2grid_sst_plots_31days.o207472067
DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_subseasonal_grid2grid_sst_plots_31days.207472067.dbqs01

6. jevs_subseasonal_grid2grid_sst_plots_90days

Log File: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr611/EVS/dev/drivers/scripts/plots/subseasonal/jevs_subseasonal_grid2grid_sst_plots_90days.o207472132
DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_subseasonal_grid2grid_sst_plots_90days.207472132.dbqs01

7. jevs_subseasonal_grid2grid_temp_plots_31days

Log File: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr611/EVS/dev/drivers/scripts/plots/subseasonal/jevs_subseasonal_grid2grid_temp_plots_31days.o207472194
DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_subseasonal_grid2grid_temp_plots_31days.207472194.dbqs01

8. jevs_subseasonal_grid2grid_temp_plots_90days

Log File: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr611/EVS/dev/drivers/scripts/plots/subseasonal/jevs_subseasonal_grid2grid_temp_plots_90days.o207472259
DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_subseasonal_grid2grid_temp_plots_90days.207472259.dbqs01

9. jevs_subseasonal_grid2obs_prepbufr_plots_31days

Log File: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr611/EVS/dev/drivers/scripts/plots/subseasonal/jevs_subseasonal_grid2obs_prepbufr_plots_31days.o207472351
DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_subseasonal_grid2obs_prepbufr_plots_31days.207472351.dbqs01

10. jevs_subseasonal_grid2obs_prepbufr_plots_90days

Log File: /lfs/h2/emc/vpppg/noscrub/mallory.row/verification/EVS_PRs/pr611/EVS/dev/drivers/scripts/plots/subseasonal/jevs_subseasonal_grid2obs_prepbufr_plots_90days.o207472380
DATA: /lfs/h2/emc/stmp/mallory.row/evs_test/prod/tmp/jevs_subseasonal_grid2obs_prepbufr_plots_90days.207472380.dbqs01

@ShannonShields-NOAA
Copy link
Contributor Author

@malloryprow and @AliciaBentley-NOAA I have thoroughly checked all of the output for each plot job (like making sure the individual job numbers match between plot_job_scripts and plot_output/job_work_dir) and everything looks correct. I quickly checked the plot tar files and confirmed that nothing looks out of place with the plots. This PR should be set to merge.

malloryprow
malloryprow approved these changes Nov 25, 2024
View reviewed changes
Copy link
Contributor

@malloryprow malloryprow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes are good and testing successful.

AliciaBentley-NOAA
AliciaBentley-NOAA approved these changes Nov 25, 2024
View reviewed changes
Copy link
Contributor

@AliciaBentley-NOAA AliciaBentley-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have reviewed the changes in this PR and think that they look great! Test results confirm this. Please merge into develop when you are ready. Thanks!

@ShannonShields-NOAA @malloryprow

@malloryprow malloryprow merged commit 5060d4a into NOAA-EMC:develop Nov 25, 2024
@ShannonShields-NOAA ShannonShields-NOAA deleted the feature/subseasonal_mpmd_plots_fix branch November 25, 2024 19:40
BinbinZhou-NOAA added a commit to BinbinZhou-NOAA/EVS that referenced this pull request Nov 26, 2024
@BinbinZhou-NOAA
Merge branch 'feature/href_fixes' of https://github.com/BinbinZhou-NO…
dd146f1 
…AA/EVS into feature/href_fixes

* 'feature/href_fixes' of https://github.com/BinbinZhou-NOAA/EVS:
  Feature/subseasonal mpmd plots fix (NOAA-EMC#611)
  feature/EVS-NWPS_regions (NOAA-EMC#609)
  Feature/sref fixes (NOAA-EMC#607)
  Fixing bug in the ceiling processing for the perfdiag (NOAA-EMC#608)
  Mesoscale NAM/RAP Fixes and Additions - no restart (NOAA-EMC#603)
  Feature/subseasonal mpmd fix for stats (NOAA-EMC#602)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@AliciaBentley-NOAA AliciaBentley-NOAA AliciaBentley-NOAA approved these changes

@malloryprow malloryprow malloryprow approved these changes

Assignees

@malloryprow malloryprow

Labels
NCO Bugzilla
Projects
None yet
Milestone
EVS v2.0.0
Development

Successfully merging this pull request may close these issues.

subseasonal: Address Bugzilla 1547 - MPMD processes share the same working directory
3 participants
@ShannonShields-NOAA @malloryprow @AliciaBentley-NOAA