Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: years rendered inconsistently with processed filenames for ts_atm_monthly_180x360_* and ts_land_monthly_* #588

Closed
wlin7 opened this issue Apr 27, 2024 · 2 comments
Closed

[Bug]: years rendered inconsistently with processed filenames for ts_atm_monthly_180x360_* and ts_land_monthly_* #588

wlin7 opened this issue Apr 27, 2024 · 2 comments
Labels
semver: bug Bug fix (will increment patch version)

Comments

@wlin7
Copy link

wlin7 commented Apr 27, 2024

What happened?

zppy for tasks of ts_atm_month_180x360_* and ts_land_monthly_* got errors like
cp: cannot stat '/lcrc/group/e3sm2/ac.wlin/E3SMv3/v3.LR.abrupt-4xCO2_0101_L/post/atm/180x360_aave/ts/monthly/10yr/*_1??_10??.nc': No such file or directory

This is for a time period of years 0001-0010. Look like the zppy generated bash file does not use 4-digit year for some path setting.
For example, for ts_atm_monthly_180x360_aave_0001-0010-0010.bash, it contains the following

cp -s $dest/*_1??_10??.nc $input_dir, which would not match the processed data in the form like FLUT_000101_001012.nc

What machine were you running on?

chrysalis

Environment

zppy in e3sm_unified_1.9.3

What command did you run?

zppy -c /home/ac.wlin/E3SM/utils/post.v3.LR.abrupt-4xCO2_0101_L.cfg

Copy your cfg file

Below is a copy of the zppy generated problematic .bash file. 
/lcrc/group/e3sm2/ac.wlin/E3SMv3/v3.LR.abrupt-4xCO2_0101_L/post/scripts/ts_atm_monthly_180x360_aave_0001-0010-0010.bash.0

(ts_atm_monthly_180x360_aave_0001-0010-0010.bash is manually modified and manually submitted to get the processing done)


The source cfg file is
/home/ac.wlin/E3SM/utils/post.v3.LR.abrupt-4xCO2_0101_L.cfg

What jobs are failing?

No response

What stack trace are you encountering?

cp: cannot stat '/lcrc/group/e3sm2/ac.wlin/E3SMv3/v3.LR.abrupt-4xCO2_0101_L/post/atm/180x360_aave/ts/monthly/10yr/*_1??_10??.nc': No such file or directory
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:143) >> --------------------------------------
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:143) >> --------------------------------------
2024-04-27 15:36:28,281_281:INFO:__init__:--------------------------------------
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:144) >> | E3SM to CMIP Configuration
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:144) >> | E3SM to CMIP Configuration
2024-04-27 15:36:28,281_281:INFO:__init__:| E3SM to CMIP Configuration
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:145) >> --------------------------------------
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:145) >> --------------------------------------
2024-04-27 15:36:28,281_281:INFO:__init__:--------------------------------------
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:146) >>     * var_list='['pr', 'tas', 'rsds', 'rlds', 'rsus']'
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:146) >>     * var_list='['pr', 'tas', 'rsds', 'rlds', 'rsus']'
2024-04-27 15:36:28,281_281:INFO:__init__:    * var_list='['pr', 'tas', 'rsds', 'rlds', 'rsus']'
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:147) >>     * input_path='/lcrc/group/e3sm2/ac.wlin/E3SMv3/v3.LR.abrupt-4xCO2_0101_L/post/atm/180x360_aave/ts/monthly/10yr/1_10'
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:147) >>     * input_path='/lcrc/group/e3sm2/ac.wlin/E3SMv3/v3.LR.abrupt-4xCO2_0101_L/post/atm/180x360_aave/ts/monthly/10yr/1_10'
2024-04-27 15:36:28,281_281:INFO:__init__:    * input_path='/lcrc/group/e3sm2/ac.wlin/E3SMv3/v3.LR.abrupt-4xCO2_0101_L/post/atm/180x360_aave/ts/monthly/10yr/1_10'
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:148) >>     * output_path='/lcrc/group/e3sm2/ac.wlin/E3SMv3/v3.LR.abrupt-4xCO2_0101_L/post/atm/180x360_aave/cmip_ts/monthly/tmp_ts_atm_monthly_180x360_aave_0001-0010-0010'
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:148) >>     * output_path='/lcrc/group/e3sm2/ac.wlin/E3SMv3/v3.LR.abrupt-4xCO2_0101_L/post/atm/180x360_aave/cmip_ts/monthly/tmp_ts_atm_monthly_180x360_aave_0001-0010-0010'
2024-04-27 15:36:28,281_281:INFO:__init__:    * output_path='/lcrc/group/e3sm2/ac.wlin/E3SMv3/v3.LR.abrupt-4xCO2_0101_L/post/atm/180x360_aave/cmip_ts/monthly/tmp_ts_atm_monthly_180x360_aave_0001-0010-0010'
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:149) >>     * precheck_path='None'
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:149) >>     * precheck_path='None'
2024-04-27 15:36:28,281_281:INFO:__init__:    * precheck_path='None'
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:150) >>     * freq='mon'
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:150) >>     * freq='mon'
2024-04-27 15:36:28,281_281:INFO:__init__:    * freq='mon'
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:151) >>     * realm='atm'
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:151) >>     * realm='atm'
2024-04-27 15:36:28,281_281:INFO:__init__:    * realm='atm'
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:152) >>     * Writing log output file to: logs/20240427_153628_277113
2024-04-27 15:36:28,281 [INFO]: __main__.py(__init__:152) >>     * Writing log output file to: logs/20240427_153628_277113
2024-04-27 15:36:28,281_281:INFO:__init__:    * Writing log output file to: logs/20240427_153628_277113
Traceback (most recent call last):
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.9.3_chrysalis/bin/e3sm_to_cmip", line 10, in <module>
    sys.exit(main())
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.9.3_chrysalis/lib/python3.10/site-packages/e3sm_to_cmip/__main__.py", line 964, in main
    app = E3SMtoCMIP(args)
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.9.3_chrysalis/lib/python3.10/site-packages/e3sm_to_cmip/__main__.py", line 154, in __init__
    self.handlers = self._get_handlers()
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.9.3_chrysalis/lib/python3.10/site-packages/e3sm_to_cmip/__main__.py", line 206, in _get_handlers
    e3sm_vars = self._get_e3sm_vars(self.input_path)
  File "/lcrc/soft/climate/e3sm-unified/base/envs/e3sm_unified_1.9.3_chrysalis/lib/python3.10/site-packages/e3sm_to_cmip/__main__.py", line 280, in _get_e3sm_vars
    raise IndexError(
IndexError: No variables were found in the input file(s) at '/lcrc/group/e3sm2/ac.wlin/E3SMv3/v3.LR.abrupt-4xCO2_0101_L/post/atm/180x360_aave/ts/monthly/10yr/1_10'.
@wlin7 wlin7 added the semver: bug Bug fix (will increment patch version) label Apr 27, 2024
@wlin7 wlin7 changed the title [Bug]: years rendered inconsistently for ts_atm_monthly_180x360_* and ts_land_monthly_* [Bug]: years rendered inconsistently with processed filenames for ts_atm_monthly_180x360_* and ts_land_monthly_* Apr 27, 2024
@forsyth2
Copy link
Collaborator

forsyth2 commented Apr 29, 2024 •
edited
Loading

@wlin7 I believe @chengzhuzhang has fixed this with #574, which will be included in zppy v2.5.0 (which will be in E3SM Unified 1.10.0)

@wlin7
Copy link
Author

wlin7 commented Apr 30, 2024

Thanks @forsyth2 . Yes, #574 should fix this. The issue was somehow introduced recently.

@wlin7 wlin7 closed this as completed Apr 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
semver: bug Bug fix (will increment patch version)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@wlin7 @forsyth2