Capgen in SCM: Differing source and module name #637
Conversation
dustinswales
requested review from
climbfuji,
gold2718,
mwaxmonsky,
peverwhee,
grantfirl and
mkavulich
as code owners
scripts/metadata_table.py
Outdated
| # DJS2024: First, try to find module_name from the metadata. Otherwise, | ||
| # use file name as module_name (default). | ||
| self.__module_name = find_module_name(self.__pobj.filename) | ||
| if (self.__module_name == ''): |
There was a problem hiding this comment.
How can this happen? We require all Fortran code to be in modules, schemes or not?
There was a problem hiding this comment.
The metadata is only needed for the DDT, not the module where it lives.
The behavior now is that IF you provide a module table entry in the DDT metadata file it will use that instead.
| @@ -223,6 +223,51 @@ def parse_metadata_file(filename, known_ddts, run_env, skip_ddt_check=False): | |||
|
|
|||
| ######################################################################## | |||
|
|
|||
| def find_module_name(filename): | |||
There was a problem hiding this comment.
I am surprised that this isn't already done by the parser. Is that because everything is based on the assumption filename=modulename?
There was a problem hiding this comment.
Yeah. Metadata tables for new DDTs default to the source filename (e.g. here). Not the best.
This is a problem for DDTs in two instances, a) DDTs housed in differing module/file names and b) DDTs in files with multiple modules.
We don't have any of the latter in the SCM/UFS, so I haven't gone down that rabbit hole.
| @@ -657,7 +657,8 @@ def __init__(self, sdfs, host_model, scheme_headers, run_env): | |||
| self.__ddt_lib = DDTLibrary('{}_api'.format(self.host_model.name), | |||
| run_env, ddts=all_ddts) | |||
| for header in [d for d in scheme_headers if d.header_type != 'ddt']: | |||
| if header.header_type != 'scheme': | |||
| # DJS2024: Schemes and modules with DDTs? | |||
| if header.header_type != 'scheme' and header.header_type != 'module': | |||
There was a problem hiding this comment.
@climbfuji Here's another thing I needed to add. Capgen wasn't happy with modules that held DDT definitions (e.g. radlw_param.f).
I convinced myself that this is not intentional and probably due to a lack of testing Capgen with respect to DDTs.
| @@ -20,7 +20,7 @@ get_filename_component(CCPP_ROOT "${TEST_ROOT}" DIRECTORY) | |||
| # | |||
| #------------------------------------------------------------------------------ | |||
| LIST(APPEND SCHEME_FILES "var_compatibility_files.txt") | |||
| LIST(APPEND HOST_FILES "test_host_data" "test_host_mod") | |||
| LIST(APPEND HOST_FILES "module_rad_ddt" "test_host_data" "test_host_mod") | |||
There was a problem hiding this comment.
@gold2718 @peverwhee @climbfuji Design philosophy question below.
When a physics scheme has a native DDT that the host needs to be aware of, the file containing the DDT needs to be at the top of both the scheme_files and host_files list (e.g. host_files and scheme_files in CCPP SCM). In the SCM/UFS, we also do the inverse and use host defined DDTs within Interstitial physics schemes, so there is a need for a pool of DDTs available to both the host and the schemes.
Would it make sense to have a third class of files provided to Capgen, containing the DDT typedefs, that are processed before the scheme and host files?
I'm fine with having to define the host/scheme lists with the typedefs at the top, just curious to hear your thoughts on this? (I'm also not volunteering to do this!)
There was a problem hiding this comment.
That third class would be a mix of scheme and host files. Something like this exists in prebuild, see for example https://github.com/NOAA-EMC/fv3atm/blob/2c902a670e89416ef49254c827e8ba45a68ce596/ccpp/config/ccpp_prebuild_config.py#L12
[ 50 character, one line summary ]
Fortran does not require module names be the same as the source file in which they are defined.
This PR allows for this in Capgen
User interface changes?: No
Fixes: #636
Testing:
Expanded var_compatability_test