Should fpm be compiler independent as much as possible with regards to flags?

[certik]

See for example #112.

We need to be able to specify specific compiler flags per file as an option, in order to allow people to get their stuff built. However, I am worried that we could become just like "cmake", where each project hardwires compiler flags as needed.

Rather, I would like to encourage fpm to know as much as possible, such as good default Debug / Release flags, and -ffast-math handling for all compilers, and any other such flags in a compiler independent manner. That way, when we want to add support for a new compiler, we just have to modify fpm, and packages should (in theory) just work, if they only use the fpm integrated facilities, as opposed to compiler dependent flags directly.

Another way to look at it: if each package adds -ffast-math as a direct compiler flag, then fpm does not know the "semantics" of it, it is just an opaque compiler flag, so it cannot help. But if each package uses fast-math built-in fpm flag, then fpm knows exactly what semantics the user is trying to achieve, and can ensure and check that all packages are compatible with it and that things just work in any compiler. And we can iteratively keep improving fpm's handling of fast-math to get better error messages and better support, as we discover bugs. We do not need to fix every package, we just have to improve fpm. The same with any other flag.

Do you agree / disagree?

[LKedward]

I think this is a fair point, particularly with recent discussion now moving towards a two- or three- dimensional space of possible flag combinations (compiler, build profile, flag type/scope). This will inevitably lead to quite lengthy manifest entries.

I agree that having fpm understand and abstract common compiler flags would be greatly beneficial. Ivan @ivan-pi has nicely summarised different compiler optimisation flags in #112 (comment) and compiler runtime checks in #50 (comment).

With that said, I think we are always going to need a way for developers to manually specify flags, which leads us back to the current proposed approach under discussion.

I think the current plan is to move forward with #112/#350 and then use the resulting infrastructure in fpm to provide abstractions to common features such as fast-math.

[certik]

I wonder if we should simply go slowly and rather design flags in fpm properly first. The minute we allow flags to be set ad-hoc, as in cmake, people will simply start doing that, and as a result, we loose this nice feature of fpm, that fpm knows the semantics of what you are doing.

Using compiler dependent flags directly should always only be used as the last measure to get something working, but it is always a sign that fpm is not good enough and needs improving.

Let me rephrase: I believe that enabling to use flags directly is actually not fixing the root of the problem, and as such it is a workaround. I think this is the wrong way to go.

One way to go forward is to always emit an fpm warning, that a compiler dependent flag was used. That way the user will always be reminded that this is not the long term solution, just a short term quick fix to get a package working, but the correct fix is to use fpm built-in features that are compiler independent and allow fpm to give much better error messages and it is more maintainable in the long term.

I have a lot of red flags about this in my head, this is a very important design decision and we should think it through before going blindly and implementing compiler dependent features without any warnings.

CC @everythingfunctional, @milancurcic, @awvwgk.

[LKedward]

The minute we allow flags to be set ad-hoc, as in cmake, people will simply start doing that, and as a result, we loose this nice feature of fpm, that fpm knows the semantics of what you are doing.

Using compiler dependent flags directly should always only be used as the last measure to get something working, but it is always a sign that fpm is not good enough and needs improving.

This is a good point which I agree with. Since Sebastian and Brad have proposed much of the current approach, I will defer to what they have to say on this.

CC @kubajj

[certik]

In some sense it is about priorities --- we need to get many packages working, but at the same time we should not jeopardize our goal and the great opportunity that we have with fpm.

Here is a practical proposal that I think will work: for now, let's emit the following warning (that can be turned off with some command line option like --Woff or --Wno-warnings) every time any compiler dependent flag is used in fpm.toml:

Warning: a compiler dependent flag was used in fpm.toml. We have enabled that feature so that packages
can be built today, however the long term maintainable solution is to improve fpm itself to handle it in
a compiler independent way and understanding the semantics. Please request the feature you need at
https://github.com/fortran-lang/fpm/issues, unless it was already requested.

[milancurcic]

but it is always a sign that fpm is not good enough and needs improving.

I don't understand why that is a sign that fpm is not good enough. Some applications need some source files compiled with -O2, because they break with -O3. And same levels of -O[0123] also vary between compilers. I don't think there's much fpm could do about that, other than adoping a de facto compiler and not support others (if you were to not allow fine-tuning of flags, of course).

[ivan-pi]

Some applications need some source files compiled with -O2, because they break with -O3

Reminds me of this tweet from @jeffhammond: https://twitter.com/science_dot/status/1380323169127387143

[everythingfunctional]

I think this is nearly orthogonal to being able to specify compiler flags manually. And the amount of effort required for every option we want to support in a "compiler agnostic" way will be huge. With such a variety of compiler flags and their meanings, each one will have to be discussed ad nauseam, probably putting together a large number of different tables and running lots of tests.

I think we should design such a feature to work in tandem with a "features" capability similar to what Cargo has. Basically some intrinsic features that fpm knows the semantics about, and involves adding the appropriate compiler flags with the proper "propagation" up and/or down the dependency tree.

I don't think the proposed solution for manually specifying compiler flags (#350) will preclude such a functionality. I do think they will possibly end up being mutually exclusive (i.e. you can't manually specify compiler flags and use "intrinsic features", as it's quite possible to end up specifying conflicting flags that way). For that reason I think adding a warning when using manually specified compiler flags is perfectly reasonable, including the --flag command line option.

[milancurcic]

I think that user not having to specify flags should be a high priority for fpm. This warrants "sane" profiles like "debug" and "release" that we already have for gfortran (maybe ifort too?), and I'd suggest even adding a 3rd profile called "fast", which would include things like -O3 and -ffast-math.

fpm should also be inclusive, so when a user really needs to set a specific flag per source file, they can.

I don't think sane profiles and custom flags are at odds.

[everythingfunctional]

I've been wanting a profile called "strict", which turns on all possible compiler warnings and makes them errors. I want to know about everything that even might be wrong with my code.

[jeffhammond]

Some applications need some source files compiled with -O2, because they break with -O3 Reminds me of this tweet from @jeffhammond: https://twitter.com/science_dot/status/1380323169127387143

It’s good to allow mixing but in the case of my tweet, the bug existed at all levels of optimization. However, I’m aware of plenty of cases where mixing -O2 and -Ofast (or equivalent) is necessary or useful.

[awvwgk]

Smart features like automatic compiler independent profiles are certainly a nice idea, but I don't think they exclude fine-grained low-level access to specify compiler flags. Nothing is more frustrating than running into a smart feature that knows what you want to do better than yourself without a mean to circumvent it.

I think we might even be able to implement the necessary logic for handling something like fast-math as an fpm package, if we design the low-level access to the compiler flags correctly. Maybe a community solution to the fast-math issue could come from the fpm-registry as:

[dependencies]
fast-math = "no-fast-math"

Maybe the fast-math package and its no-fast-math version will be a horrible mess of package manifest to handle all the nifty details for different compiler vendors, but it would be a well contained and manageable mess, downstream your package manifest will be clean and expressive.

[certik]

I don't have all the answers, but I would like us to keep discussing this and figure out a good design.

Another angle is that I want to be able to compile the packages in various "profiles", such as Debug, Release, Strict, fast-math, no fast-math, etc. Ideally I want to set this at a command line for the final application, and fpm would recompile everything correctly.

So if we allow to specify compiler options directly, it needs to play well with that design.

In the "Release" profile, I know that different people have different requirements. Some people want strict standard conformance, and within that as fast as possible (e.g., no -ffast-math). Some other people want the fastest code, no matter what (e.g., -ffast-math). It would be nice to figure out the most common "profiles" that would cover most people's needs.

Part of the solution could be not to hardwire all this into fpm itself, but somehow allow the user to modify / customize the "profiles" somehow.

[awvwgk]

To have user defined profiles we could work towards an fpm config file, maybe under $PREFIX/share/fpm/config.toml or $HOME/.fpm/config.toml which can contain compiler profiles. Sharing those profiles, maybe as special part of the fpm-registry, could give us a feedback how people are using fpm and also various Fortran compilers.

[milancurcic]

I agree custom profiles will be important. I don't know for sure, but I think custom profiles would be more specific to the project (manifest approach) than the user (fpm config file). Perhaps something like this in the manifest (it may have been already proposed elsewhere):

[profile]
[profile.gnu]
[profile.gnu.release] # overrides default GNU release profile
flags = "-O2 -std=f2018"

[profile.gnu.mycustomprofile] # sets a new custom GNU profile
flags = "-O1 -fbacktrace -fcheck=bounds"

[profile.intel] # ditto for Intel profiles
...

If no profiles are set in the manifest, default ones are available (e.g. release, debug, fast). If a profile with an existing name is added to the manifest, user will override the default profile. If a new profile is added to the manifest, it will allow the user to use that profile, e.g. fpm run --profile mycustomprofile.

This mechanism would also allow the user to define profiles for a compiler not yet supported by fpm, without having to patch fpm itself.

It would be nice to figure out the most common "profiles" that would cover most people's needs.

An easy way to start would be to poll fpm contributors (in GitHub issue or on Discourse) and use those as a first guess. Then let users suggest changes and we'd evaluate them on a case-by-case basis.

[certik]

What @milancurcic is proposing is moving more and more towards the Spack model (which I was going to suggest people to look into and be aware of for our discussion), where each package exposes "features" that is supports (typically compiling with some optional dependencies, debug/release mode, etc.), and then you select those from a command line, such as:

https://spack.readthedocs.io/en/latest/basic_usage.html#specs-dependencies

You can even specify per package compiler flags. It's a good way of handling these issues for a source package manager.

[everythingfunctional]

What @milancurcic just suggested is very similar to what I proposed in #350. @kubajj proposal for GSoC is to implement that.

[awvwgk]

Another option to search for community compiler profiles would be https://fortran-lang.org/packages. This data mining effort is a lot of work, because hunting for compiler profiles in build systems is one of the least fun tasks I could imagine.

[certik]

Let's discuss it at the Fortran call this Thursday: https://fortran-lang.discourse.group/t/fortran-monthly-call-april-2021/954/6?u=certik.

The proposal at #350 is fine, but ultimately it just allows to specify compiler dependent flags that are opaque to fpm. That is only the first step. I would like to brainstorm and discuss ways to improve fpm so that one does not have to use that.

Perhaps let's list the use cases. Here are some off the top of my head that I had to handle in the past, I'll just list them for gfortran, there are similar options for other compilers. Here are options that you have to specify otherwise the code would not even compile:

• -std=legacy for older code (perhaps there are other ways to do it)
• -fallow-argument-mismatch for mpich and apparently QUADPACK
• -ffree-line-length-none to allow longer line lengths (especially if there is some macro expansion)
• -cpp to manually trigger the preprocessor if .f90 is used instead of .F90
• -ffree-form, -ffixed-form to force source type if it cannot be inferred correctly from the extension
• -fPIC (depends on the platform)
• -fdefault-real-8 (some codes require it)

Then there are options regarding correctness of output / optimizations:

• -ffast-math (some codes work with it, some don't, and some users want it, some don't)
• -O3, -Ofast sometimes could be too aggressive
• -funroll-loops, -march=native.

Then there are warning options that I frequently use, this is related to Debug/Release:

• -Wall -Wextra: enables most warnings
• -Wimplicit-interface: warns about calling functions implicitly
• -Werror: turns warning into errors, used at the CI
• -g: debugging symbols
• -fcheck=all: arrays bounds checking
• -fbacktrace: prints good stacktraces when a runtime error occurs
• -ffpe-trap=invalid,zero,overflow -finit-real=snan -finit-integer=-99999999 initializes all floating point numbers to nan and gives a runtime error when a nan is used (with a stacktrace if the previous flag is used), it can't do it for integers, but at least it initializes them to a very large negative number which almost always breaks the code (such as array bounds checking) at some point in practice.

Enabling features:

• -fopenmp enables OpenMP
• GPU options, especially for other compilers
• MPI options (Support MPI natively #354)
• -static Sometimes (but not always) I like to link applications statically for easier distribution / faster startup.

I think these are most of the options that I use, so it's not that many actually (given that GFortran has hundreds of options I believe).

So for my use cases, at the very least fpm would have to support all of the above, so that I don't have to use flags directly.

@everythingfunctional, @milancurcic, @awvwgk and others, why don't you list all the flags flags that you use? Once we have most of the use cases, let's brainstorm how to make fpm aware of these, that will cover 99% of use cases, and then for the corner cases one will simply use compiler options directly.

@everythingfunctional you mentioned that you are worried about the amount of effort to support every option. I suggest we support the most used options, but not every option for every compiler. I only use about ~20 options (listed above) out of hundreds, so it's a very small percentage, yet it covers very large percentage of use cases.

The general design that we have been following so far for fpm is to support the most used scenarios (90% or more) natively and semi-automatically, with minimal effort, and for the rest 10% or less we simply allow it one way or another (specifying compiler flags directly, or specifying a Bash build script to build the package).

[everythingfunctional]

The only other flag I can think of that you didn't cover is -std=f2018.

I think the idea of "features" would work well for this, with a couple of caveats:

• We should try to pick features that are orthogonal for all compilers (i.e. does -fbacktrace imply -g? I think it probably does, at least for some compilers, so those wouldn't be orthogonal), and we should try to make sure we don't pick features that could be conflicting for some compilers (at least for ones likely to be used together)
• Looking at your examples it seems we probably want "features" to be specified per profile. I.e. I want to turn on trapping uninitialized data for Debug but not for Release)

I think we should look at the most common usage patterns we see emerging when we allow developers to specify their own profiles and flags manually, and use that to guide us.

[certik]

@everythingfunctional yes, exactly.

One other distinction, besides what you said, would be to split the options for things that are optional (e.g., Debug vs Release flags) and those that have to be specified otherwise the package will no even build, or it would work incorrectly. That is the first group. For that, having some compiler independent options like long-lines, fixed-form, default-real-8, etc. and one would simply set them in fpm.toml and fpm would make sure it works in all compilers. I think these options would not go into a "profile".

For the optional flags we have to figure out some "profile" approach. But if the above options is all that we are talking about in practice (more or less), I am positive we can find some really good design.

[MarDiehl]

I have sets for Gfortran and Intel
https://github.com/eisenforschung/DAMASK/blob/master/cmake/Compiler-GNU.cmake
https://github.com/eisenforschung/DAMASK/blob/master/cmake/Compiler-Intel.cmake

and (untested) PGI/Nvidia
https://github.com/eisenforschung/DAMASK/blob/master/cmake/Compiler-PGI.cmake

[awvwgk]

This is what is used in the conda-build setup for GFortran on conda-forge in a native compilation.

Micro architecture cross-compilation (very architecture specific):

• -march=... fallback architecture, either native for usage on a single machine or something old for distribution
• -mtune=... additional microarchitectures to create additional optimized code paths for (mainly for distribution)

Security:

• -ftree-vectorize: Perform vectorization on trees
• -fPIC generate position-independent code suitable for linking dynamic libraries
• -fstack-protector-strong extra checks against buffer overflows
• -fno-plt use global offsets table instead of procedure linkage table for external function calls
• -fvar-tracking-assignments improved debugging information

Compiler performance:

• -ffunction-sections put each function in a separate section, allows linker to eliminate unused functions
• -pipe use pipes instead of temporary files for communication in compiler

Packaging:

• -isystem change of system root
• -fdebug-prefix-map remapping feature for use with system root

[awvwgk]

There are some options that go with non-standard extensions, some compilers like NVHPC have enabled non-standard extensions by default and provide arguments to disable those.

C style escape character:

• gfortran off by default, enable with -fbackslash
• ifort off by default, enable with -assume bscc
• nvfortran on by default, disable with -Mbackslash
• (classic) flang on by default, disable with -Mbackslash

[certik]

I've also used:

• GFortran: -fno-realloc-lhs
• Intel: -standard-semantics to fix an issue with logical interoperability

[MarDiehl]

I would prefer to stay as close as possible to the latest standard. -fno-realloc-lhs should be enabled by a package maintainer if needed, but we should nudge people in the opposite direction by not setting it as default.

[awvwgk]

-standard-semantics is one of ifort flags that creates issues for me on a regular basis. From linking errors if specified inconsistently in the dependency chain to spurious internal compiler errors (stdlib doesn't compile with -standard-semantics).

What you want is -fpscomp logicals to fix this compatibility issue.

[certik]

Good points.

Btw, my understanding is that if a package compiles and works with -fno-realloc-lhs, then it will also work without this option (perhaps a bit slower), but not the other way around.

[MarDiehl]

I think

real, dimension(3) :: a = 1.0
real, dimension(2) :: b = 0.0

a = b
print*, a

is technically undefined for pre-lhs-realloc, but usually gives

(0.0, 0,0 1.0)

while the current standard would give

(0.0, 0.0)

so if a code needs -fno-realloc-lhs it is based on undefined behavior which is normally handled in a sensible way.

[certik]

The codes I've seen that use -fno-realloc-lhs usually use it for performance reasons, as compilers used to generate faster code, and also alerted you with a runtime error if you forgot to allocate first. I haven't done any benchmarking to see if it is still the case.