CL.EXE is not able to compile a simple test program. when using CMake in certain environments (GHA)

[mmomtchev]

https://github.com/mmomtchev/proj.js/actions/runs/8471786425/job/23212574144

The Windows build has a very sneaky problem:

| CMake Error at C:/Users/runneradmin/AppData/Roaming/xPacks/@xpack-dev-tools/cmake/3.26.5-1.1/.content/share/cmake-3.26/Modules/CMakeTestCCompiler.cmake:67 (message):
| The C compiler

| "C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/MSVC/14.38.33130/bin/HostX64/x64/cl.EXE"

| is not able to compile a simple test program.

| It fails with the following output:

| Change Dir: D:/a/proj.js/proj.js/build/native/subprojects/PROJ/__CMake_build/CMakeFiles/CMakeScratch/TryCompile-79bcw7

| Run Build Command(s):D:/a/proj.js/proj.js/xpacks/.bin/ninja -v cmTC_399ea && [1/2] C:\PROGRA~1\MICROS~2\2022\ENTERP~1\VC\Tools\MSVC\1438~1.331\bin\HostX64\x64\cl.EXE  /nologo   /DWIN32 /D_WINDOWS  /Zi /Ob0 /Od /RTC1 -MDd /showIncludes /FoCMakeFiles\cmTC_399ea.dir\testCCompiler.c.obj /FdCMakeFiles\cmTC_399ea.dir\ /FS -c D:\a\proj.js\proj.js\build\native\subprojects\PROJ\__CMake_build\CMakeFiles\CMakeScratch\TryCompile-79bcw7\testCCompiler.c
| [2/2] cmd.exe /C "cd . && C:\Users\runneradmin\AppData\Roaming\xPacks\@xpack-dev-tools\cmake\3.26.5-1.1\.content\bin\cmake.exe -E vs_link_exe --intdir=CMakeFiles\cmTC_399ea.dir --rc=D:\a\proj.js\proj.js\node_modules\.bin\rc --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x64\mt.exe --manifests  -- C:\PROGRA~1\MICROS~2\2022\ENTERP~1\VC\Tools\MSVC\1438~1.331\bin\HostX64\x64\link.exe /nologo CMakeFiles\cmTC_399ea.dir\testCCompiler.c.obj  /out:cmTC_399ea.exe /implib:cmTC_399ea.lib /pdb:cmTC_399ea.pdb /version:0.0 /machine:x64  /debug /INCREMENTAL /subsystem:console  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
| FAILED: cmTC_[399](https://github.com/mmomtchev/proj.js/actions/runs/8471786425/job/23212574144#step:7:400)ea.exe
| cmd.exe /C "cd . && C:\Users\runneradmin\AppData\Roaming\xPacks\@xpack-dev-tools\cmake\3.26.5-1.1\.content\bin\cmake.exe -E vs_link_exe --intdir=CMakeFiles\cmTC_399ea.dir --rc=D:\a\proj.js\proj.js\node_modules\.bin\rc --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x64\mt.exe --manifests  -- C:\PROGRA~1\MICROS~2\2022\ENTERP~1\VC\Tools\MSVC\1438~1.331\bin\HostX64\x64\link.exe /nologo CMakeFiles\cmTC_399ea.dir\testCCompiler.c.obj  /out:cmTC_399ea.exe /implib:cmTC_399ea.lib /pdb:cmTC_399ea.pdb /version:0.0 /machine:x64  /debug /INCREMENTAL /subsystem:console  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
| LINK Pass 1: command "C:\PROGRA~1\MICROS~2\2022\ENTERP~1\VC\Tools\MSVC\1438~1.331\bin\HostX64\x64\link.exe /nologo CMakeFiles\cmTC_399ea.dir\testCCompiler.c.obj /out:cmTC_399ea.exe /implib:cmTC_399ea.lib /pdb:cmTC_399ea.pdb /version:0.0 /machine:x64 /debug /INCREMENTAL /subsystem:console kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIFESTFILE:CMakeFiles\cmTC_399ea.dir/intermediate.manifest CMakeFiles\cmTC_399ea.dir/manifest.res" failed (exit code 1104) with the following output:
| LINK : fatal error LNK1104: cannot open file 'CMakeFiles\cmTC_399ea.dir\manifest.res'
| ninja: build stopped: subcommand failed.

I spent hours looking for the problem - do you see the --rc CLI option. rc.exe is a Windows Visual Studio utility that generates manifests. xpm install an npm package that happens to install node_modules/.bin/rc which - depending on the environment - can take precedence. It does not happen when using a system-installed CMake - there must be something with the system paths when xpm installs the environment.

[ilg-ul]

Thank you for investigating. However this is not a cmake-xpack bug, neither an xpm bug, but a use case issue, the unwanted program is there because you have a reference to a npm package that added it there.

Did you identify which node package installed that unwanted rc?

xpm install an npm package that happens to install node_modules/.bin/rc

It is true that early xpm installed npm packages too, but recent xpm no longer installs npm packages, xpm installs only xpm packages in xpacks and npm installs npm packages in node_modules.

The environment used by xpm run is pretty deterministic, and is controlled by the order of the paths, with xpacks/.bin on top, then node_modules/.bin then the user path.

So, if you want reproducible builds, you must be sure all your desired tools are either in xpacks or in node_modules; the user/system path is not a reliable location and must be avoided.

[ilg-ul]

I took a look at your configuration, it is pretty complicated, with lots of npm dependencies.

If you need reproducibility, you probably have to modularise your project, for example move the native part in a separate folder, have a package.json there with the xpm configuration only and no npm dependencies, and invoke xpm run action -C folder directly from the top CI script, not from the top npm, since the top npm run will add the unwanted tools to your path.

[mmomtchev]

[email protected] /home/mmom/src/proj.js
└─┬ [email protected]
  └─┬ @ilg/[email protected]
    └─┬ [email protected]
      └─┬ [email protected]
        ├─┬ [email protected]
        │ └── [email protected]
        └─┬ [email protected]
          └── [email protected] deduped

[ilg-ul]

Ok, so it is a deep dependency, that cannot be avoided directly.

One possible workaround is to install xpm with -g, right after installing node, and it will no longer appear in node_modules (and hope that no other package has a dependency on rc).

Another suggestion is to use gcc instead of cl (if possible), for example https://github.com/xpack-dev-tools/gcc-xpack/.

Or to prevent cmake to autodiscover tools, and provide explicit paths to tools you want (this is a good advice in general).