Argument parsing is broken on Windows

[sijk]

I suspect that this is because of the patches xpack applies to cmake to run processes inside cmd, which alters the way arguments are parsed. Whatever the reason, cmake-xpack is broken on Windows and there is no workaround.

I have provided two examples of the problem in the following repo: https://github.com/sijk/cmake-xpack-broken-example

[ilg-ul]

Thank you for your report.

I'll investigate, but it might take a while.

[ilg-ul]

Running your tests was quite challenging for me, since they require Visual Studio for nmake and the compiler.

I updated the patch to use the cmd.exe sub-shell only for non-exe processes, and now the result is:

C:\Users\ilg\tmp\cmake-xpack-broken-example\example1>C:\Users\ilg\OneDrive\Desktop\xpack-cmake-3.27.9-1-win32-x64\xpack-cmake-3.27.9-1\bin\cmake.exe --build .
MSBuild version 17.8.5+b5265ef37 for .NET Framework

  1>Checking Build System
  Creating directories for 'cmake-xpack'
  Building Custom Rule C:/Users/ilg/tmp/cmake-xpack-broken-example/example1/CMakeLists.txt
  Performing download step (git clone) for 'cmake-xpack'
  Cloning into 'cmake-xpack'...
  HEAD is now at 81fef80 templates/jekyll update
  Performing update step for 'cmake-xpack'
  No patch step for 'cmake-xpack'
  No configure step for 'cmake-xpack'
  No build step for 'cmake-xpack'
  No install step for 'cmake-xpack'
  Completed 'cmake-xpack'
  Building Custom Rule C:/Users/ilg/tmp/cmake-xpack-broken-example/example1/CMakeLists.txt

and

C:\Users\ilg\tmp\cmake-xpack-broken-example\example2>C:\Users\ilg\OneDrive\Desktop\xpack-cmake-3.27.9-1-win32-x64\xpack-cmake-3.27.9-1\bin\cmake.exe . -DPython_EXECUTABLE=C:/Users/ilg/AppData/Local/Microsoft/WindowsApps/python.exe
-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.22621.0 to target Windows 6.2.9200.
-- The C compiler identification is MSVC 19.38.33135.0
-- The CXX compiler identification is MSVC 19.38.33135.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Python: C:/Users/ilg/AppData/Local/Microsoft/WindowsApps/python.exe (found version "3.12.2") found components: Interpreter
-- Configuring done (5.9s)
-- Generating done (0.0s)
-- Build files have been written to: C:/Users/ilg/tmp/cmake-xpack-broken-example/example2

The pre-release binary is available from:

• https://github.com/xpack-dev-tools/pre-releases/releases/tag/experimental

Please try it for your projects, and let me know if it if functional.

Please note that the fix brings back the same functionality as the original cmake for running .exe processes, but running .cmd scripts is not guaranteed to work, since not all command lines can be executed via a sub-shell, for example the python code used by cmake fails, because it invokes multi-line scripts, which apparently are not supported by cmd.exe.

[sijk]

Thanks for looking into it. I've tested the experimental release linked above.

• example1 works
• example2 works if you give it the path to the system python but not if you give it the path to a venv:

  cd example2
  python -m venv venv
  ...\xpack-cmake-3.27.9-1\bin\cmake.exe . -DPython_EXECUTABLE=%CD%\venv\Scripts\python.exe
  

I have no idea why that is and I don't have any more time to spend on debugging it, sorry -- I've moved to using a devcontainer instead of xpacks for a standard build environment. I'm curious -- why do you need to patch the cmake source at all? Upstream cmake works fine on both of these examples.

[ilg-ul]

why do you need to patch the cmake source at all? Upstream cmake works fine on both of these examples.

Upstream cmake is able to execute only .exe files; it is not able to start programs via .cmd, .bat, etc, which require a sub-shell. On Windows, since there are no symbolic links, the npm/xpm ecosystem uses .cmd shims to start programs. I did not invent this, npm did.

[ilg-ul]

... not if you give it the path to a venv:

I cannot reproduce this:

C:\Users\ilg\tmp\cmake-xpack-broken-example\example2>python -m venv venv

C:\Users\ilg\tmp\cmake-xpack-broken-example\example2>C:\Users\ilg\OneDrive\Desktop\xpack-cmake-3.27.9-1-win32-x64\xpack-cmake-3.27.9-1\bin\cmake.exe . -DPython_EXECUTABLE=%CD%\venv\Scripts\python.exe
-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.22621.0 to target Windows 6.2.9200.
-- The C compiler identification is MSVC 19.38.33135.0
-- The CXX compiler identification is MSVC 19.38.33135.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Python: C:\Users\ilg\tmp\cmake-xpack-broken-example\example2\venv\Scripts\python.exe (found version "3.12.2") found components: Interpreter
-- Configuring done (7.0s)
-- Generating done (0.0s)
-- Build files have been written to: C:/Users/ilg/tmp/cmake-xpack-broken-example/example2

C:\Users\ilg\tmp\cmake-xpack-broken-example\example2>

[ilg-ul]

Starting with September 2024, v3.28, I removed the patch to start processes via a sub-shell.

This should solve this issue. Can you confirm this?