Pipeline compilation performance in MoltenVK

[riperiperi]

Hey,

I'm a developer on the Ryujinx project (nintendo switch emulator) and worked on the recent port to MacOS which used MoltenVK, though it's still being upstreamed. I've got a few questions surrounding pipeline compilation, in case there's some way I could improve the performance.

As an emulator, we pretty much have to compile shaders constantly at runtime. We use a dispatch queue for draws with the special ability to collect multiple shader programs ahead of where the backend is currently drawing for multithreaded compilation, and we also have a disk shader cache.

The Ryujinx shader cache works by storing a single pipeline state for each guest shader (and optionally cached spirv), and batch creating pipelines from that on launch to preload them all into memory, keeping the shader modules in memory. This is done to optimize the possible creation of similar pipelines using the same shader collection in future, as on other drivers this costs a fraction of a millisecond (0.01ms). Games do this a lot at runtime as there's a lot of pipeline state we don't upload as dynamic, or can't.

I have a few questions:

• What is the main cost when compiling a pipeline "variant", where one or two things are different from a previous compile? I noticed that it is rather consistent - I was getting around 15ms for compiling a new pipeline that only changed the blend constant colour, but this was easily eliminated by adding it to dynamic state. I only noticed this on MoltenVK, as a game was changing the colour every frame, and the cost was eliminated by including this in dynamic state. Games often do this for reasons we can't remove, though, such as vertex buffer/attribute changes, attachment changes, blend state changes etc. Can it be avoided, or should we just add more dynamic state as it becomes available?
  • I imagine saving the pipeline cache to disk may get around us paying for variants after shader cache load, but this won't affect first run performance. 😓
• Can pipelines be compiled in parallel in MoltenVK (spirv translation/metal shader compile)? I noticed that our batch compilation step before launch has a rather low utilization compared to what it does on PC graphics drivers, and gets through them rather slowly. On Windows we can find games that compile our maximum number of 8 shaders concurrently at runtime, so it is also important for first run performance. We're compiling shader modules and pipelines from background threads.

Regardless, I was majorly impressed with how well MoltenVK handled our Vulkan backend, after the portability subset limitations were dealt with. Thanks for working on this project!

[billhollings]

Currently, pipeline caching in MoltenVK is limited to archiving/reloading the shader MSL source, after conversion from SPIR-V. It does not cache either the compiled MSL or the Metal pipeline state in binary form. When pipeline caching was introduced to MoltenVK, this was then only option, as Metal did not yet support binary pipeline archives, or shader caching.

Metal has since moved on, and now provides both, at some level. Issue #1765 identifies the requirement to incorporate the newer Metal capabilities, and is where we are tracking this development.

Can pipelines be compiled in parallel in MoltenVK (spirv translation/metal shader compile)?

Yes. Shader compilation in MoltenVK supports this.

[riperiperi]

Thanks for the answer, good to see that the pipeline cache will be improved into the future. I'll need to investigate further into why our shader compilation doesn't seem to be going as parallel as it could, I figured something was blocking compiles from continuing.