KEP-2401: Kubeflow LLM Trainer V2

[Electronic-Waste]

This is the tracking issue for the Kubeflow LLM Trainer V2, a submodule of Kubeflow Training V2: #2170

We aim to solve:

• KEP-2170: Design Trainer for the LLM Runtimes #2321
• KEP-2170: Create LLM training runtime for Llama 3.1 8B #2212

However, the LLM Trainer V2 design is very complex and needs further discussion. So we decided to open a separate issue tracking it.

Initial Design (Google Doc): Kubeflow Training V2 LLM Trainer Design

/area runtime
/cc @kubeflow/wg-training-leads @deepanker13 @saileshd1402 @seanlaii @helenxie-bit @astefanutti @varshaprasad96 @franciscojavierarceo @thesuperzapper @rimolive @juliusvonkohout @jbottum @varodrig @Doris-xm @truc0

[Electronic-Waste]

/kind feature