Able to use kubernetes Job as evaluation runner

[bjwswang]

Our evaluation flow will have 3 jobs:

Job1: Download dataset files from our system datasource

• download dataset files for evaluation @0xff-dev

Job2: Generate ragas test dataset based on the Stage1 files

• arctl eval gen_test_dataset feat: generate evaluation test dataset #529 @bjwswang

Job3: Evaluate

• with python ragas feat: add ragas_once pip package #513 @Lanture1064
• JudgeLLM uses OpenAI ChatGPT

At the end of this evaluation,we should keep

• ragas evaluation scores

Job4: Upload metrics

• upload metrics to system datasource @Lanture1064

The overall workflow will be

To implement this evaluation runner,we also need to :

• implement evaluations.rag controller #566
• matain a evaluatoin runner image

[nkwangleiGIT]

the chart of evaluation flow can be added to doc center.

[bjwswang]

@0xff-dev @Lanture1064