Exercise 02 - Use SAP AI Launchpad to Deploy a Large Language Model with Generative AI Hub on SAP AI Core
SAP AI Launchpad is a multi-tenant SaaS application on SAP BTP. You can use SAP AI Launchpad to manage AI use cases across different AI runtimes. SAP AI Launchpad also provides generative AI capabilities via the Generative AI Hub and is available in the Cloud Foundry environment. You can also connect HANA as an AI runtime or use SAP AI Services to work with the HANA Predictive Analysis Library (PAL) or the SAP AI Service Data Attribute Recommendation.
👉 Go to your BTP subaccount CAP AI CodeJam.
👉 Navigate to Instances and Subscriptions and select SAP AI Launchpad from Subscriptions to open SAP AI Launchpad.
SAP AI Core tenants use resource groups to isolate AI resources and workloads. Scenarios (e.g., foundation-models) and executables (a template for training a model or creating a deployment) are shared across all resource groups.
Make sure to create a NEW resource group. DO NOT USE THE DEFAULT RESOURCE GROUP!
👉 Open the SAP AI Core Administration tab and select Resource Groups.
👉 Create a new resource group with your initials.
👉 Go back to Workspaces.
The creation of the Resource Group will take a minute to be created. Refresh the Workspace page until your Resource Group shows up.
👉 Select your connection and your resource group.
👉 Ensure it is selected. It should appear at the top next to SAP AI Launchpad.
You will need the name of your resource group in Exercise 09-create-connection-configuration.
With Generative AI Hub on SAP AI Core, you have access to all major large language models (LLMs). There are open-source models that SAP has deployed, such as the Falcon model, and models that SAP is a proxy for, like the GPT models, Google models, Amazon Bedrock models, and more. To use one of the provided LLMs for a custom use case, you need to create a deployment configuration for the model. Using this configuration, you can then deploy the model. You will receive a deployment URL that you can use to query the model of your choice.
👉 Open the ML Operations tab.
👉 Go to Scenarios.
👉 Select the foundation-models scenario.
Scenarios related to generative AI are the only pre-configured scenarios provided by SAP. For all other custom machine learning models you wish to train or deploy, you will need to create your own scenario.
👉 Select the Executables tab.
👉 Select the serving executable azure-openai to view the available Azure OpenAI models.
👉 Copy the name of the model you want to deploy a proxy for.
For this CodeJam, you will use gpt-4o-mini. After that, you will create a configuration.
👉 Click on Configurations.
👉 Create a new configuration.
👉 Enter a configuration name, e.g., conf-gpt-4o-mini, select the foundation-models scenario, version, and the executable azure-openai.
👉 Click Next.
👉 Paste the model name gpt-4o-mini into the modelName field and click Next.
👉 Click Review at the bottom of the page.
👉 Review the configuration and click Create.
👉 Click on Create Deployment to create a deployment for that configuration.
This will not actually deploy the model but will deploy a proxy that will return a URL for you to use to query the LLM you specified in the configuration.
👉 For the duration, select Standard.
You can also select Custom to have the deployment available for a limited time.
👉 Click Review.
👉 Click Create.
The deployment status will change from UNKNOWN to PENDING, then to RUNNING. Once the deployment is running, you will receive a URL to query the model. Wait a couple of minutes, then refresh the page for the URL to appear.
Using the URL, client id, and client secret from the SAP AI Core service key, you can now query the model using any programming language or API platform.
👉 To implement a retrieval-augmented generation (RAG) use case, we also need to deploy an embedding model. The embeddings for our text chunks will then be stored in a vector database (e.g., SAP HANA Cloud Vector Engine).
To deploy the embedding model, repeat the steps above using the model name text-embedding-ada-002 instead of gpt-4o-mini:
At this point, you will have learned how to create your own resource group, which models are available via the Generative AI Hub, and how to deploy LLMs in SAP AI Launchpad.
- What is SAP AI Launchpad, and what can it be used for?
Answer
SAP AI Launchpad is a multi-tenant SaaS application on SAP Business Technology Platform. It helps you manage AI use cases across different AI runtimes and provides generative AI capabilities via the SAP Generative AI Hub.- What is the purpose of creating a new resource group in SAP AI Launchpad, and how should it be done?
Answer
Creating a new resource group in SAP AI Launchpad helps isolate AI resources and workloads for better management.- How do you deploy a proxy for a large language model (LLM) on SAP AI Core using SAP AI Launchpad?
Answer
To deploy a proxy for an LLM, you need to:- Open the ML Operations tab in SAP AI Launchpad.
- Select the foundation-models scenario, and then choose the model executable (e.g., azure-openai).
- Create a new configuration, specifying the model name (e.g., gpt-4o-mini).
- After reviewing the configuration, click Create.
- Then, click Create Deployment to deploy a proxy. Once the deployment status changes to RUNNING, you will receive a URL to query the model.
- SAP AI Launchpad - Help Portal (Documentation)
- SAP AI Core Terminology
- Available Models in the Generative AI Hub