Lack of Documentation and 400 Bad Request Error in BedrockRuntime ConverseRequest Tools #5568
Labels
documentation
This is a problem with documentation.
p2
This is a standard priority issue
service-api
This issue is due to a problem in a service API, not the SDK implementation.
Comments
KaisNeffati
added
documentation
KaisNeffati
changed the title
|
@KaisNeffati thank you for you feedback, I'll pass it to the Bedrock team. Just in cause you don't know, you can submit documentation feedback directly in the page, just click the "Feedback" button in the top right-side and fill the form with as much detail as you can, the feedback will go straight to the team that owns that particular page.
|
debora-ito
added
service-api
|
Hi Debora, thank you for your response. Below is the complete log : |
|
You didn't call the api with correct parameter in tool configuration. Here is an example of code that works: package com.hb;
import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
import software.amazon.awssdk.core.document.Document;
import software.amazon.awssdk.core.exception.SdkClientException;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClient;
import software.amazon.awssdk.services.bedrockruntime.model.ContentBlock;
import software.amazon.awssdk.services.bedrockruntime.model.ConversationRole;
import software.amazon.awssdk.services.bedrockruntime.model.ConverseResponse;
import software.amazon.awssdk.services.bedrockruntime.model.Message;
import software.amazon.awssdk.services.bedrockruntime.model.Tool;
import software.amazon.awssdk.services.bedrockruntime.model.ToolConfiguration;
import software.amazon.awssdk.services.bedrockruntime.model.ToolInputSchema;
import software.amazon.awssdk.services.bedrockruntime.model.ToolSpecification;
public class Main {
public static String converse() {
var client = createClient();
var modelId = "anthropic.claude-3-5-sonnet-20240620-v1:0";
var toolConfig = createToolConfig();
var message = createMessage("What is the temperature in Paris?");
try {
ConverseResponse response = client.converse(request -> request
.modelId(modelId)
.toolConfig(toolConfig)
.messages(message)
.inferenceConfig(config -> config
.maxTokens(512)
.temperature(0.5F)
.topP(0.9F))
);
System.out.println("Response:" + response);
var responseText = response.output().message().content().get(0).text();
System.out.println("Text Response:" + responseText);
return responseText;
} catch (SdkClientException e) {
System.err.printf("ERROR: Can't invoke '%s'. Reason: %s", modelId, e.getMessage());
throw new RuntimeException(e);
}
}
private static Message createMessage(String inputText) {
return Message.builder()
.content(ContentBlock.fromText(inputText))
.role(ConversationRole.USER)
.build();
}
private static ToolConfiguration createToolConfig() {
return ToolConfiguration.builder()
.tools(Tool.builder()
.toolSpec(ToolSpecification.builder()
.name("currentTemperature")
.description("Returns the current temperature of a city")
.inputSchema(ToolInputSchema.builder()
.json(createDocument())
.build())
.build())
.build())
.build();
}
private static BedrockRuntimeClient createClient() {
return BedrockRuntimeClient.builder()
.credentialsProvider(DefaultCredentialsProvider.create())
.region(Region.US_EAST_1)
.build();
}
private static Document createDocument() {
var cityParameter = Document.mapBuilder()
.putString("type", "string")
.putString("description", "City name")
.build();
var properties = Document.mapBuilder()
.putDocument("city", cityParameter)
.build();
var required = Document.listBuilder()
.addString("city")
.build();
return Document.mapBuilder()
.putString("type", "object")
.putDocument("properties", properties)
.putDocument("required", required)
.build();
}
public static void main(String[] args) {
converse();
}
}Output: |
|
Another example with your code logic: package com.hb;
import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
import software.amazon.awssdk.core.document.Document;
import software.amazon.awssdk.core.exception.SdkClientException;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClient;
import software.amazon.awssdk.services.bedrockruntime.model.ContentBlock;
import software.amazon.awssdk.services.bedrockruntime.model.ConversationRole;
import software.amazon.awssdk.services.bedrockruntime.model.ConverseRequest;
import software.amazon.awssdk.services.bedrockruntime.model.ConverseResponse;
import software.amazon.awssdk.services.bedrockruntime.model.Message;
import software.amazon.awssdk.services.bedrockruntime.model.StopReason;
import software.amazon.awssdk.services.bedrockruntime.model.Tool;
import software.amazon.awssdk.services.bedrockruntime.model.ToolConfiguration;
import software.amazon.awssdk.services.bedrockruntime.model.ToolInputSchema;
import software.amazon.awssdk.services.bedrockruntime.model.ToolResultBlock;
import software.amazon.awssdk.services.bedrockruntime.model.ToolResultContentBlock;
import software.amazon.awssdk.services.bedrockruntime.model.ToolResultStatus;
import software.amazon.awssdk.services.bedrockruntime.model.ToolSpecification;
import software.amazon.awssdk.services.bedrockruntime.model.ToolUseBlock;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;
import java.util.Optional;
import java.util.logging.Level;
import java.util.logging.Logger;
public class Main {
private static final Logger LOG = Logger.getLogger(Main.class.getName());
public static String converse() {
var client = createClient();
var modelId = "anthropic.claude-3-5-sonnet-20240620-v1:0";
var toolConfig = createToolConfig();
var messages = new ArrayList<>(List.of(createUserMessage("What is the temperature in Paris?")));
try {
var response = sendConverse(client, modelId, toolConfig, messages);
messages.add(response.output().message());
while (StopReason.TOOL_USE.equals(response.stopReason())) {
handleToolUse(response, messages);
response = sendConverse(client, modelId, toolConfig, messages);
}
LOG.info("messages=%s".formatted(messages));
LOG.info("response=%s".formatted(response));
var responseText = response.output().message().content().get(0).text();
LOG.info("textResponse=%s".formatted(responseText));
return responseText;
} catch (SdkClientException e) {
System.err.printf("ERROR: Can't invoke '%s'. Reason: %s", modelId, e.getMessage());
throw new RuntimeException(e);
}
}
private static Message createUserMessage(String inputText) {
return Message.builder()
.role(ConversationRole.USER)
.content(ContentBlock.fromText(inputText))
.build();
}
private static ToolConfiguration createToolConfig() {
return ToolConfiguration.builder()
.tools(Tool.builder()
.toolSpec(ToolSpecification.builder()
.name("currentTemperature")
.description("Returns the current temperature of a city")
.inputSchema(ToolInputSchema.builder()
.json(createToolSpecDocument())
.build())
.build())
.build())
.build();
}
private static BedrockRuntimeClient createClient() {
return BedrockRuntimeClient.builder()
.credentialsProvider(DefaultCredentialsProvider.create())
.region(Region.US_EAST_1)
.build();
}
private static Document createToolSpecDocument() {
var cityParameter = Document.mapBuilder()
.putString("type", "string")
.putString("description", "City name")
.build();
var properties = Document.mapBuilder()
.putDocument("city", cityParameter)
.build();
var required = Document.listBuilder()
.addString("city")
.build();
return Document.mapBuilder()
.putString("type", "object")
.putDocument("properties", properties)
.putDocument("required", required)
.build();
}
private static ConverseResponse sendConverse(BedrockRuntimeClient client, String modelId, ToolConfiguration toolConfig, List<Message> messages) {
var converse = ConverseRequest.builder()
.modelId(modelId)
.toolConfig(toolConfig)
.messages(messages)
.inferenceConfig(config -> config
.maxTokens(512)
.temperature(0.5F)
.topP(0.9F))
.build();
LOG.info("converseRequest=%s".formatted(converse));
return client.converse(converse);
}
private static void handleToolUse(ConverseResponse response, List<Message> messages) {
var toolRequests = response.output()
.message()
.content()
.stream()
.filter(contentBlock -> Objects.nonNull(contentBlock.toolUse()))
.toList();
for (var toolRequest : toolRequests) {
var tool = toolRequest.toolUse();
LOG.info("tool=%s id=%s".formatted(tool.name(), tool.toolUseId()));
ToolResultBlock toolResult = processToolRequest(tool);
messages.add(Message.builder()
.role(ConversationRole.USER)
.content(ContentBlock.builder().toolResult(toolResult).build())
.build());
}
}
private static ToolResultBlock processToolRequest(ToolUseBlock tool) {
if ("currentTemperature".equals(tool.name())) {
try {
var input = tool.input();
var inputMap = input.asMap();
var cityName = Optional.ofNullable(inputMap.get("city"))
.map(Document::asString)
.orElse("");
double result = currentTemperature(cityName);
return ToolResultBlock.builder()
.toolUseId(tool.toolUseId())
.content(ToolResultContentBlock.builder().json(createToolResultDocument(result)).build())
.status(ToolResultStatus.SUCCESS)
.build();
} catch (Exception ex) {
LOG.log(Level.SEVERE, "Error processing tool request: " + ex.getMessage(), ex);
return ToolResultBlock.builder()
.toolUseId(tool.toolUseId())
.content(ToolResultContentBlock.builder().text(ex.getMessage()).build())
.status(ToolResultStatus.ERROR)
.build();
}
}
return null;
}
private static double currentTemperature(String cityName) {
if ("Paris".equalsIgnoreCase(cityName)) {
return 25.0;
}
return 40.0;
}
private static Document createToolResultDocument(double temperature) {
return Document.mapBuilder()
.putString("type", "object")
.putDocument("content", Document.listBuilder()
.addNumber(temperature)
.build())
.build();
}
public static void main(String[] args) {
converse();
}
}Console output: |
|
Many thanks @herbert-beckman! My code is working perfectly now! I hope this solution helps others implement it until the official documentation becomes available to everyone |
|
I just checked the internal correspondence, and the Bedrock team agrees the documentation can be improved. I'll go ahead and close this issue, as there's no actions pending from the Java SDK team. |
|
This issue is now closed. Comments on closed issues are hard for our team to see. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the issue
I am encountering two issues when trying to utilize the Amazon BedrockRuntime ConverseRequest Tools API:
1. Lack of Documentation: There is insufficient documentation on how to properly set up and utilize the ConverseRequest for tool usage with Bedrock models. The official documentation does not provide enough clarity regarding request structure, model interaction, and tool configurations.
2. 400 Bad Request: When executing the code below, I consistently receive a 400 Bad Request error from the Bedrock service. This error is unclear, as the request structure appears to match the expected API, but further debugging is difficult due to the lack of clear examples in the documentation.
Here is the code that I used:
Steps to Reproduce:
Expected Result:
Actual Result:
Suggested Improvements:
Links
https://docs.aws.amazon.com/fr_fr/bedrock/latest/userguide/tool-use.html
The text was updated successfully, but these errors were encountered: