Skip to content

[Inference API] Add Chat Completion to Amazon Bedrock for the Inference API - #139411

Merged
jonathan-buttner merged 81 commits into
elastic:mainfrom
jonathan-buttner:ia-bedrock-completions
Jan 22, 2026
Merged

jonathan-buttner merged 81 commits into
elastic:mainfrom
jonathan-buttner:ia-bedrock-completions

Conversation

@jonathan-buttner

@jonathan-buttner jonathan-buttner commented Dec 11, 2025 •

Copy link
Copy Markdown
Contributor

This PR implements chat completion for amazon bedrock. It's based on this PR: #133697

Testing

Create the endpoint

PUT _inference/chat_completion/test-chat?timeout=30s
{
    "service": "amazonbedrock",
    "service_settings": {
        "provider": "anthropic",
        "model": "anthropic.claude-3-5-sonnet-20240620-v1:0",
        "region": "us-east-1",
        "access_key": "<access key>",
        "secret_key": "<secret>"
    }
}
Complex request
POST _inference/chat_completion/test-chat/_stream
{
    "messages": [
        {
            "role": "user",
            "content": "test"
        },
        {
            "role": "assistant",
            "content": "tool call",
            "tool_calls": [
                {
                    "function": {
                        "name": "context",
                        "arguments": "{}"
                    },
                    "id": "803434",
                    "type": "function"
                }
            ]
        },
        {
            "role": "tool",
            "content": "{\"screen_description\":\"The user is looking at http://localhost:5601/app/observability/overview?rangeFrom=now-15m&rangeTo=now. The current time range is 2024-12-13T01:18:25.752Z - 2024-12-13T01:33:25.752Z.\\n\\nThe user is viewing the Overview page which shows a summary of the following apps: {\\\"universal_profiling\\\":{\\\"hasData\\\":false,\\\"status\\\":\\\"success\\\"},\\\"alert\\\":{\\\"hasData\\\":false,\\\"status\\\":\\\"success\\\"},\\\"uptime\\\":{\\\"hasData\\\":false,\\\"indices\\\":\\\"heartbeat-*\\\",\\\"status\\\":\\\"success\\\"},\\\"infra_metrics\\\":{\\\"hasData\\\":false,\\\"indices\\\":\\\"metrics-*,metricbeat-*\\\",\\\"status\\\":\\\"success\\\"},\\\"ux\\\":{\\\"hasData\\\":false,\\\"indices\\\":\\\"traces-apm*,apm-*,traces-*.otel-*,logs-apm*,apm-*,logs-*.otel-*,metrics-apm*,apm-*,metrics-*.otel-*\\\",\\\"status\\\":\\\"success\\\"},\\\"infra_logs\\\":{\\\"hasData\\\":false,\\\"indices\\\":\\\"logs-*-*,logs-*,filebeat-*\\\",\\\"status\\\":\\\"success\\\"},\\\"apm\\\":{\\\"hasData\\\":false,\\\"indices\\\":{\\\"transaction\\\":\\\"traces-apm*,apm-*,traces-*.otel-*\\\",\\\"span\\\":\\\"traces-apm*,apm-*,traces-*.otel-*\\\",\\\"error\\\":\\\"logs-apm*,apm-*,logs-*.otel-*\\\",\\\"metric\\\":\\\"metrics-apm*,apm-*,metrics-*.otel-*\\\",\\\"onboarding\\\":\\\"apm-*\\\",\\\"sourcemap\\\":\\\"apm-*\\\"},\\\"status\\\":\\\"success\\\"}}\",\"learnings\":[]}",
            "tool_call_id": "803434"
        }
    ],
    "tool_choice": "auto",
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "query",
                "description": "This function generates, executes and/or visualizes a query\n      based on the user's request. It also explains how ES|QL works and how to\n      convert queries from one language to another. Make sure you call one of\n      the get_dataset functions first if you need index or field names. This\n      function takes no input.",
                "parameters": {
                    "type": "object",
                    "properties": {}
                }
            }
        },
        {
            "type": "function",
            "function": {
                "name": "get_alerts_dataset_info",
                "description": "Use this function to get information about alerts data.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "start": {
                            "type": "string",
                            "description": "The start of the current time range, in datemath, like now-24h or an ISO timestamp"
                        },
                        "end": {
                            "type": "string",
                            "description": "The end of the current time range, in datemath, like now-24h or an ISO timestamp"
                        }
                    }
                }
            }
        },
        
        {
            "type": "function",
            "function": {
                "name": "execute_connector",
                "description": "Use this function when user explicitly asks to call a kibana connector.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "id": {
                            "type": "string",
                            "description": "The id of the connector"
                        },
                        "params": {
                            "type": "object",
                            "description": "The connector parameters"
                        }
                    },
                    "required": [
                        "id",
                        "params"
                    ]
                }
            }
        }
    ]
}

Evgenii-Kazannik and others added 30 commits October 17, 2025 13:40
…Completions-support' into Add-Amazon-Bedrock-Unified-Chat-Completions-support

# Conflicts:
#	x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/amazonbedrock/request/completion/ToolAwareUnifiedPublisher.java
…Completions-support' into Add-Amazon-Bedrock-Unified-Chat-Completions-support
@jonathan-buttner jonathan-buttner added the Feature:GenAI Features around GenAI label Dec 16, 2025

@DonalEvans DonalEvans left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of small things, other than that looks good.

}
default -> Document.mapBuilder().build();
default -> throw new IllegalArgumentException(
Strings.format("Unsupported type: %s while convert to Bedrock document representation", value.getClass().getSimpleName())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick, "while converting to..."

Comment on lines +113 to +116
assertEquals(0.7, updatedSettings.temperature(), 0.001);
assertEquals(0.8, updatedSettings.topP(), 0.001);
assertEquals(0.9, updatedSettings.topK(), 0.001);
assertEquals(256, updatedSettings.maxNewTokens(), 0.001);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need the deltas in the assertions here and elsewhere in the file? Why would we expect the value in the AmazonBedrockCompletionTaskSettings to be different from the value in the map used to create them? Also, a delta that's less than 1 for an integer value doesn't really make sense, so the assertion on the value returned by maxNewTokens() should definitely not use a delta, even if there's a valid reason for the other assertions to use one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants