Bedrock Gateway

Each endpoint is a Bedrock runtime client with its own account, region, and credentials. BGW selects an endpoint per-request based on the team's routing strategy, endpoint priority, and health status. Adding multiple endpoints lets you pool quota across accounts or regions.

Application Inference Profiles (AIPs)

• AIPs are configured per-model on each endpoint. Use the AIP Overrides panel after creating an endpoint to map specific models to tagged AIP ARNs; models without an override use the system CRI for the endpoint's region scope.

Credentials

• Gateway's own account — Uses the gateway's IAM role or access keys. Works for same-account Bedrock and for cross-account access where the target account grants access via a resource policy on the inference profile.
• Assume an IAM role — Gateway calls STS AssumeRole before each Bedrock call. Use this when the Bedrock quota lives in a different account. Add an External ID to prevent confused-deputy attacks if the role trust policy includes it.

Required IAM permissions

• bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream — required for inference
• bedrock:ListInferenceProfiles — required for model discovery and health checks
• servicequotas:ListServiceQuotas — optional; needed for quota visibility in this portal

Prompt caching & routing strategy

Bedrock caches the model's internal state (KV cache) for repeated prompt prefixes. Cache hits cost 0.1× the standard input price (90% savings), while cache writes cost 1.25× (25% premium). The cache has a 5-minute sliding TTL — each hit resets the timer. An extended 1-hour TTL is available for Claude 4.5+ models.

• Per-account, per-model — cache is not shared across AWS accounts or models. Each endpoint's account has its own cache.
• Cross-region inference — caching works with CRIS, but requests landing in different regions may miss the cache.
• Exact prefix match — even a single token change in the prefix invalidates the cache. System prompts, tool definitions, and static context should come first.
• Minimum tokens — 1,024 tokens per checkpoint for Sonnet; 4,096 for Opus and Haiku. Below the minimum, the request succeeds but nothing is cached.

Why Sticky User is the default: Claude Code sessions have large, stable prompt prefixes (system prompt + tools + conversation history). Routing the same user to the same endpoint maximizes cache hits. Round Robin spreads users across endpoints, causing each endpoint to see different prefixes and miss the cache — this can significantly increase costs.

Amazon Bedrock does not support Anthropic's web_search server tool natively. When Claude Code asks to search the web, this gateway intercepts the request, executes the search using your configured provider, and returns results in the format Claude expects.

This is a well-known pattern used by API gateways like LiteLLM Proxy to enable web search on providers that don't support it natively. The search happens server-side — Claude Code sees the results as if they came from Anthropic's own search infrastructure.

What's different from Anthropic Direct API

• Search queries go to your chosen provider (not Anthropic's infrastructure)
• No intermediate "Searching..." progress — results arrive together after the search completes
• Citations referencing encrypted_content won't work (CC still shows titles + URLs)
• The web_search tool is transparently replaced with a regular tool definition for Bedrock

Privacy note: Search queries may contain context from your conversation. If you're working with sensitive code or proprietary information, consider which search provider you trust with those queries.

Configure a single notification destination for all gateway events (budget alerts, rate limits). Events are delivered asynchronously every 30 seconds. Three destination types are supported:

• Webhook — HTTPS endpoint receives JSON POST with event payload
• SNS Topic — Your own AWS SNS topic. Add a resource-based policy granting the gateway task role sns:Publish
• EventBridge — Your own custom event bus. Add a resource-based policy granting events:PutEvents

BYO resources: Create your own SNS topic or EventBridge bus, then add a resource-based policy allowing the BGW task role to publish. The Task Role ARN is available as a CDK output (TaskRoleArn).

Workflow: Configure a destination → Save as draft → Test delivery → Activate. You can change or deactivate the destination at any time without a redeploy.