Couchbase builds multi-model AI assistant with Amazon Bedrock and Claude
Over 99.99% uptime with cross-region inference and zero-code model swaps.
Couchbase, in collaboration with AWS, built a multi-model AI architecture for its Capella iQ developer assistant, powered by Amazon Bedrock and Anthropic's Claude family of LLMs. The system is designed to handle diverse inference workloads—from generating SQL++ queries and recommending indexes to supporting multi-turn conversational flows—without relying on a single model. The production architecture spans two AWS Regions (us-east-1 and us-west-2) for high availability, using Amazon EKS to host microservices (cp-api, cp-internal-api, cp-ns pods). A VPC interface endpoint provides private connectivity to Amazon Bedrock, ensuring data never traverses the public internet.
Amazon Bedrock's Cross-Region Inference (CRIS) automatically routes requests to the optimal US region (us-east-1, us-east-2, or us-west-2) during demand spikes or regional degradation, maintaining over 99.99% uptime with zero application-level logic. The team established a benchmark suite covering core Capella iQ tasks—code generation, reasoning, and conversational coherence—to select Claude as the primary model family. A critical design choice is the model-agnostic inference layer: model upgrades or provider changes require only configuration updates at the namespace layer, no code changes, no downtime, and no impact to developers. The result is a flexible, scalable, and resilient enterprise AI assistant that meets diverse customer deployment preferences.
- Couchbase adopted Amazon Bedrock with Anthropic's Claude models for Capella iQ, spanning two AWS Regions (us-east-1, us-west-2) with automatic cross-region failover.
- The architecture uses Amazon EKS microservices and a VPC interface endpoint to keep inference traffic private, meeting enterprise security and data residency requirements.
- Model upgrades require only configuration updates at the namespace layer—no code changes or downtime—enabling frictionless model swapping without impacting the developer experience.
Why It Matters
Couchbase's architecture demonstrates how enterprises can build resilient, model-agnostic AI assistants with Amazon Bedrock, scaling inference across regions without code changes.