Amazon Bedrock AgentCore Browser automates web insight extraction at scale

Amazon has unveiled a new solution built on Amazon Bedrock AgentCore that automates the extraction of insights from dozens of websites, addressing a pain point for design teams tracking competitors, marketing teams monitoring trends, and product managers gathering market intelligence. The system replaces fragile rule-based scrapers, which often break when sites are redesigned or migrate to JavaScript-rendered frontends. At the core is the AgentCore Browser, a fully managed browser service that reliably renders JavaScript-heavy pages, ensuring the pipeline stays resilient as websites change.

The architecture uses an event-driven pattern that separates content collection from AI processing. An Amazon EventBridge schedule triggers a Lambda function every 15 minutes to check RSS feeds and deduplicate articles against Amazon S3. For each new article, the function opens a browser session via AgentCore and connects with Playwright over the Chrome DevTools Protocol (CDP), capturing full page renders, screenshots, and images. S3 upload events then trigger an SQS queue, and a second Lambda extracts clean text from raw HTML. Amazon Bedrock generates summaries, identifies themes and entities, and creates vector embeddings, which are indexed into Amazon OpenSearch Serverless for keyword and vector search. The complete implementation is open-sourced on GitHub, and the architecture extends to use cases like competitor tracking, content curation, and compliance monitoring, making it a powerful tool for teams that need to stay ahead of industry shifts.