Bo Chen's constrained agent framework makes web scraping failure-safe with zero LLM tokens

Researchers have proposed a constrained, verifiable agent framework that makes LLM-based web scraping failure-safe by replacing free-form code generation with typed JSON collector configurations. The framework, introduced by Bo Chen, uses a six-type collector taxonomy, template and utility-function constraints, static Airflow DAG execution, rule-based quality checking, and structured feedback correction. Instead of relying on LLMs to generate unreliable web scrapers for each task, the framework produces deterministic configurations that are verified before execution.

Experiments on 138 tasks showed the taxonomy supports description-based requirement typing, but confirmed that stable instantiation requires completing source, field, and execution constraints beyond initial descriptions. On 80 independently source-verified tasks, the framework ran with zero execution-stage LLM tokens and achieved the lowest average wall-clock time. While initial one-shot quality is moderate, the reusable, deterministic, and verifiable execution path makes it ideal for repeated scheduled data collection, trading flexibility for reliability and cost savings.