AI audit of 70,075 EPA comments reveals who shapes regulations

A new research paper from Jianing Fan and Yue Yao introduces an AI-assisted framework called obligation-level responsiveness auditing, designed to measure whether public comments actually influence specific regulatory duties in U.S. agency rulemaking. The approach goes beyond prior rule-level or aggregate-corpus analyses by extracting proposed and final-rule obligations, matching comments to the specific duties they address, and classifying whether outcomes reflect substantive change or mere editorial refinement. The researchers applied this framework to 70,075 comments across 36 EPA anchor rulemakings, drawn from a larger corpus of 786,197 comments across 6,145 dockets spanning 2010-2022. Each load-bearing component of the framework was validated against blind human judgment, ensuring reliability.

Three descriptive findings emerged. First, public engagement is associated with revision, but only at a modest within-docket magnitude. Second, whether commenters support or oppose a rule does not clearly differentiate outcomes—an informative null result that contradicts simple preference-aggregation theories. Third, under a permissive reconstruction of commenter type, organizational-majority engagement concentrates in editorial-refinement outcomes rather than substantive-modification outcomes at the cross-docket level. A blind human audit preserved this third finding under corrected labels, and the researchers found that text-similarity methods alone are insufficient for distinguishing editorial from substantive regulatory change—a measurement-validity lesson they treat as a supporting methodological contribution. The paper, accepted as a full oral presentation at ACM EAAMO '26 in Munich, locates the equity asymmetry upstream of agency response: in differential capacity across commenter populations to identify, interpret, and contest specific legal obligations.