ROS2 lacks auditable decision provenance as EU AI Act deadlines loom
Post reveals four critical gaps in tracing why robots make decisions under regulations.
A ROS2 developer, altunbulakemre75, has raised a pressing issue in the ROS discourse: the absence of a systematic way to trace the reasoning behind autonomous robot decisions. While rosbag captures topic data (the ‘what’), reconstructing the ‘why’ for safety officers, insurers, or regulators currently involves manual after-the-fact analysis. The developer identifies four specific gaps: rule provenance is invisible (no link from BehaviorTree node to the specific safety policy that authorized it); guardrails act as one-way safety stops without producing a signed record of the downgrade (e.g., ‘planner wanted X, guardrail limited to Y’); LLM-in-the-loop decision chains lack standard hooks to log whether the model’s suggestion was followed, overridden, or sanitized; and the EU AI Act’s Articles 12 and 14 now demand traceability and human oversight, yet most teams fall back on rosbag plus a spreadsheet—an approach that won’t withstand a regulator audit as CE marking deadlines approach in 2027.
Respondent gbiggs argues that requirement traceability tools already exist for point #1, and that #2 reflects common safety standards in factory settings. However, he agrees the overall debugging process—labor-intensive for aerospace but ill-suited to broader autonomous systems—needs better tooling. The thread also references related working groups (ROS2 Security WG, Safety-Critical WG) and prior discussions on reproducible safety and observability. The original poster is considering open-sourcing an opinionated implementation covering rule provenance, signed audit chains, guardrail-downgrade patterns, and LLM sanitization, and is seeking feedback on missed prior work or real-world deployment pain points.
- Rule provenance missing: No first-class link from a robot action (e.g., 'stopped') to the specific safety policy rule that triggered it.
- Guardrails lack audit trail: Kill switches and velocity caps prevent harm but produce no signed record of planner intent vs. actual downgraded output.
- EU AI Act deadlines: Articles on logging and human oversight in force now; CE marking for high-risk systems due by 2027—current rosbag+spreadsheet approaches insufficient.
Why It Matters
Without standardized decision auditing, autonomous systems risk regulatory noncompliance, reduced public trust, and costly post-incident investigations.