New neuro-symbolic system eliminates LLM requirement errors with 94.6% accuracy
LLMs generate flawed requirements? This architecture fixes them with a three-valued uncertainty scorer.
Large Language Models (LLMs) offer natural-language flexibility for automated requirements elicitation but frequently produce structurally invalid or logically inconsistent outputs. Ahmed Ibrahim's latest paper presents a novel neuro-symbolic multi-agent architecture that addresses this by operationalizing the Object-Oriented Method for Requirements Authoring and Management (OOMRAM) lattice. In this system, the LLM acts as a non-deterministic heuristic for lattice traversal, while a deterministic symbolic validator enforces all structural constraints. Introduced is a three-valued uncertainty framework—Truth (T), Indeterminacy (I), Falsity (F)—to classify and score the LLM's requirement decisions both before and after validation, providing a formal measure of neural uncertainty.
Evaluated across 37 natural-language project visions in eleven application families, the system completely eliminated structural inconsistencies in 35 out of 37 cases (94.6%). The remaining two cases contained only 6 unresolved structural errors, representing 0.39% of all decisions, attributed solely to iteration limits. Three-valued analysis revealed that 24.7% of all decisions are indeterminate—structurally valid but discretionary choices not explicitly mandated by stakeholders. The study concludes that offloading structural integrity to a deterministic symbolic layer successfully guarantees conformance, while the uncertainty scoring facilitates safe LLM deployment in formal requirements engineering, a critical step for AI-assisted software development.
- Eliminated structural inconsistencies in 35/37 cases (94.6%), with only 0.39% errors due to iteration limits
- Introduced three-valued (Truth, Indeterminacy, Falsity) uncertainty scoring for LLM generated requirements
- 24.7% of all decisions were indeterminate but structurally valid, highlighting areas for stakeholder clarification
Why It Matters
Enables safe deployment of LLMs in formal requirements engineering by guaranteeing structural conformance and quantifying neural uncertainty.