CitePrism uses LLMs and human review to audit citation integrity

CitePrism, introduced by researchers Gowrika Mahesh and colleagues, is a transparent hybrid decision-support framework for editorial citation auditing. It integrates LLM-assisted contextual reasoning, embedding-based semantic similarity, metadata verification, and integrity-oriented flags (e.g., self-citation patterns) with human-in-the-loop analyst review. The system extracts citation neighborhoods, enriches reference metadata, computes fused relevance scores, and supports configurable threshold-based triage. In a preliminary validation on a single case-study manuscript with 104 references from pavement engineering, CitePrism achieved Cohen's kappa = 0.429 agreement with human binary relevance labels. At operating threshold tau = 17, it flagged all human-labeled irrelevant citations, though it also generated false positives requiring analyst review. The authors emphasize this is pilot-stage decision support, not an autonomous misconduct detector, and broader validation across manuscripts, domains, and settings is required before operational use.

The potential impact of CitePrism lies in scaling the labor-intensive process of citation auditing, which remains largely manual and fragmented in academic publishing. By surfacing metadata issues, self-citation concerns, and relevance mismatches, the system enables editors and reviewers to quickly triage problematic references. The use of a human-in-the-loop ensures that false positives are caught, maintaining editorial rigor. However, the study's single-manuscript design and moderate kappa score highlight the need for more extensive testing. If validated, CitePrism could become a valuable tool for maintaining bibliographic integrity in scholarly communication, reducing the spread of inaccurate or irrelevant citations that undermine research credibility. The framework is open for further development, with code, data, and interface screenshots included in the paper's appendices.