PoQ-Judge framework slashes LLM evaluation costs 72% without reference answers

Decentralized LLM inference networks face a critical challenge: how to verify output quality without centralized oversight or expensive reference answers. PoQ-Judge, introduced by a team of researchers (Tian et al.), addresses this by training dedicated 'judge' models that score query-response pairs directly. The framework explores three architectures—a lightweight TextCNN, a MiniLM cross-encoder, and a more powerful DeBERTa—across the quality-cost tradeoff. Using a two-stage training pipeline (UltraFeedback plus GPT-labeled in-domain data), the DeBERTa-based judge achieves a Pearson correlation of 0.747 against a ground-truth proxy on held-out data, surpassing earlier reference-dependent methods. As a purely reference-free component in composite scoring, it reaches 0.645 Pearson, matching the best single reference-based evaluator while eliminating the need for sample answers.

A key innovation is cascade evaluation: routing easy queries to the cheap TextCNN judge and only escalating harder cases to DeBERTa. This reduces inference cost by 72.7% while retaining near-peak quality. Online calibration reveals that semantic quality is the dominant evaluation dimension. However, results are significantly stronger on question-answering tasks than on summarization, pointing to proxy quality as the main remaining limitation. For decentralized networks like Bittensor or Gensyn, PoQ-Judge offers a practical, cost-aware proof-of-quality mechanism that could enable trustless, scalable LLM inference without centralized validators.