# A Multi-Dimensional Quality Scoring Framework for Decentralized LLM Inference with Proof of Quality Source: [A Multi-Dimensional Quality Scoring Framework for Decentralized LLM Inference with Proof of Quality]() Collection index: [llms.txt]() Data updated: 2026-03-06T05:00:05.534Z Authors: Arther Tian, Alex Ding, Frank Chen, Simon Wu, Aaron Chan Published: 2026-03-04T13:05:46.000Z ## Abstract Decentralized large language model (LLM) inference networks can pool heterogeneous compute to scale serving, but they require lightweight and incentive-compatible mechanisms to assess output quality. Prior work introduced cost-aware Proof of Quality (PoQ) and adaptive robust PoQ to allocate rewards under evaluator heterogeneity and adversarial behavior. In this paper, we focus on the quality signal itself and propose a multi-dimensional quality scoring framework that decomposes output quality into modular dimensions, including model and cost priors, structure quality, semantic quality, query-output alignment, and agreement/uncertainty. Using logged outputs from QA and summarization tasks, we systematically audit dimension reliability and show that seemingly reasonable dimensions can be task-dependent and even negatively correlated with reference quality without calibration. While the default composite underperforms a strong single semantic evaluator, ablations reveal that removing unreliable dimensions and re-normalizing weights yields a calibrated composite that matches or exceeds the best single- evaluator and consensus baselines. Finally, we integrate the composite score as a drop-in quality signal in PoQ and demonstrate complementary benefits with robust aggregation and adaptive trust weighting under adversarial evaluator attacks. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Response Quality Evaluation on Held-out n=300 (test)](): Response Quality Evaluation; Held-out n=300 (test); Pearson Correlation Coefficient: 0.647. - [Correlation analysis with ground truth on Evaluation Dataset 2000 samples](): Correlation analysis with ground truth; Evaluation Dataset 2000 samples; Pearson Correlation Coefficient: 0.733. ## Original sources and implementation [Original paper]() [PDF]()