# Spectral Geometry for Deep Learning: Compression and Hallucination Detection via Random Matrix Theory

Source: [Spectral Geometry for Deep Learning: Compression and Hallucination Detection via Random Matrix Theory](<https://www.sota2.com/research/paper/69784663a2b28e761d87c34e>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-28T05:00:08.385Z

Authors: Davide Ettori

Published: 2026-01-24T08:07:22.000Z

## Abstract

Large language models and deep neural networks achieve strong performance but suffer from reliability issues and high computational cost. This thesis proposes a unified framework based on spectral geometry and random matrix theory to address both problems by analyzing the eigenvalue structure of hidden activations. The first contribution, EigenTrack, is a real-time method for detecting hallucinations and out-of-distribution behavior in language and vision-language models using spectral features and their temporal dynamics. The second contribution, RMT-KD, is a principled compression method that identifies informative spectral components and applies iterative knowledge distillation to produce compact and efficient models while preserving accuracy. Together, these results show that spectral statistics provide interpretable and robust signals for monitoring uncertainty and guiding compression in large-scale neural networks.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Hallucination Detection on LLaMa 1 (test)](<https://www.sota2.com/research/sota/hallucination-detection-on-llama-1-test.md>): Hallucination Detection; LLaMa 1 (test); AUROC: 0.894.

- [OOD Detection on LLaMa 1 (test)](<https://www.sota2.com/research/sota/ood-detection-on-llama-1-test.md>): OOD Detection; LLaMa 1 (test); AUROC: 0.924.

- [Natural Language Understanding on GLUE (Custom Metrics)](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue-custom-metrics.md>): Natural Language Processing; GLUE; Red Score: 80.9.

- [Image Classification on CIFAR-10 (Red. and Acc.)](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-red-and-acc.md>): Image Classification; CIFAR-10; Red.: 0.477.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2601.17357>)

[PDF](<https://arxiv.org/pdf/2601.17357.pdf>)
