# Information-Theoretic Limits of Safety Verification for Self-Improving Systems

Source: [Information-Theoretic Limits of Safety Verification for Self-Improving Systems](<https://www.sota2.com/research/paper/69cb55870636e5bf4afca49c>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-04T05:00:04.515Z

Authors: Arsenios Scrivens

Published: 2026-03-30T16:34:37.000Z

## Abstract

Can a safety gate permit unbounded beneficial self-modification while maintaining bounded cumulative risk? We formalize this question through dual conditions -- requiring sum delta\_n \< infinity (bounded risk) and sum TPR\_n = infinity (unbounded utility) -- and establish a theory of their (in)compatibility.   Classification impossibility (Theorem 1): For power-law risk schedules delta\_n = O(n^{-p}) with p \> 1, any classifier-based gate under overlapping safe/unsafe distributions satisfies TPR\_n \<= C\_alpha \* delta\_n^beta via Holder's inequality, forcing sum TPR\_n \< infinity. This impossibility is exponent-optimal (Theorem 3). A second independent proof via the NP counting method (Theorem 4) yields a 13% tighter bound without Holder's inequality.   Universal finite-horizon ceiling (Theorem 5): For any summable risk schedule, the exact maximum achievable classifier utility is U\*(N, B) = N \* TPR\_NP(B/N), growing as exp(O(sqrt(log N))) -- subpolynomial. At N = 10^6 with budget B = 1.0, a classifier extracts at most U\* ~ 87 versus a verifier's ~500,000.   Verification escape (Theorem 2): A Lipschitz ball verifier achieves delta = 0 with TPR \> 0, escaping the impossibility. Formal Lipschitz bounds for pre-LayerNorm transformers under LoRA enable LLM-scale verification. The separation is strict. We validate on GPT-2 (d\_LoRA = 147,456): conditional delta = 0 with TPR = 0.352. Comprehensive empirical validation is in the companion paper \[D2\].

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Safety Gate Capacity Scaling on Safety Scaling Testbed N=100-10,000](<https://www.sota2.com/research/sota/safety-gate-capacity-scaling-on-safety-scaling-testbed-n-100-10-000.md>): Safety Gate Capacity Scaling; Safety Scaling Testbed N=100-10,000; Classifier Ceiling (B=0.01): 12.4.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2603.28650>)

[PDF](<https://arxiv.org/pdf/2603.28650.pdf>)