# Roof

Source: [Roof](<https://www.sota2.com/research/dataset/roof>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Benchmarks

Page 1 of 2. This is an index page, not the full collection.

- [Next index page](<https://www.sota2.com/research/dataset/roof.md?page=2>): Follow this link to discover more pages.

- [Language Modeling on Proof-pile](<https://www.sota2.com/research/sota/language-modeling-on-proof-pile.md>): Language Modeling; Proof-pile.

- [Logical Reasoning on ProofW (Accuracy)](<https://www.sota2.com/research/sota/logical-reasoning-on-proofw-accuracy.md>): Logical Reasoning; ProofW.

- [Logical Reasoning on ProofWriter (test)](<https://www.sota2.com/research/sota/logical-reasoning-on-proofwriter-test.md>): Logical Reasoning; ProofWriter (test).

- [Logical Reasoning on ProofWriter (Accuracy and Toxic Rate)](<https://www.sota2.com/research/sota/logical-reasoning-on-proofwriter-accuracy-and-toxic-rate.md>): Logical Reasoning; ProofWriter.

- [Logical Reasoning on ProofWriter (Accuracy)](<https://www.sota2.com/research/sota/logical-reasoning-on-proofwriter-accuracy.md>): Logical Reasoning; ProofWriter.

- [Language Modeling on Arxiv Proof-pile](<https://www.sota2.com/research/sota/language-modeling-on-arxiv-proof-pile.md>): Language Modeling; Arxiv Proof-pile.

- [Math Problem Solving on 200 IMO-level math problems IMO-AnswerBench, IMO-ProofBench, ArXivMath (test)](<https://www.sota2.com/research/sota/math-problem-solving-on-200-imo-level-math-problems-imo-answerbench-imo-proofbench-arxivmath-test.md>): Math Problem Solving; 200 IMO-level math problems IMO-AnswerBench, IMO-ProofBench, ArXivMath (test).

- [Membership Proofing on Synthetic Membership Proof Sets](<https://www.sota2.com/research/sota/membership-proofing-on-synthetic-membership-proof-sets.md>): Membership Proofing; Synthetic Membership Proof Sets.

- [Mathematical Reasoning on Math ID GSM8k ProofNet](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math-id-gsm8k-proofnet.md>): Mathematical Reasoning; Math ID GSM8k ProofNet.

- [Autoformalization on ProofNet](<https://www.sota2.com/research/sota/autoformalization-on-proofnet.md>): Autoformalization; ProofNet.

- [Theorem Proving on ProofNet (test)](<https://www.sota2.com/research/sota/theorem-proving-on-proofnet-test.md>): Theorem Proving; ProofNet (test).

- [Formal Theorem Proving on ProofNet](<https://www.sota2.com/research/sota/formal-theorem-proving-on-proofnet.md>): Formal Theorem Proving; ProofNet.

- [Logical Reasoning on ProofWriter](<https://www.sota2.com/research/sota/logical-reasoning-on-proofwriter.md>): Logical Reasoning; ProofWriter.

- [Step-level correctness assessment on ProofNet (test)](<https://www.sota2.com/research/sota/step-level-correctness-assessment-on-proofnet-test.md>): Step-level correctness assessment; ProofNet (test).

- [Deductive Reasoning on ProofWriter (End-to-end Accuracy (%))](<https://www.sota2.com/research/sota/deductive-reasoning-on-proofwriter-end-to-end-accuracy-percent.md>): Deductive Reasoning; ProofWriter.

- [Logical Reasoning on ProofWriter OWA (3-hop)](<https://www.sota2.com/research/sota/logical-reasoning-on-proofwriter-owa-3-hop.md>): Logical Reasoning; ProofWriter OWA (3-hop).

- [Logical Reasoning on ProofWriter CWA 3-hop](<https://www.sota2.com/research/sota/logical-reasoning-on-proofwriter-cwa-3-hop.md>): Logical Reasoning; ProofWriter CWA 3-hop.

- [Step-level Reasoning Verification on ProofNet](<https://www.sota2.com/research/sota/step-level-reasoning-verification-on-proofnet.md>): Step-level reasoning verification; ProofNet.

- [Deductive Reasoning on ProofWriter (Pass@1, Variance)](<https://www.sota2.com/research/sota/deductive-reasoning-on-proofwriter-pass-at-1-variance.md>): Deductive Reasoning; ProofWriter.

- [Math Proof Verification on ProofBench](<https://www.sota2.com/research/sota/math-proof-verification-on-proofbench.md>): Math Proof Verification; ProofBench.

- [Language Modeling on Proof-pile (test)](<https://www.sota2.com/research/sota/language-modeling-on-proof-pile-test.md>): Language Modeling; Proof-pile (test).

- [Mesh Quality Evaluation on roof image](<https://www.sota2.com/research/sota/mesh-quality-evaluation-on-roof-image.md>): Mesh Quality Evaluation; roof image.

- [Reasoning quality evaluation on PROOFWRITER](<https://www.sota2.com/research/sota/reasoning-quality-evaluation-on-proofwriter.md>): Reasoning quality evaluation; PROOFWRITER.

- [Explanation Refinement on ProofWriter](<https://www.sota2.com/research/sota/explanation-refinement-on-proofwriter.md>): Explanation Refinement; ProofWriter.

- [Mathematical Proof Reasoning on IMO-ProofBench Advanced](<https://www.sota2.com/research/sota/mathematical-proof-reasoning-on-imo-proofbench-advanced.md>): Mathematical Proof Reasoning; IMO-ProofBench Advanced (full set).

- [Long-form reasoning on NuminaProof (test)](<https://www.sota2.com/research/sota/long-form-reasoning-on-numinaproof-test.md>): Long-form reasoning; NuminaProof (test).

- [Reasoning on ProofWriter](<https://www.sota2.com/research/sota/reasoning-on-proofwriter.md>): Reasoning; ProofWriter.

- [Logical Reasoning on ProofWriter (held-out)](<https://www.sota2.com/research/sota/logical-reasoning-on-proofwriter-held-out.md>): Logical Reasoning; ProofWriter (held-out).

- [Auto-formalization on ProofNet (test)](<https://www.sota2.com/research/sota/auto-formalization-on-proofnet-test.md>): Auto-formalization; ProofNet (test).

- [Deductive logical reasoning on ProofWriter (test)](<https://www.sota2.com/research/sota/deductive-logical-reasoning-on-proofwriter-test.md>): Deductive logical reasoning; ProofWriter (test).

- [Language Modeling on Proofpile (test)](<https://www.sota2.com/research/sota/language-modeling-on-proofpile-test.md>): Language Modeling; Proofpile (test).

- [Autoformalization on ProofNet (test)](<https://www.sota2.com/research/sota/autoformalization-on-proofnet-test.md>): Autoformalization; ProofNet (test).

- [Lemma Judging on NaturalProofs (test)](<https://www.sota2.com/research/sota/lemma-judging-on-naturalproofs-test.md>): Lemma Judging; NaturalProofs (test).

- [Formal Theorem Proving on ProofNet (test)](<https://www.sota2.com/research/sota/formal-theorem-proving-on-proofnet-test.md>): Formal Theorem Proving; ProofNet (test).

- [Mathematical Reasoning on ProofNet](<https://www.sota2.com/research/sota/mathematical-reasoning-on-proofnet.md>): Mathematical Reasoning; ProofNet.

- [Proof writing on IMO-ProofBench](<https://www.sota2.com/research/sota/proof-writing-on-imo-proofbench.md>): Proof writing; IMO-ProofBench.

- [Statement generation on ProofNet N = 186 (test)](<https://www.sota2.com/research/sota/statement-generation-on-proofnet-n-186-test.md>): Statement generation; ProofNet N = 186 (test).

- [Logical Reasoning on ProofW (Accuracy, Token Cost)](<https://www.sota2.com/research/sota/logical-reasoning-on-proofw-accuracy-token-cost.md>): Logical Reasoning; ProofW.

- [Theorem Proving on ProofNet (val)](<https://www.sota2.com/research/sota/theorem-proving-on-proofnet-val.md>): Theorem Proving; ProofNet (val).

- [Lean Theorem Proving on PROOFNET (186 problems)](<https://www.sota2.com/research/sota/lean-theorem-proving-on-proofnet-186-problems.md>): Lean theorem proving; PROOFNET (186 problems).

- [Mathematical Proof on IMO-ProofBench](<https://www.sota2.com/research/sota/mathematical-proof-on-imo-proofbench.md>): Mathematical Proof; IMO-ProofBench Basic, Advanced, Overall.

- [Mathematical Proof Reward Modeling on Proof-RM IMO CMO USAMO 2024-2025 (test)](<https://www.sota2.com/research/sota/mathematical-proof-reward-modeling-on-proof-rm-imo-cmo-usamo-2024-2025-test.md>): Mathematical Proof Reward Modeling; Proof-RM IMO CMO USAMO competition-level problems 2024-2025 (test).

- [Logical Reasoning on ProofWriter depth-5 OWA setting (Accuracy)](<https://www.sota2.com/research/sota/logical-reasoning-on-proofwriter-depth-5-owa-setting-accuracy.md>): Logical Reasoning; ProofWriter depth-5 OWA setting.

- [Language Modeling on ProofPile (16K)](<https://www.sota2.com/research/sota/language-modeling-on-proofpile-16k.md>): Language Modeling; ProofPile (16K).

- [Language Modeling on ProofPile (4K)](<https://www.sota2.com/research/sota/language-modeling-on-proofpile-4k.md>): Language Modeling; ProofPile (4K).

- [Deductive Reasoning on ProofWriter](<https://www.sota2.com/research/sota/deductive-reasoning-on-proofwriter.md>): Deductive Reasoning; ProofWriter.

- [Language Modeling on Proof-pile 128 32k (test)](<https://www.sota2.com/research/sota/language-modeling-on-proof-pile-128-32k-test.md>): Language Modeling; Proof-pile 128 32k-length documents (test).

- [Theorem Proving on INT Proof length 15](<https://www.sota2.com/research/sota/theorem-proving-on-int-proof-length-15.md>): Theorem Proving; INT Proof length 15.

- [Theorem Proving on INT Proof length 10](<https://www.sota2.com/research/sota/theorem-proving-on-int-proof-length-10.md>): Theorem Proving; INT Proof length 10.

- [Theorem Proving on INT (Proof length 5)](<https://www.sota2.com/research/sota/theorem-proving-on-int-proof-length-5.md>): Theorem Proving; INT (Proof length 5).

- [Mathematical Proof Reasoning on IMO-ProofBench-Advanced USAMO 2025 (5 problems)](<https://www.sota2.com/research/sota/mathematical-proof-reasoning-on-imo-proofbench-advanced-usamo-2025-5-problems.md>): Mathematical Proof Reasoning; IMO-ProofBench-Advanced USAMO 2025 (5 problems).

- [Mathematical Proof Reasoning on IMO-ProofBench-Advanced Novel (22 problems)](<https://www.sota2.com/research/sota/mathematical-proof-reasoning-on-imo-proofbench-advanced-novel-22-problems.md>): Mathematical Proof Reasoning; IMO-ProofBench-Advanced Novel (22 problems).

- [Mathematical Proof Correctness on First Proof 16 (Expert Evaluation)](<https://www.sota2.com/research/sota/mathematical-proof-correctness-on-first-proof-16-expert-evaluation.md>): Mathematical proof correctness; First Proof 16 (Expert Evaluation).

- [ZKP Performance Evaluation on Zero-Knowledge Proof (ZKP) Protocols](<https://www.sota2.com/research/sota/zkp-performance-evaluation-on-zero-knowledge-proof-zkp-protocols.md>): ZKP Performance Evaluation; Zero-Knowledge Proof (ZKP) Protocols.

- [Logical Reasoning on ProofWriter (Acc, TTFT, Delay (s))](<https://www.sota2.com/research/sota/logical-reasoning-on-proofwriter-acc-ttft-delay-s.md>): Logical Reasoning; ProofWriter.

- [Theorem Proving on ProofNet (all)](<https://www.sota2.com/research/sota/theorem-proving-on-proofnet-all.md>): Theorem Proving; ProofNet (all).

- [Logical Reasoning on ProofWriter (Acc, TNFT, Tokens, Delay)](<https://www.sota2.com/research/sota/logical-reasoning-on-proofwriter-acc-tnft-tokens-delay.md>): Logical Reasoning; ProofWriter.

- [Mathematical Reasoning on ProofNet (test)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-proofnet-test.md>): Mathematical Reasoning; ProofNet (test).

- [Language Modeling on ProofPile 100K](<https://www.sota2.com/research/sota/language-modeling-on-proofpile-100k.md>): Language Modeling; ProofPile 100K.

- [Language Modeling on ProofPile 32K](<https://www.sota2.com/research/sota/language-modeling-on-proofpile-32k.md>): Language Modeling; ProofPile 32K.

- [Language Modeling on Proof-Pile (PPL and LongPPL)](<https://www.sota2.com/research/sota/language-modeling-on-proof-pile-ppl-and-longppl.md>): Language Modeling; Proof-Pile.

- [Formal Theorem Proving on ProofNet (val)](<https://www.sota2.com/research/sota/formal-theorem-proving-on-proofnet-val.md>): Formal Theorem Proving; ProofNet (val).

- [Video Deinterlacing on Roof (test)](<https://www.sota2.com/research/sota/video-deinterlacing-on-roof-test.md>): Video Deinterlacing; Roof (test).

- [Auto-formalization on ProofNet](<https://www.sota2.com/research/sota/auto-formalization-on-proofnet.md>): Auto-formalization; ProofNet.

- [Omission Detectability Analysis on Proof of Solvency Omission Detection (N=10^7, k=100) 1.0](<https://www.sota2.com/research/sota/omission-detectability-analysis-on-proof-of-solvency-omission-detection-n-10-7-k-100-1-0.md>): Omission Detectability Analysis; Proof of Solvency Omission Detection (N=10^7, k=100) 1.0 (Evaluation Scenario).

- [Inverse Model Generalization on SOLAR (Solar Rooftop Generation)](<https://www.sota2.com/research/sota/inverse-model-generalization-on-solar-solar-rooftop-generation.md>): Inverse Model Generalization; SOLAR (Solar Rooftop Generation).

- [Decentralized Federated Learning on CIFAR-10 Roofnet topology](<https://www.sota2.com/research/sota/decentralized-federated-learning-on-cifar-10-roofnet-topology.md>): Decentralized Federated Learning; CIFAR-10 Roofnet topology.

- [Automated Theorem Proving on seL4 proof corpus (full library)](<https://www.sota2.com/research/sota/automated-theorem-proving-on-sel4-proof-corpus-full-library.md>): Automated Theorem Proving; seL4 proof corpus (full library).

- [Theorem Retrieval on Raw Proof State](<https://www.sota2.com/research/sota/theorem-retrieval-on-raw-proof-state.md>): Theorem Retrieval; Raw Proof State.

- [Theorem Retrieval on Augmented Proof State](<https://www.sota2.com/research/sota/theorem-retrieval-on-augmented-proof-state.md>): Theorem Retrieval; Augmented Proof State.

- [Statement Autoformalization on ProofNet](<https://www.sota2.com/research/sota/statement-autoformalization-on-proofnet.md>): Statement Autoformalization; ProofNet.

- [Mathematical Proof Construction on IMOProofBench](<https://www.sota2.com/research/sota/mathematical-proof-construction-on-imoproofbench.md>): Mathematical Proof Construction; IMOProofBench.

- [Proof Verification on 21-proof (Diagnostic Set)](<https://www.sota2.com/research/sota/proof-verification-on-21-proof-diagnostic-set.md>): Proof Verification; 21-proof (Diagnostic Set).

- [Formal Reasoning on ProofNet](<https://www.sota2.com/research/sota/formal-reasoning-on-proofnet.md>): Formal Reasoning; ProofNet.

- [Geometric Proving on UniGeo proof part](<https://www.sota2.com/research/sota/geometric-proving-on-unigeo-proof-part.md>): Geometric Proving; UniGeo proof part.

- [Unsupervised Semantic Segmentation on Roof Subassembly Damage Detection](<https://www.sota2.com/research/sota/unsupervised-semantic-segmentation-on-roof-subassembly-damage-detection.md>): Unsupervised Semantic Segmentation; Roof Subassembly Damage Detection.

- [Autoformalization and Proving on ProofNet N=186 (test)](<https://www.sota2.com/research/sota/autoformalization-and-proving-on-proofnet-n-186-test.md>): Autoformalization and Proving; ProofNet N=186 (test).

- [Proof of Liabilities Evaluation on Synthetic Proof of Liabilities (PoL) Environment](<https://www.sota2.com/research/sota/proof-of-liabilities-evaluation-on-synthetic-proof-of-liabilities-pol-environment.md>): Proof of Liabilities Evaluation; Synthetic Proof of Liabilities (PoL) Environment.

- [Next Step Generation on PeanoBench 21 incorrect proofs](<https://www.sota2.com/research/sota/next-step-generation-on-peanobench-21-incorrect-proofs.md>): Next Step Generation; PeanoBench 21 incorrect proofs.

- [Hint/Question Generation on PeanoBench 21 incorrect proofs](<https://www.sota2.com/research/sota/hint-question-generation-on-peanobench-21-incorrect-proofs.md>): Hint/Question Generation; PeanoBench 21 incorrect proofs.