Formal Theorem Proving on Isar n-ary tree consensus Individual Lemmas and Average
100Success RateIsabeLLM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| IsabeLLMModel backbone=R1, RAG usage=No, Lemma=subtree_height2026.06 | 100 | 1 | |
| IsabeLLMModel backbone=R1, RAG usage=No, Lemma=height_mono2026.06 | 100 | 1 | |
| IsabeLLMModel backbone=R1, RAG usage=No, Lemma=weaken_distance2026.06 | 100 | 1 | |
| IsabeLLMModel backbone=R1, RAG usage=No, Lemma=weaken_depth2026.06 | 100 | 1 | |
| IsabeLLMModel backbone=R1, RAG usage=No, Lemma=check_add_mining2026.06 | 100 | 1 | |
| IsabeLLMModel backbone=R1, RAG usage=No, Lemma=consensus2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Chimera, RAG usage=Yes, Lemma=subtree_height2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Nemotron, RAG usage=Yes, Lemma=subtree_height2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=GPT-OSS, RAG usage=Yes, Lemma=subtree_height2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Chimera, RAG usage=Yes, Lemma=height_mono2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Nemotron, RAG usage=Yes, Lemma=height_mono2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=GPT-OSS, RAG usage=Yes, Lemma=height_mono2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Chimera, RAG usage=Yes, Lemma=obtain_max2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Nemotron, RAG usage=Yes, Lemma=obtain_max2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=GPT-OSS, RAG usage=Yes, Lemma=obtain_max2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Chimera, RAG usage=Yes, Lemma=foldr_max_eq2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Nemotron, RAG usage=Yes, Lemma=sub_longest2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Chimera, RAG usage=Yes, Lemma=weaken_distance2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Nemotron, RAG usage=Yes, Lemma=weaken_distance2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=GPT-OSS, RAG usage=Yes, Lemma=weaken_distance2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Chimera, RAG usage=Yes, Lemma=weaken_depth2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Nemotron, RAG usage=Yes, Lemma=weaken_depth2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=GPT-OSS, RAG usage=Yes, Lemma=weaken_depth2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Chimera, RAG usage=Yes, Lemma=check_add_mining2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Nemotron, RAG usage=Yes, Lemma=check_add_mining2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=GPT-OSS, RAG usage=Yes, Lemma=check_add_mining2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Chimera, RAG usage=Yes, Lemma=height_add_honest2026.06 | 100 | 1.2 | |
| IsabeLLM-RAGModel backbone=Nemotron, RAG usage=Yes, Lemma=height_add_honest2026.06 | 100 | 1.5 | |
| IsabeLLM-RAGModel backbone=GPT-OSS, RAG usage=Yes, Lemma=height_add_honest2026.06 | 100 | 2.5 | |
| IsabeLLM-RAGModel backbone=Chimera, RAG usage=Yes, Lemma=check_add_honest2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Nemotron, RAG usage=Yes, Lemma=check_add_honest2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Chimera, RAG usage=Yes, Lemma=bounded_check2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Nemotron, RAG usage=Yes, Lemma=bounded_check2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Chimera, RAG usage=Yes, Lemma=consensus2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Nemotron, RAG usage=Yes, Lemma=consensus2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=GPT-OSS, RAG usage=Yes, Lemma=consensus2026.06 | 100 | 1 | |
| IsabeLLM-RAGModel backbone=Chimera, RAG usage=Yes, Lemma=Average2026.06 | 94.4 | 1.06 | |
| IsabeLLMModel backbone=R1, RAG usage=No, Lemma=obtain_max2026.06 | 90 | 1.4 | |
| IsabeLLMModel backbone=R1, RAG usage=No, Lemma=check_add_honest2026.06 | 90 | 1.2 | |
| IsabeLLM-RAGModel backbone=Chimera, RAG usage=Yes, Lemma=branch_height2026.06 | 90 | 1.1 | |
| IsabeLLM-RAGModel backbone=Chimera, RAG usage=Yes, Lemma=sub_longest2026.06 | 90 | 1.1 | |
| IsabeLLM-RAGModel backbone=Nemotron, RAG usage=Yes, Lemma=Average2026.06 | 87.5 | 1.15 | |
| IsabeLLMModel backbone=R1, RAG usage=No, Lemma=height_add_honest2026.06 | 80 | 1.7 | |
| IsabeLLM-RAGModel backbone=Nemotron, RAG usage=Yes, Lemma=foldr_max_eq2026.06 | 80 | 1.1 | |
| IsabeLLM-RAGModel backbone=Chimera, RAG usage=Yes, Lemma=sub_branch2026.06 | 80 | 1.1 | |
| IsabeLLM-RAGModel backbone=Chimera, RAG usage=Yes, Lemma=height_add_mining2026.06 | 80 | 1.1 | |
| IsabeLLMModel backbone=R1, RAG usage=No, Lemma=Average2026.06 | 76.3 | 1.31 | |
| IsabeLLMModel backbone=R1, RAG usage=No, Lemma=sub_longest2026.06 | 70 | 1.1 | |
| IsabeLLMModel backbone=R1, RAG usage=No, Lemma=bounded_check2026.06 | 70 | 1 | |
| IsabeLLM-RAGModel backbone=Chimera, RAG usage=Yes, Lemma=common_prefix2026.06 | 70 | 1.4 | |
| IsabeLLM-RAGModel backbone=Nemotron, RAG usage=Yes, Lemma=height_add_mining2026.06 | 70 | 2 | |
| IsabeLLM-RAGModel backbone=GPT-OSS, RAG usage=Yes, Lemma=Average2026.06 | 67.5 | 1.44 | |
| IsabeLLMModel backbone=R1, RAG usage=No, Lemma=common_prefix2026.06 | 60 | 1.5 | |
| IsabeLLMModel backbone=R1, RAG usage=No, Lemma=height_add_mining2026.06 | 60 | 2 | |
| IsabeLLM-RAGModel backbone=GPT-OSS, RAG usage=Yes, Lemma=branch_height2026.06 | 60 | 2 | |
| IsabeLLMModel backbone=R1, RAG usage=No, Lemma=foldr_max_eq2026.06 | 50 | 2 | |
| IsabeLLMModel backbone=R1, RAG usage=No, Lemma=sub_branch2026.06 | 50 | 1.8 | |
| IsabeLLM-RAGModel backbone=Nemotron, RAG usage=Yes, Lemma=branch_height2026.06 | 50 | 1.2 | |
| IsabeLLM-RAGModel backbone=Nemotron, RAG usage=Yes, Lemma=sub_branch2026.06 | 50 | 1.4 | |
| IsabeLLM-RAGModel backbone=Nemotron, RAG usage=Yes, Lemma=common_prefix2026.06 | 50 | 1.2 | |
| IsabeLLM-RAGModel backbone=GPT-OSS, RAG usage=Yes, Lemma=bounded_check2026.06 | 50 | 1.1 | |
| IsabeLLM-RAGModel backbone=GPT-OSS, RAG usage=Yes, Lemma=sub_longest2026.06 | 40 | 1 | |
| IsabeLLM-RAGModel backbone=GPT-OSS, RAG usage=Yes, Lemma=sub_branch2026.06 | 40 | 1 | |
| IsabeLLM-RAGModel backbone=GPT-OSS, RAG usage=Yes, Lemma=check_add_honest2026.06 | 40 | 2 | |
| IsabeLLM-RAGModel backbone=GPT-OSS, RAG usage=Yes, Lemma=foldr_max_eq2026.06 | 30 | 3 | |
| IsabeLLM-RAGModel backbone=GPT-OSS, RAG usage=Yes, Lemma=height_add_mining2026.06 | 20 | 2 | |
| IsabeLLMModel backbone=R1, RAG usage=No, Lemma=branch_height2026.06 | 0 | — | |
| IsabeLLM-RAGModel backbone=GPT-OSS, RAG usage=Yes, Lemma=common_prefix2026.06 | 0 | — |