Lemma Discovery on AFP Recent formalizations (Swap Distance, Linear Orders) 2026 (test)
41.7Lemma Discovery RateLEMMANAID DeepSeek-coder-6.7B
Evaluation Results
| Method | Links | |
|---|---|---|
| LEMMANAID DeepSeek-coder-6.7BStrategy=Template, Backbone=DeepSeek-coder-6.7B, Evaluation=Fine-tuned2025.04 | 41.7 | |
| Neural DeepSeek-coder-6.7BStrategy=Lemma-object, Backbone=DeepSeek-coder-6.7B, Evaluation=Fine-tuned2025.04 | 38.1 | |
| Claude Opus 4.5Strategy=Lemma-object, Evaluation=Few-shot2025.04 | 34.5 | |
| LEMMANAID DeepSeek-coder-1.3BStrategy=Template, Backbone=DeepSeek-coder-1.3B, Evaluation=Fine-tuned2025.04 | 30.9 | |
| LEMMANAID Llama-3.2-1BStrategy=Template, Backbone=Llama-3.2-1B, Evaluation=Fine-tuned2025.04 | 28.1 | |
| Neural DeepSeek-coder-1.3BStrategy=Lemma-object, Backbone=DeepSeek-coder-1.3B, Evaluation=Fine-tuned2025.04 | 25.2 | |
| GPT-5.2Strategy=Lemma-object, Evaluation=Few-shot2025.04 | 25.2 | |
| Neural Llama-3.2-1BStrategy=Lemma-object, Backbone=Llama-3.2-1B, Evaluation=Fine-tuned2025.04 | 23 | |
| Claude Opus 4.5Strategy=Template, Evaluation=Few-shot2025.04 | 19.4 | |
| GPT-5.2Strategy=Template, Evaluation=Few-shot2025.04 | 15.8 |