Reasoning on Sudoku (test)
100AccuracyLT
Evaluation Results
| Method | Links | |
|---|---|---|
| LTSize=6M, Depth T=22025.10 | 100 | |
| LTSize=6M, Depth T=32025.10 | 100 | |
| CCDDSize=6M, Depth T=22025.10 | 100 | |
| CCDDSize=6M, Depth T=32025.10 | 100 | |
| MDMSize=6M, Depth T=202025.10 | 99.9 | |
| MDMSize=6M, Depth T=22025.10 | 88.6 | |
| LlamaSize=13B, Depth T=Seq. Len.2025.10 | 33.8 | |
| LlamaSize=7B, Depth T=Seq. Len.2025.10 | 27.1 | |
| GPT2 scratchSize=303M, Depth T=Seq. Len.2025.10 | 19.4 | |
| GPT2 scratchSize=6M, Depth T=Seq. Len.2025.10 | 16.2 | |
| EDITSequence Length=128, 0-shot=true, SFT=true2025.11 | 0.161 | |
| LLaDASFT=true, Sequence Length=128, 0-shot=true2025.11 | 0.114 | |
| EDITSequence Length=256, 0-shot=true, SFT=true2025.11 | 0.113 | |
| LLaDASFT=false, Sequence Length=128, 0-shot=true2025.11 | 0.104 | |
| LLaDASFT=true, Sequence Length=256, 0-shot=true2025.11 | 0.082 | |
| EDITSequence Length=512, 0-shot=true, SFT=true2025.11 | 0.076 | |
| LLaDASFT=false, Sequence Length=256, 0-shot=true2025.11 | 0.064 | |
| LLaDASFT=false, Sequence Length=512, 0-shot=true2025.11 | 0.063 | |
| LLaDASFT=true, Sequence Length=512, 0-shot=true2025.11 | 0.05 |