Reasoning on FLenQA 500 tokens
74AccuracyLIME+1
Evaluation Results
| Method | Links | |
|---|---|---|
| LIME+1Size=1B, Matching criteria=generates and matches only one word2025.12 | 74 | |
| LIME+1Size=500M, Matching criteria=generates and matches only one word2025.12 | 73.8 | |
| LIME+1Model Size=2B2025.12 | 73.5 | |
| LIME+1Size=2B, Matching criteria=generates and matches only one word2025.12 | 73.5 | |
| LIME+1Model Size=2B2025.12 | 73.5 | |
| LIMEModel Size=2B2025.12 | 65.3 | |
| LIMESize=2B, Matching criteria=match the first eight generated words2025.12 | 65.3 | |
| LIMEModel Size=2B2025.12 | 65.3 | |
| BaseModel Size=2B2025.12 | 49.5 | |
| BaselineSize=2B, Matching criteria=match the first eight generated words2025.12 | 49.5 | |
| Base (DCLM-BASELINE)Model Size=2B2025.12 | 49.5 | |
| LIMESize=500M, Matching criteria=match the first eight generated words2025.12 | 48.8 | |
| BaselineSize=1B, Matching criteria=match the first eight generated words2025.12 | 39 | |
| LIMESize=1B, Matching criteria=match the first eight generated words2025.12 | 32 | |
| BaselineSize=500M, Matching criteria=match the first eight generated words2025.12 | 22 |