Semantic Needle-In-A-Haystack on S-NIAH
52.4AccuracyRLM
Evaluation Results
| Method | Links | |
|---|---|---|
| RLMParadigm=P2, Model Family=Llama, Model Scale=405B2026.03 | 52.4 | |
| λ-RLMParadigm=P3, Model Family=Qwen3, Model Scale=235B2026.03 | 51.3 | |
| λ-RLMParadigm=P3, Model Family=Llama, Model Scale=405B2026.03 | 49.8 | |
| RLMParadigm=P2, Model Family=Qwen3, Model Scale=235B2026.03 | 46.8 | |
| λ-RLMParadigm=P3, Model Family=Llama, Model Scale=70B2026.03 | 44.2 | |
| λ-RLMParadigm=P3, Model Family=Qwen3, Model Scale=32B2026.03 | 41.8 | |
| λ-RLMParadigm=P3, Model Family=Mistral, Model Scale=Cdsrl2026.03 | 40.2 | |
| λ-RLMParadigm=P3, Model Family=Mistral, Model Scale=8x22B2026.03 | 38.6 | |
| DirectParadigm=P1, Model Family=Llama, Model Scale=405B2026.03 | 35.2 | |
| DirectParadigm=P1, Model Family=Qwen3, Model Scale=235B2026.03 | 31.7 | |
| RLMParadigm=P2, Model Family=Llama, Model Scale=70B2026.03 | 31.5 | |
| RLMParadigm=P2, Model Family=Mistral, Model Scale=Cdsrl2026.03 | 29.7 | |
| RLMParadigm=P2, Model Family=Qwen3, Model Scale=32B2026.03 | 28.3 | |
| RLMParadigm=P2, Model Family=Mistral, Model Scale=8x22B2026.03 | 26.4 | |
| λ-RLMParadigm=P3, Model Family=Qwen3, Model Scale=8B2026.03 | 24.6 | |
| DirectParadigm=P1, Model Family=Llama, Model Scale=70B2026.03 | 22.6 | |
| λ-RLMParadigm=P3, Model Family=Llama, Model Scale=8B2026.03 | 22.1 | |
| DirectParadigm=P1, Model Family=Mistral, Model Scale=8x22B2026.03 | 19.1 | |
| λ-RLMParadigm=P3, Model Family=Mistral, Model Scale=7B2026.03 | 18.5 | |
| DirectParadigm=P1, Model Family=Qwen3, Model Scale=32B2026.03 | 18.4 | |
| DirectParadigm=P1, Model Family=Mistral, Model Scale=Cdsrl2026.03 | 14.3 | |
| RLMParadigm=P2, Model Family=Llama, Model Scale=8B2026.03 | 10.2 | |
| RLMParadigm=P2, Model Family=Qwen3, Model Scale=8B2026.03 | 8.4 | |
| RLMParadigm=P2, Model Family=Mistral, Model Scale=7B2026.03 | 6.1 | |
| DirectParadigm=P1, Model Family=Llama, Model Scale=8B2026.03 | 4.1 | |
| DirectParadigm=P1, Model Family=Qwen3, Model Scale=8B2026.03 | 3.2 | |
| DirectParadigm=P1, Model Family=Mistral, Model Scale=7B2026.03 | 2.8 |