Coding Question Answering on CodeQA
62.1AccuracyRLM
Evaluation Results
| Method | Links | |
|---|---|---|
| RLMParadigm=P2, Model Family=Llama, Model Scale=405B2026.03 | 62.1 | |
| RLMParadigm=P2, Model Family=Qwen3, Model Scale=235B2026.03 | 58.6 | |
| λ-RLMParadigm=P3, Model Family=Llama, Model Scale=405B2026.03 | 55.7 | |
| λ-RLMParadigm=P3, Model Family=Qwen3, Model Scale=235B2026.03 | 54.2 | |
| RLMParadigm=P2, Model Family=Mistral, Model Scale=Cdsrl2026.03 | 49.3 | |
| λ-RLMParadigm=P3, Model Family=Qwen3, Model Scale=32B2026.03 | 47.1 | |
| RLMParadigm=P2, Model Family=Llama, Model Scale=70B2026.03 | 46.4 | |
| λ-RLMParadigm=P3, Model Family=Mistral, Model Scale=Cdsrl2026.03 | 44.8 | |
| λ-RLMParadigm=P3, Model Family=Llama, Model Scale=70B2026.03 | 43.8 | |
| λ-RLMParadigm=P3, Model Family=Mistral, Model Scale=8x22B2026.03 | 42.5 | |
| RLMParadigm=P2, Model Family=Qwen3, Model Scale=32B2026.03 | 36.8 | |
| λ-RLMParadigm=P3, Model Family=Qwen3, Model Scale=8B2026.03 | 35.2 | |
| λ-RLMParadigm=P3, Model Family=Llama, Model Scale=8B2026.03 | 32.8 | |
| RLMParadigm=P2, Model Family=Mistral, Model Scale=8x22B2026.03 | 32.4 | |
| λ-RLMParadigm=P3, Model Family=Mistral, Model Scale=7B2026.03 | 27.6 | |
| DirectParadigm=P1, Model Family=Llama, Model Scale=405B2026.03 | 26.3 | |
| DirectParadigm=P1, Model Family=Qwen3, Model Scale=235B2026.03 | 24 | |
| DirectParadigm=P1, Model Family=Llama, Model Scale=70B2026.03 | 22.1 | |
| DirectParadigm=P1, Model Family=Mistral, Model Scale=Cdsrl2026.03 | 21.2 | |
| DirectParadigm=P1, Model Family=Qwen3, Model Scale=32B2026.03 | 20.4 | |
| DirectParadigm=P1, Model Family=Mistral, Model Scale=8x22B2026.03 | 18.6 | |
| RLMParadigm=P2, Model Family=Qwen3, Model Scale=8B2026.03 | 18.5 | |
| RLMParadigm=P2, Model Family=Llama, Model Scale=8B2026.03 | 16.7 | |
| RLMParadigm=P2, Model Family=Mistral, Model Scale=7B2026.03 | 12.3 | |
| DirectParadigm=P1, Model Family=Llama, Model Scale=8B2026.03 | 9.2 | |
| DirectParadigm=P1, Model Family=Qwen3, Model Scale=8B2026.03 | 8.7 | |
| DirectParadigm=P1, Model Family=Mistral, Model Scale=7B2026.03 | 7.4 |