Long-context Question Answering on QuALITY
73.63AccuracyQwen2.5
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen2.5Parameters=7B, Complexity Type=Quadratic, Evaluation Framework=vLLM, Evaluation Method=generation-based, Chain-of-thought (CoT) Usage=non-CoT2025.09 | 73.63 | |
| SpikingBrain-76BParameters=12B/76B, Complexity Type=Hybrid, Evaluation Framework=vLLM, Evaluation Method=generation-based, Chain-of-thought (CoT) Usage=non-CoT2025.09 | 69.56 | |
| Llama3Parameters=8B, Complexity Type=Quadratic, Evaluation Framework=vLLM, Evaluation Method=generation-based, Chain-of-thought (CoT) Usage=non-CoT2025.09 | 66.25 | |
| SpikingBrain-7BParameters=7B, Complexity Type=Linear, Evaluation Framework=vLLM, Evaluation Method=generation-based, Chain-of-thought (CoT) Usage=non-CoT2025.09 | 60.12 | |
| MixtralParameters=13B/47B, Complexity Type=Quadratic, Evaluation Framework=vLLM, Evaluation Method=generation-based, Chain-of-thought (CoT) Usage=non-CoT2025.09 | 51.34 |