Mathematical Reasoning on GSM8K and AIME24
46.9Accuracy (GSM8K)FluxAttn (FA-SSA)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| FluxAttn (FA-SSA)Backbone=Qwen3-8B2026.04 | 46.9 | 42.35 | 44.63 | |
| FluxAttn (FA-TA)Backbone=Llama-3.1-8B-Instruct2026.04 | 46.9 | 38.3 | 42.6 | |
| TriangleMixBackbone=Llama-3.1-8B-Instruct2026.04 | 46.3 | 37.05 | 41.68 | |
| FluxAttn (FA-SSA)Backbone=Llama-3.1-8B-Instruct, Configuration=Variant 22026.04 | 45.9 | 37.35 | 41.63 | |
| FluxAttn (FA-SSA)Backbone=Qwen3-8B, Configuration=Variant 22026.04 | 45.3 | 43.2 | 44.25 | |
| FluxAttn (FA-SSA)Backbone=Llama-3.1-8B-Instruct2026.04 | 45.3 | 37.05 | 41.18 | |
| FluxAttn (FA-TA)Backbone=Qwen3-4B2026.04 | 45 | 40.35 | 42.68 | |
| DuoAttentionBackbone=Llama-3.1-8B-Instruct2026.04 | 44.4 | 33.65 | 39.03 | |
| FluxAttn (FA-XA)Backbone=Llama-3.1-8B-Instruct2026.04 | 44.4 | 33.65 | 39.03 | |
| FluxAttn (FA-XA)Backbone=Qwen3-8B2026.04 | 43.2 | 35.65 | 39.43 | |
| FluxAttn (FA-TA)Backbone=Qwen3-8B2026.04 | 43 | 39.05 | 41.03 | |
| Llama-3.1-8B-InstructBackbone=Llama-3.1-8B-Instruct2026.04 | 42.3 | 30.35 | 36.33 | |
| FluxAttn (FA-XA)Backbone=Qwen3-4B2026.04 | 42.2 | 40.35 | 41.28 | |
| PruLongBackbone=Llama-3.1-8B-Instruct2026.04 | 41.3 | 29.85 | 35.58 | |
| DuoAttentionBackbone=Qwen3-8B2026.04 | 41.2 | 35.65 | 38.43 | |
| TriangleMixBackbone=Qwen3-8B2026.04 | 41.2 | 44.15 | 42.68 | |
| Qwen3-8BBackbone=Qwen3-8B2026.04 | 40.6 | 32.35 | 36.48 | |
| PruLongBackbone=Qwen3-8B2026.04 | 40.4 | 32.35 | 36.38 | |
| TriangleMixBackbone=Qwen3-4B2026.04 | 40.3 | 37.25 | 38.78 | |
| FluxAttn (FA-SSA)Backbone=Qwen3-4B2026.04 | 40.3 | 37.05 | 38.68 | |
| FluxAttn (FA-SSA)Backbone=Qwen3-4B, Configuration=Variant 22026.04 | 39.9 | 37.25 | 38.58 | |
| Qwen3-4BBackbone=Qwen3-4B2026.04 | 39.7 | 30.35 | 35.03 | |
| DuoAttentionBackbone=Qwen3-4B2026.04 | 39.7 | 37.05 | 38.38 | |
| PruLongBackbone=Qwen3-4B2026.04 | 39.7 | 30.35 | 35.03 |