Reasoning on OpenCompass (test)
69.58CMMLUQwen1.5-A2.7B
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen1.5-A2.7Brouting=top4, backbone=Qwen1.5-MoE-A2.7B-Chat, QPS=162026.02 | 69.58 | 80.46 | 34.97 | 14.38 | 51.86 | 60.6 | 45.73 | 30.6 | 48.52 | 17.29 | |
| SERErouting=top2, rho=0.3, backbone=Qwen1.5-MoE-A2.7B-Chat, QPS=162026.02 | 68.49 | 79.97 | 34.61 | 14.66 | 51.63 | 58.35 | 42.68 | 27.6 | 47.25 | 13.93 | |
| SERErouting=top2, rho=0, backbone=Qwen1.5-MoE-A2.7B-Chat, QPS=162026.02 | 68.12 | 80.15 | 33.16 | 14.06 | 50.19 | 58.35 | 46.95 | 26.2 | 47.15 | 13.83 | |
| Qwen1.5-A2.7Brouting=top2, backbone=Qwen1.5-MoE-A2.7B-Chat, QPS=162026.02 | 66.69 | 75.87 | 32.26 | 12.92 | 44.28 | 51.36 | 32.02 | 27.4 | 42.85 | 13.53 | |
| SERErouting=top1, rho=0.3, backbone=Qwen1.5-MoE-A2.7B-Chat, QPS=162026.02 | 65.83 | 78.69 | 33.62 | 9.74 | 39.88 | 53.12 | 17.07 | 20.6 | 39.82 | 12.95 | |
| SERErouting=top1, rho=0, backbone=Qwen1.5-MoE-A2.7B-Chat, QPS=162026.02 | 60.09 | 79.85 | 32.71 | 7.58 | 33.36 | 52.12 | 17.07 | 20.2 | 37.87 | 12.13 | |
| Qwen1.5-A2.7Brouting=top1, backbone=Qwen1.5-MoE-A2.7B-Chat, QPS=162026.02 | 45.12 | 48.35 | 29.47 | 5.24 | 26.16 | 46.13 | 15.85 | 14.8 | 28.89 | 11.47 | |
| HC-SMoEexperts=40, backbone=Qwen1.5-MoE-A2.7B-Chat, QPS=162026.02 | 45.11 | 74.95 | 29.01 | 4.26 | 27.67 | 42.64 | 4.88 | 1.8 | 28.79 | 14.2 | |
| LYNXrouting=top2, backbone=Qwen1.5-MoE-A2.7B-Chat, QPS=162026.02 | 42.57 | 78.62 | 23.59 | 9.56 | 29.57 | 34.41 | 8.54 | 7.4 | 29.28 | 14.49 | |
| LYNXrouting=top1, backbone=Qwen1.5-MoE-A2.7B-Chat, QPS=162026.02 | 16.6 | 77.68 | 15.1 | 0.68 | 2.12 | 10.22 | 0 | 0.2 | 15.33 | 12.95 | |
| HC-SMoEexperts=30, backbone=Qwen1.5-MoE-A2.7B-Chat, QPS=162026.02 | 7.93 | 34.19 | 29.14 | 1.72 | 8.95 | 29.18 | 0.61 | 0 | 13.97 | 13.3 |