Chinese Language Understanding on CMMLU (test)
0.574CMMLU ScoreFBS-Full
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| FBS-FullLatency (ms)=532, TFLOPs (rel)=0.7, Bypass/Skip (%)=36, Hardware=single A100, Sequence context=512 to 128, Decoding=greedy2026.01 | 0.574 | — | — | — | |
| FFN-SkipLLMLatency (ms)=625, TFLOPs (rel)=0.86, Bypass/Skip (%)=35, Hardware=single A100, Sequence context=512 to 128, Decoding=greedy2026.01 | 0.565 | — | — | — | |
| FlexiDepthLatency (ms)=610, TFLOPs (rel)=0.83, Bypass/Skip (%)=20, Hardware=single A100, Sequence context=512 to 128, Decoding=greedy2026.01 | 0.562 | — | — | — | |
| Baseline (target)Latency (ms)=760, TFLOPs (rel)=1, Bypass/Skip (%)=0, Hardware=single A100, Sequence context=512 to 128, Decoding=greedy2026.01 | 0.557 | 100 | 100 | — | |
| Self-SpecLatency (ms)=585, TFLOPs (rel)=0.77, Bypass/Skip (%)=42, Hardware=single A100, Sequence context=512 to 128, Decoding=greedy2026.01 | 0.557 | 99.8 | 99.2 | — | |
| KangarooLatency (ms)=565, TFLOPs (rel)=0.75, Bypass/Skip (%)=48, Hardware=single A100, Sequence context=512 to 128, Decoding=greedy2026.01 | 0.557 | 99.9 | 99.5 | — | |
| Moxin-Chinese2025.12 | 0.45 | — | — | 45.76 | |
| ymcui/Chinese-LLaMA-Alpaca-2-13B2025.12 | 0.432 | — | — | 44.3 | |
| hfl/chinese-llama-2-13b2025.12 | 0.41 | — | — | 43.25 | |
| Linly-AI/Chinese-LLaMA-2-13B-hf2025.12 | 0.399 | — | — | 42.48 | |
| gywy/Mistral-7B-v0.1-chinese2025.12 | 0.374 | — | — | 36.45 | |
| Linly-AI/Chinese-LLaMA-2-7B-hf2025.12 | 0.312 | — | — | 30.14 | |
| hfl/chinese-llama-2-7b2025.12 | 0.274 | — | — | 33.38 |