Language Understanding on Chinese Benchmarks CMMLU & AGI-Eval
0.4453CMMLUFullCPT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| FullCPTBackbone=Llama-2 13B, CPT=FullCPT, Shot=3-shot2024.06 | 0.4453 | 0.4002 | 0.4228 | |
| SATBackbone=Llama-2 13B, CPT=SAT, Shot=3-shot2024.06 | 0.4436 | 0.3978 | 0.4207 | |
| SPDFBackbone=Llama-2 13B, CPT=SPDF, Shot=3-shot2024.06 | 0.4071 | 0.372 | 0.3896 | |
| OriginalBackbone=Llama-2 13B, CPT=Original, Shot=3-shot2024.06 | 0.3808 | 0.3773 | 0.3791 | |
| FullCPTBackbone=Llama-2 7B, CPT=FullCPT, Shot=3-shot2024.06 | 0.3288 | 0.3194 | 0.3241 | |
| SATBackbone=Llama-2 7B, CPT=SAT, Shot=3-shot2024.06 | 0.3279 | 0.3184 | 0.3232 | |
| OriginalBackbone=Llama-2 7B, CPT=Original, Shot=3-shot2024.06 | 0.3227 | 0.3004 | 0.3116 | |
| SPDFBackbone=Llama-2 7B, CPT=SPDF, Shot=3-shot2024.06 | 0.3189 | 0.3179 | 0.3184 |