Multi-task Language Understanding on MMLU-Pro (Best and Average Accuracy)
71.4Best AccuracyVecCISC + KMeans
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VecCISC + KMeansLLM Backbone=Llama3.3 70B2026.05 | 71.4 | 71.3 | |
| CISCLLM Backbone=Llama3.3 70B2026.05 | 71.3 | 71.3 | |
| VecCISC + HACLLM Backbone=Llama3.3 70B2026.05 | 71.3 | 71.3 | |
| SC BaselineLLM Backbone=Llama3.3 70B2026.05 | 69.9 | 69.9 | |
| VecCISC (random)LLM Backbone=Llama3.3 70B2026.05 | 68.4 | 67.7 | |
| VecCISC + KMeansLLM Backbone=GPT 4o-mini2026.05 | 66.2 | 66.1 | |
| VecCISC + HACLLM Backbone=GPT 4o-mini2026.05 | 66.1 | 66.1 | |
| CISCLLM Backbone=GPT 4o-mini2026.05 | 65.9 | 65.9 | |
| SC BaselineLLM Backbone=GPT 4o-mini2026.05 | 64.6 | 64.6 | |
| VecCISC (random)LLM Backbone=GPT 4o-mini2026.05 | 64.6 | 63.4 | |
| VecCISC + KMeansLLM Backbone=Qwen2.5 7B2026.05 | 61 | 60.7 | |
| VecCISC + HACLLM Backbone=Qwen2.5 7B2026.05 | 60.9 | 60.9 | |
| CISCLLM Backbone=Qwen2.5 7B2026.05 | 60.2 | 60.2 | |
| SC BaselineLLM Backbone=Qwen2.5 7B2026.05 | 59.3 | 59.3 | |
| VecCISC + KMeansLLM Backbone=Llama3.1 8B2026.05 | 57.9 | 57.7 | |
| VecCISC + HACLLM Backbone=Llama3.1 8B2026.05 | 57.8 | 57.8 | |
| CISCLLM Backbone=Llama3.1 8B2026.05 | 57.6 | 57.6 | |
| SC BaselineLLM Backbone=Llama3.1 8B2026.05 | 56 | 56 | |
| VecCISC (random)LLM Backbone=Qwen2.5 7B2026.05 | 55.6 | 54.6 | |
| VecCISC (random)LLM Backbone=Llama3.1 8B2026.05 | 47.1 | 45.7 | |
| VecCISC + KMeansLLM Backbone=Mistral 7B2026.05 | 28.4 | 28.3 | |
| VecCISC + HACLLM Backbone=Mistral 7B2026.05 | 28.3 | 28.3 | |
| CISCLLM Backbone=Mistral 7B2026.05 | 28.2 | 28.2 | |
| SC BaselineLLM Backbone=Mistral 7B2026.05 | 27.8 | 27.8 | |
| VecCISC (random)LLM Backbone=Mistral 7B2026.05 | 21.8 | 20.8 |