Multitask Language Understanding on MMLU (Acc, AUROC, ECE)
71.4AccuracyC3RL (Ours)
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| C3RL (Ours)Model family=Qwen2.5VL-7B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 71.4 | 0.653 | 0.092 | |
| C3RL w/o RefModel family=Qwen2.5VL-7B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 71.2 | 0.648 | 0.111 | |
| RLVRModel family=Qwen2.5VL-7B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 71.1 | 0.602 | 0.239 | |
| RLCRModel family=Qwen2.5VL-7B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 70.7 | 0.607 | 0.175 | |
| SCModel family=Qwen2.5VL-7B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=12026.07 | 69.9 | 0.759 | 0.141 | |
| BaseModel family=Qwen2.5VL-7B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 68.3 | 0.587 | 0.188 | |
| SaySelfModel family=Qwen2.5VL-7B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 64 | 0.731 | 0.129 | |
| C3RL (Ours)Model family=Llama-3.2-3B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 64 | 0.68 | 0.089 | |
| C3RL w/o RefModel family=Llama-3.2-3B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 63.5 | 0.688 | 0.087 | |
| RLVRModel family=Llama-3.2-3B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 63.4 | 0.537 | 0.208 | |
| RLCRModel family=Llama-3.2-3B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 62.3 | 0.67 | 0.153 | |
| SFT+RefModel family=Qwen2.5VL-7B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 62.1 | 0.706 | 0.098 | |
| SCModel family=Llama-3.2-3B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=12026.07 | 59.9 | 0.733 | 0.158 | |
| BaseModel family=Llama-3.2-3B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 59.6 | 0.586 | 0.271 | |
| SFT+RefModel family=Llama-3.2-3B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 52.8 | 0.628 | 0.162 | |
| SaySelfModel family=Llama-3.2-3B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 37.4 | 0.715 | 0.11 |