Foundation Model Evaluation on AGIEval
58.1AccuracyRLVR
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| RLVRModel family=Qwen2.5VL-7B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 58.1 | 0.534 | 37.7 | |
| C3RL w/o RefModel family=Qwen2.5VL-7B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 58.1 | 0.682 | 19.7 | |
| C3RL (Ours)Model family=Qwen2.5VL-7B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 57.8 | 0.688 | 15.4 | |
| RLCRModel family=Qwen2.5VL-7B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 57.5 | 0.664 | 20.5 | |
| SCModel family=Qwen2.5VL-7B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=12026.07 | 53.7 | 0.711 | 20.8 | |
| BaseModel family=Qwen2.5VL-7B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 51 | 0.543 | 37.5 | |
| SaySelfModel family=Qwen2.5VL-7B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 50.8 | 0.745 | 14.7 | |
| SFT+RefModel family=Qwen2.5VL-7B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 49.1 | 0.753 | 10.5 | |
| RLVRModel family=Llama-3.2-3B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 37.3 | 0.522 | 46.8 | |
| C3RL (Ours)Model family=Llama-3.2-3B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 37.3 | 0.66 | 24.6 | |
| C3RL w/o RefModel family=Llama-3.2-3B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 37.2 | 0.652 | 21.8 | |
| BaseModel family=Llama-3.2-3B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 34.8 | 0.557 | 42.3 | |
| RLCRModel family=Llama-3.2-3B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 34.4 | 0.658 | 11.1 | |
| SFT+RefModel family=Llama-3.2-3B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 32.1 | 0.594 | 24 | |
| SCModel family=Llama-3.2-3B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=12026.07 | 28.6 | 0.644 | 35.2 | |
| SaySelfModel family=Llama-3.2-3B-Instruct, Decoding temperature=0.7, top_k=-1, top_p=1, Seed=0, 42, 20252026.07 | 19.4 | 0.611 | 7.2 |