Scientific Question Answering on GPQA (pass@1, pass@256)
18.2pass@1CISPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CISPOBackbone=Llama-3.2-3B-Instruct, Group Size (N)=82026.02 | 18.2 | 49.7 | |
| GRPOBackbone=Llama-3.2-3B-Instruct, Group Size (N)=82026.02 | 17.5 | 55.1 | |
| F-DAPOBackbone=Llama-3.2-3B-Instruct, Group Size (N)=82026.02 | 17 | 53.7 | |
| F-CISPOBackbone=Llama-3.2-3B-Instruct, Group Size (N)=82026.02 | 17 | 47.7 | |
| DAPOBackbone=Llama-3.2-3B-Instruct, Group Size (N)=82026.02 | 15.7 | 47 | |
| DAPOBackbone=Qwen2.5-7B, Group Size (N)=82026.02 | 15.4 | 54.9 | |
| F-CISPOBackbone=Qwen2.5-7B, Group Size (N)=82026.02 | 15.4 | 67.1 | |
| F-GRPOBackbone=Llama-3.2-3B-Instruct, Group Size (N)=82026.02 | 15.2 | 57.6 | |
| F-GRPOBackbone=Qwen2.5-7B, Group Size (N)=82026.02 | 15 | 57.3 | |
| F-DAPOBackbone=Qwen2.5-7B, Group Size (N)=82026.02 | 15 | 57.4 | |
| CISPOBackbone=Qwen2.5-7B, Group Size (N)=82026.02 | 12.6 | 55.5 | |
| F-CISPOBackbone=Qwen2.5-1.5B-Math, Group Size (N)=82026.02 | 12 | 64.2 | |
| GRPOBackbone=Qwen2.5-7B, Group Size (N)=82026.02 | 11.3 | 46.2 | |
| F-DAPOBackbone=Qwen2.5-1.5B-Math, Group Size (N)=82026.02 | 9.2 | 60.7 | |
| F-GRPOBackbone=Qwen2.5-1.5B-Math, Group Size (N)=82026.02 | 8.8 | 56.5 | |
| DAPOBackbone=Qwen2.5-1.5B-Math, Group Size (N)=82026.02 | 8.6 | 59.4 | |
| CISPOBackbone=Qwen2.5-1.5B-Math, Group Size (N)=82026.02 | 7.4 | 48.4 | |
| GRPOBackbone=Qwen2.5-1.5B-Math, Group Size (N)=82026.02 | 6.6 | 50.1 |