Physics-aware symbolic simulation on PhysCodeBench (test)
33.5Code-based ScoreSMRF + SFT + DPO
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SMRF + SFT + DPOFramework=Multi-Agent Framework (SMRF), Configuration=+ SFT + DPO2026.04 | 33.5 | 34.2 | 67.7 | |
| SMRF + SFTFramework=Multi-Agent Framework (SMRF), Configuration=+ SFT2026.04 | 27.3 | 28.4 | 55.7 | |
| DRDQ-32B + SFT + DPOModel Category=Single-Agent Fine-tuned, Tuning=SFT + DPO2026.04 | 18.7 | 19.2 | 37.9 | |
| DRDQ-32B + SFTModel Category=Single-Agent Fine-tuned, Tuning=SFT2026.04 | 17.5 | 18.4 | 35.9 | |
| Claude-3.5-SonnetModel Category=Vanilla Proprietary Models, Protocol=Zero-shot2026.04 | 17.2 | 19.1 | 36.3 | |
| SMRF BaseFramework=Multi-Agent Framework (SMRF), Configuration=vanilla agents2026.04 | 16.4 | 17.7 | 34.1 | |
| GPT-4oModel Category=Vanilla Proprietary Models, Protocol=Zero-shot2026.04 | 16 | 18.3 | 34.3 | |
| Gemini-2.0-ProModel Category=Vanilla Proprietary Models, Protocol=Zero-shot2026.04 | 15 | 17 | 32 | |
| DeepSeek-R1Model Category=Vanilla Open-source Models, Protocol=Zero-shot2026.04 | 14 | 15.8 | 29.8 | |
| DRDQ-32B (vanilla)Model Category=Vanilla Open-source Models, Protocol=Zero-shot, Full Name=DeepSeek-R1-Distill-Qwen-32B2026.04 | 12.2 | 15.8 | 28 | |
| QwQ-32BModel Category=Vanilla Open-source Models, Protocol=Zero-shot2026.04 | 6.8 | 9 | 15.8 | |
| Qwen-2.5-32BModel Category=Vanilla Open-source Models, Protocol=Zero-shot2026.04 | 0.7 | 1.1 | 1.8 |