Scientific Question Answering on Scientific Disciplines In-Domain
64.9Chemistry AccuracyFT (tuned)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| FT (tuned)Param. (%)=100, GPU Hours=20802026.02 | 64.9 | 58.8 | 82.1 | 83.1 | 74.3 | 72.6 | 7.376 | |
| FT (ori.)Param. (%)=100, GPU Hours=19202026.02 | 61.9 | 54.5 | 64.6 | 83 | 70.7 | 66.9 | -1.135 | |
| LoRA-MoE (tuned)Param. (%)=0.902, GPU Hours=3282026.02 | 61.3 | 60.3 | 82.3 | 83.9 | 76 | 72.8 | 7.236 | |
| LoRA-MoE (ori.)Param. (%)=1.315, GPU Hours=3442026.02 | 53.5 | 55 | 68.3 | 84.7 | 74.1 | 67.1 | -1.78 | |
| Discipline LoRAParam. (%)=0.264 × 5, GPU Hours=-2026.02 | 53 | 58.1 | 71.3 | 84.5 | 74.5 | 68.3 | 0 | |
| QWen 2.5 7B InstructParam. (%)=0, GPU Hours=-2026.02 | 50.9 | 49.3 | 63 | 77 | 71.5 | 62.3 | — |