Procedure Generation on Reaction-Procedure dataset (test)
72BLEU-2QFANG-8B
Evaluation Results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| QFANG-8BModel scale (parameters)=8B, Optimization strategy (SFT/RL)=RL, In-context learning (shots)=02025.12 | 72 | 61.3 | 57.4 | 0.3 | 4.3 | 78.8 | 72.1 | 54.5 | 61.1 | 69.7 | 70.9 | |
| QFANG-32BModel scale (parameters)=32B, Optimization strategy (SFT/RL)=SFT, In-context learning (shots)=02025.12 | 70.2 | 59.7 | 57.2 | 0.1 | 3.2 | 78.8 | 71.5 | 53.3 | 60.2 | 68.8 | 70.1 | |
| QFANG-8BModel scale (parameters)=8B, Optimization strategy (SFT/RL)=SFT, In-context learning (shots)=02025.12 | 69.9 | 59.4 | 56.8 | 0.1 | 2.8 | 77.9 | 71.2 | 52.9 | 59.8 | 68.6 | 69.4 | |
| GPT-5 (high)In-context learning (shots)=12025.12 | 67.1 | 56.4 | 51.5 | 0.8 | 2.9 | 54.1 | 67.6 | 49.4 | 55.3 | 64.7 | 59.9 | |
| GPT-5 (high)In-context learning (shots)=32025.12 | 65 | 54.4 | 51.7 | 0.6 | 2.6 | 55.4 | 68.5 | 50.1 | 55.9 | 66.9 | 61.1 | |
| o4-mini (high)In-context learning (shots)=32025.12 | 56.8 | 45.5 | 44.7 | 0 | 0.3 | 24 | 62.9 | 39.9 | 49.2 | 61.5 | 26.2 | |
| o4-mini (high)In-context learning (shots)=12025.12 | 55.2 | 43.8 | 42.7 | 0 | 0.1 | 17.6 | 60.4 | 36.9 | 46.6 | 58.9 | 26.6 | |
| Nearest NeighborIn-context learning (shots)=0, Fingerprint=DRFP2025.12 | 54.5 | 39.1 | 46.6 | 1 | 2.4 | 30.2 | 60 | 38.4 | 47.3 | 50 | 65.1 | |
| GPT-4oIn-context learning (shots)=32025.12 | 41.5 | 31.8 | 32.7 | 0 | 0 | 2.4 | 53.1 | 30.1 | 39.4 | 57.4 | 8.4 | |
| GPT-4oIn-context learning (shots)=12025.12 | 39.6 | 29.9 | 31.3 | 0 | 0 | 1.4 | 50.4 | 26.2 | 36.5 | 55.1 | 6.1 |