Explanation Generation on TrendFact (test)
86.1ECSFactISR(QwQ-32B)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| FactISR(QwQ-32B)Backbone=QwQ-32B2024.10 | 86.1 | 60.1 | 21.75 | 78.35 | 51.56 | 31.07 | 45.92 | |
| QwQ-32BParameters=32B2024.10 | 85.31 | 56.21 | 19.4 | 77.12 | 48.96 | 28.64 | 42.47 | |
| FactISR(Qwen3-32B)Backbone=Qwen3-32B2024.10 | 84.26 | 59.26 | 22.81 | 79.12 | 53.65 | 33.34 | 48.33 | |
| DeepSeek-R1-0528Version=05282024.10 | 83.99 | 56.81 | 20.41 | 77.64 | 50.16 | 29.54 | 43.83 | |
| Qwen3-32B(Think)Reasoning=enabled, Parameters=32B2024.10 | 83.32 | 55.75 | 21.81 | 78.36 | 51.88 | 31.38 | 46.03 | |
| QwQ-32B-PreviewParameters=32B2024.10 | 78.43 | 48.2 | 15.73 | 75.25 | 46.99 | 27.81 | 40.48 | |
| GPT-4.12024.10 | 77.26 | 54.88 | 22.14 | 77.35 | 53.8 | 33.41 | 47.36 | |
| DeepSeek-V3-0324Version=03242024.10 | 76.74 | 54.27 | 23.6 | 79.18 | 54.23 | 34.27 | 48.87 | |
| Qwen3-32B(No think)Reasoning=disabled, Parameters=32B2024.10 | 75.74 | 50.01 | 24.91 | 80.49 | 56.49 | 36.58 | 51.06 | |
| Qwen2.5-72B-instructParameters=72B2024.10 | 72.07 | 50.61 | 26.32 | 80.15 | 56.37 | 37.12 | 50.8 | |
| DeepSeek-R1Evidence condition=Gold Evidence2024.10 | 0.9115 | — | 0.2705 | 0.8143 | 0.5832 | 0.3821 | 0.5188 | |
| o1-previewEvidence condition=Gold Evidence2024.10 | 0.8986 | — | 0.2693 | 0.8022 | 0.5602 | 0.396 | 0.5206 | |
| GPT-4.1Evidence condition=Gold Evidence2024.10 | 0.8622 | — | 0.2958 | 0.827 | 0.6191 | 0.4189 | 0.5561 | |
| QwQ-32B-PreviewEvidence condition=Gold Evidence2024.10 | 0.8198 | — | 0.2093 | 0.7804 | 0.533 | 0.3459 | 0.4669 | |
| DeepSeek-V3Evidence condition=Gold Evidence2024.10 | 0.7812 | — | 0.3573 | 0.8432 | 0.6596 | 0.4805 | 0.6087 | |
| Qwen-72B-instructEvidence condition=Gold Evidence2024.10 | 0.7787 | — | 0.3366 | 0.8364 | 0.6441 | 0.4589 | 0.5906 |