Counterfactual Reasoning on Counterfactual Eval (dev)
63.4Mean ScorePE2
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| PE2Initialization=Induction, Sub-task=syntax_svo2023.11 | 63.4 | 7.74 | |
| PE2Initialization=Induction, Sub-task=chess_original2023.11 | 61.6 | 3.01 | |
| APOInitialization=Induction, Sub-task=chess_original2023.11 | 59.2 | 2.04 | |
| PE2Initialization=Induction, Sub-task=arithmetic_base162023.11 | 58 | 3.1 | |
| APOInitialization=Induction, Sub-task=syntax_svo2023.11 | 57.8 | 7.88 | |
| APEInitialization=Induction, Sub-task=chess_original2023.11 | 57.6 | 4.13 | |
| Iter. APEInitialization=Induction, Sub-task=chess_original2023.11 | 56 | 3.52 | |
| PE2Initialization=Induction, Sub-task=syntax_sov2023.11 | 55.6 | 4.8 | |
| APOInitialization=Induction, Sub-task=arithmetic_base162023.11 | 55.4 | 5.46 | |
| Iter. APEInitialization=Induction, Sub-task=syntax_svo2023.11 | 49.8 | 9.24 | |
| PE2Initialization=Induction, Sub-task=chess_cf2023.11 | 49.6 | 5.57 | |
| APEInitialization=Induction, Sub-task=syntax_svo2023.11 | 46.2 | 8.57 | |
| APOInitialization=Induction, Sub-task=syntax_sov2023.11 | 46.2 | 7.47 | |
| PE2Initialization=Induction, Sub-task=syntax_ovs2023.11 | 45.2 | 5.91 | |
| PE2Initialization=Induction, Sub-task=12-task average2023.11 | 44.93 | — | |
| Iter. APEInitialization=Induction, Sub-task=syntax_sov2023.11 | 44.6 | 3.14 | |
| PE2Initialization=Induction, Sub-task=syntax_vos2023.11 | 44 | 10.45 | |
| APOInitialization=Induction, Sub-task=chess_cf2023.11 | 43 | 4.15 | |
| PE2Initialization=Induction, Sub-task=syntax_vso2023.11 | 42.4 | 7.42 | |
| PE2Initialization=Induction, Sub-task=syntax_osv2023.11 | 41.8 | 4.79 | |
| Iter. APEInitialization=Induction, Sub-task=chess_cf2023.11 | 40.2 | 3.12 | |
| APEInitialization=Induction, Sub-task=chess_cf2023.11 | 40 | 3.58 | |
| APOInitialization=Induction, Sub-task=arithmetic_base82023.11 | 38.8 | 2.56 | |
| APEInitialization=Induction, Sub-task=syntax_sov2023.11 | 38.2 | 8.28 | |
| APOInitialization=Induction, Sub-task=12-task average2023.11 | 38.03 | — | |
| PE2Initialization=Induction, Sub-task=arithmetic_base82023.11 | 37.8 | 3.54 | |
| APOInitialization=Induction, Sub-task=syntax_vso2023.11 | 36.8 | 11.41 | |
| APEInitialization=Induction, Sub-task=arithmetic_base162023.11 | 34.4 | 10.61 | |
| APOInitialization=Induction, Sub-task=syntax_ovs2023.11 | 33.8 | 6.01 | |
| APEInitialization=Induction, Sub-task=syntax_vos2023.11 | 31.6 | 14.37 | |
| Iter. APEInitialization=Induction, Sub-task=arithmetic_base162023.11 | 30.6 | 6.09 | |
| Iter. APEInitialization=Induction, Sub-task=syntax_vos2023.11 | 30.6 | 13.92 | |
| APEInitialization=Induction, Sub-task=syntax_ovs2023.11 | 30 | 10.58 | |
| Iter. APEInitialization=Induction, Sub-task=syntax_ovs2023.11 | 30 | 10.58 | |
| Iter. APEInitialization=Induction, Sub-task=12-task average2023.11 | 29.52 | — | |
| PE2Initialization=Induction, Sub-task=arithmetic_base92023.11 | 29.4 | 15.32 | |
| APEInitialization=Induction, Sub-task=12-task average2023.11 | 29.02 | — | |
| APOInitialization=Induction, Sub-task=syntax_vos2023.11 | 28.8 | 9.13 | |
| APOInitialization=Induction, Sub-task=syntax_osv2023.11 | 27.8 | 11.02 | |
| APEInitialization=Induction, Sub-task=syntax_vso2023.11 | 26 | 11.35 | |
| Iter. APEInitialization=Induction, Sub-task=syntax_vso2023.11 | 24.4 | 11.64 | |
| APOInitialization=Induction, Sub-task=arithmetic_base92023.11 | 23.8 | 19.74 | |
| Iter. APEInitialization=Induction, Sub-task=arithmetic_base82023.11 | 20.2 | 16.57 | |
| APEInitialization=Induction, Sub-task=arithmetic_base82023.11 | 19 | 15.53 | |
| Iter. APEInitialization=Induction, Sub-task=syntax_osv2023.11 | 13.6 | 10.71 | |
| Iter. APEInitialization=Induction, Sub-task=arithmetic_base92023.11 | 12 | 16.84 | |
| APEInitialization=Induction, Sub-task=syntax_osv2023.11 | 11.6 | 8.89 | |
| APEInitialization=Induction, Sub-task=arithmetic_base92023.11 | 11.2 | 15.38 | |
| PE2Initialization=Induction, Sub-task=arithmetic_base112023.11 | 10.4 | 5.5 | |
| APOInitialization=Induction, Sub-task=arithmetic_base112023.11 | 5 | 2.19 | |
| APEInitialization=Induction, Sub-task=arithmetic_base112023.11 | 2.4 | 1.5 | |
| Iter. APEInitialization=Induction, Sub-task=arithmetic_base112023.11 | 2.2 | 1.33 |