Counterfactual Reasoning on MCQA (test)
90.75AP (Exact Counterfactual Accuracy)PLOT-pca-DAS
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| PLOT-pca-DASAP layer/dim=L18/32, AT layer/dim=L24/1202026.05 | 90.75 | 82.25 | 86.5 | 455.2 | |
| Full DASAP layer/dim=L17/96, AT layer/dim=L24/23042026.05 | 89.5 | 96.8 | 93.1 | 13,107.6 | |
| PLOT-DASAP layer/dim=L18/128, AT layer/dim=L24/23042026.05 | 88.25 | 98 | 93.13 | 1,148.6 | |
| PLOT-native-DASAP layer/dim=L18/1152, AT layer/dim=L24/23042026.05 | 85.75 | 98 | 91.87 | 346.1 | |
| PLOT-pcaAP layer/dim=L18/8, AT layer/dim=L24/962026.05 | 85.5 | 88.5 | 87 | 188 | |
| PLOT-nativeAP layer/dim=L18/768, AT layer/dim=L24/5762026.05 | 83.75 | 98.5 | 91.13 | 204.6 |