Logical Reasoning on BBH-7
100Boolean Expressions AccuracySPEAR
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SPEARBootstrap resampling=N=10002026.05 | 100 | — | — | 76.6 | 98.4 | 98.4 | 95.2 | 100 | 88 | 93.8 | 97.3 | 91.2 | |
| TextGradBootstrap resampling=N=10002026.05 | 99.2 | — | — | 72.3 | 0.8 | 5.6 | 19.2 | 99.2 | 42.4 | 48.4 | 8.5 | 78.3 | |
| GEPABootstrap resampling=N=10002026.05 | 97.6 | — | — | 71.3 | 12.8 | 2.4 | 72.8 | 98.4 | 84 | 62.8 | 29.3 | 87.9 | |
| SeedBootstrap resampling=N=10002026.05 | 66.4 | — | — | 63.8 | 0 | 2.4 | 0 | 99.2 | 0 | 33.1 | 0.8 | 57.4 |