Logical Reasoning on ProsQA
98.5AccuracyTHOUGHTSTEER
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| THOUGHTSTEERArchitecture=COCONUT (GPT-2 124M), Poison rate (rho)=15%2026.04 | 98.5 | — | — | 99.5 | 100 | |
| OneLatentNumber of latent tokens=12026.02 | 97.8 | 11.1 | 8.81 | — | — | |
| COCONUTNumber of latent tokens=62026.02 | 89.2 | 6.6 | 13.5 | — | — | |
| iCoTNumber of latent tokens=02026.02 | 87.4 | 12.8 | 6.82 | — | — | |
| CoTNumber of latent tokens=02026.02 | 76.2 | 2.29 | 33.3 | — | — | |
| No CoTNumber of latent tokens=02026.02 | 60.4 | 7.06 | 8.55 | — | — |