Coreference Resolution on WSC (test)
82.7AccuracyPromptAgent
Evaluation results
| Method | Links | |
|---|---|---|
| PromptAgentVenue=ICLR 24, Execution Engine=GPT-4o-mini2026.01 | 82.7 | |
| UPAVenue=-, Execution Engine=GPT-4o-mini2026.01 | 82.7 | |
| SPOVenue=EMNLP 26, Execution Engine=GPT-4o-mini2026.01 | 81.1 | |
| APEVenue=ICLR 23, Execution Engine=GPT-4o-mini2026.01 | 80.2 | |
| OPROVenue=ICLR 24, Execution Engine=GPT-4o-mini2026.01 | 80.2 | |
| RaRVenue=arXiv 23, Execution Engine=GPT-4o-mini2026.01 | 79.1 | |
| Step-BackVenue=ICLR 24, Execution Engine=GPT-4o-mini2026.01 | 78.7 | |
| TextGradVenue=Nature 25, Execution Engine=GPT-4o-mini2026.01 | 78 | |
| CoTVenue=NeurIPS 22, Execution Engine=GPT-4o-mini2026.01 | 77.8 | |
| PromptBreederVenue=ICML 24, Execution Engine=GPT-4o-mini2026.01 | 76.7 | |
| IOVenue=-, Execution Engine=GPT-4o-mini2026.01 | 72.4 | |
| ZoVHModel=OPT-13B, N=2, Forward pass budget=10,0002026.05 | 62.18 | |
| ZoVHModel=OPT-1.3B, N=1, Forward pass budget=10,0002026.05 | 60.26 | |
| ZoVHModel=OPT-13B, N=1, Forward pass budget=10,0002026.05 | 59.29 | |
| ZoVHModel=OPT-1.3B, N=2, Forward pass budget=10,0002026.05 | 57.69 | |
| MeZOModel=OPT-1.3B, Forward pass budget=10,0002026.05 | 54.17 | |
| HiZOOModel=OPT-1.3B, Forward pass budget=10,0002026.05 | 54.17 | |
| MeZOModel=OPT-13B, Forward pass budget=10,0002026.05 | 51.28 | |
| HiZOOModel=OPT-13B, Forward pass budget=10,0002026.05 | 46.47 |