Coreference Resolution on WSC (Acc@1 & Avg. Perf.)
85.2Accuracy@1DisambiguSLM
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DisambiguSLMLLM=GPT, Avg. Cost ($)=0.022026.04 | 85.2 | 68.6 | |
| DisambiguSLMLLM=LLaMA, Avg. Cost ($)=0.022026.04 | 83.8 | 66 | |
| DisambiguSLMLLM=DeepSeek, Avg. Cost ($)=0.022026.04 | 83 | 66.3 | |
| Prompt AgentLLM=GPT, Avg. Cost ($)=2.712026.04 | 82.7 | 65 | |
| SPOLLM=GPT, Avg. Cost ($)=0.152026.04 | 82 | 66.9 | |
| Prompt AgentLLM=DeepSeek, Avg. Cost ($)=2.712026.04 | 80.3 | 62.7 | |
| APELLM=GPT, Avg. Cost ($)=9.072026.04 | 80.2 | 64.8 | |
| OPROLLM=GPT, Avg. Cost ($)=4.512026.04 | 80.2 | 66.6 | |
| SPOLLM=DeepSeek, Avg. Cost ($)=0.152026.04 | 80.1 | 64.8 | |
| SPOLLM=LLaMA, Avg. Cost ($)=0.152026.04 | 79.6 | 64 | |
| Prompt AgentLLM=LLaMA, Avg. Cost ($)=2.712026.04 | 79.4 | 62 | |
| RephraseLLM=GPT, Avg. Cost ($)=-2026.04 | 79.1 | 61.1 | |
| Step-backLLM=GPT, Avg. Cost ($)=-2026.04 | 78.7 | 65 | |
| TextGradLLM=GPT, Avg. Cost ($)=13.142026.04 | 78 | 63.9 | |
| OPROLLM=DeepSeek, Avg. Cost ($)=4.512026.04 | 77.9 | 63.9 | |
| CoTLLM=GPT, Avg. Cost ($)=-2026.04 | 77.8 | 63.8 | |
| APELLM=DeepSeek, Avg. Cost ($)=9.072026.04 | 77.3 | 62.4 | |
| OPROLLM=LLaMA, Avg. Cost ($)=4.512026.04 | 76.8 | 63.1 | |
| Prompt BreederLLM=GPT, Avg. Cost ($)=4.822026.04 | 76.7 | 64.5 | |
| APELLM=LLaMA, Avg. Cost ($)=9.072026.04 | 76.1 | 61.7 | |
| TextGradLLM=DeepSeek, Avg. Cost ($)=13.142026.04 | 75.9 | 61.6 | |
| RephraseLLM=DeepSeek, Avg. Cost ($)=-2026.04 | 75.6 | 58.8 | |
| Step-backLLM=DeepSeek, Avg. Cost ($)=-2026.04 | 75.1 | 62.3 | |
| TextGradLLM=LLaMA, Avg. Cost ($)=13.142026.04 | 75 | 60.9 | |
| CoTLLM=DeepSeek, Avg. Cost ($)=-2026.04 | 74.9 | 61.2 | |
| Prompt BreederLLM=DeepSeek, Avg. Cost ($)=4.822026.04 | 74.8 | 62.2 | |
| RephraseLLM=LLaMA, Avg. Cost ($)=-2026.04 | 74.2 | 57.93 | |
| Prompt BreederLLM=LLaMA, Avg. Cost ($)=4.822026.04 | 74.1 | 61.4 | |
| Step-backLLM=LLaMA, Avg. Cost ($)=-2026.04 | 74 | 61.4 | |
| CoTLLM=LLaMA, Avg. Cost ($)=-2026.04 | 73.9 | 60.4 | |
| Naïve promptingLLM=GPT, Avg. Cost ($)=-2026.04 | 72.4 | 61.6 | |
| Naïve promptingLLM=DeepSeek, Avg. Cost ($)=-2026.04 | 69.7 | 59.2 | |
| Naïve promptingLLM=LLaMA, Avg. Cost ($)=-2026.04 | 68.4 | 58.2 |