PMI ranking estimation on ChaosNLI (500 held-out pairs)
0.82Spearman RhoPromptNCE
Evaluation Results
| Method | Links | |
|---|---|---|
| PromptNCEModel=Claude Sonnet 4, Evaluation Type=Zero-shot2026.05 | 0.82 | |
| PromptNCE (emp.)Model=Claude Sonnet 4, Evaluation Type=Empirical marginal (oracle upper bound)2026.05 | 0.81 | |
| Decomposed PMIModel=Claude Sonnet 4, Evaluation Type=Zero-shot2026.05 | 0.75 | |
| InfoNCE (emp.)Model=Claude Sonnet 4, Evaluation Type=Empirical marginal (oracle upper bound)2026.05 | 0.73 | |
| PromptNCE (emp.)Model=GPT-5.2, Evaluation Type=Empirical marginal (oracle upper bound)2026.05 | 0.73 | |
| InfoNCEModel=Claude Sonnet 4, Evaluation Type=Zero-shot2026.05 | 0.73 | |
| MarginalNCEModel=Claude Sonnet 4, Evaluation Type=Zero-shot2026.05 | 0.73 | |
| PromptNCEModel=GPT-5.2, Evaluation Type=Zero-shot2026.05 | 0.73 | |
| Direct PMIModel=Claude Sonnet 4, Evaluation Type=Zero-shot2026.05 | 0.72 | |
| Decomposed PMI (emp.)Model=Claude Sonnet 4, Evaluation Type=Empirical marginal (oracle upper bound)2026.05 | 0.7 | |
| InfoNCE (emp.)Model=GPT-5.2, Evaluation Type=Empirical marginal (oracle upper bound)2026.05 | 0.7 | |
| Decomposed PMIModel=GPT-5.2, Evaluation Type=Zero-shot2026.05 | 0.7 | |
| Decomposed PMI (emp.)Model=GPT-5.2, Evaluation Type=Empirical marginal (oracle upper bound)2026.05 | 0.69 | |
| InfoNCEModel=GPT-5.2, Evaluation Type=Zero-shot2026.05 | 0.69 | |
| MarginalNCEModel=GPT-5.2, Evaluation Type=Zero-shot2026.05 | 0.69 | |
| Direct PMIModel=GPT-5.2, Evaluation Type=Zero-shot2026.05 | 0.49 |