Natural Language Inference on ContractNLI re-annotated (test)
83AccuracyClaude
Evaluation results
| Method | Links | |
|---|---|---|
| ClaudeClassification Approach=LLM-Formal Reasoning2026.06 | 83 | |
| ClaudeClassification Approach=Solver-Formal Reasoning2026.06 | 74.5 | |
| QwenClassification Approach=LLM-Formal Reasoning2026.06 | 69.1 | |
| DeepseekClassification Approach=Simple2026.06 | 68.4 | |
| GPTClassification Approach=Simple2026.06 | 66 | |
| GPTClassification Approach=LLM-Formal Reasoning2026.06 | 65.3 | |
| QwenClassification Approach=Simple2026.06 | 65.1 | |
| DeepseekClassification Approach=LLM-Formal Reasoning2026.06 | 64.2 | |
| ClaudeClassification Approach=Simple2026.06 | 63.1 | |
| DeepseekClassification Approach=Structured2026.06 | 62.2 | |
| QwenClassification Approach=Structured2026.06 | 61.5 | |
| GPTClassification Approach=Solver-Formal Reasoning2026.06 | 60.3 | |
| GPTClassification Approach=Structured2026.06 | 58.8 | |
| LlamaClassification Approach=LLM-Formal Reasoning2026.06 | 58.6 | |
| LlamaClassification Approach=Simple2026.06 | 55.9 | |
| ClaudeClassification Approach=Structured2026.06 | 53.2 | |
| QwenClassification Approach=Solver-Formal Reasoning2026.06 | 48.3 | |
| DeepseekClassification Approach=Solver-Formal Reasoning2026.06 | 45.3 | |
| LlamaClassification Approach=Solver-Formal Reasoning2026.06 | 42.1 | |
| LlamaClassification Approach=Structured2026.06 | 40 |