Legal Reasoning on CaseHOLD (test)
89.22Test AccuracyAutoAdapt
Evaluation Results
| Method | Links | |
|---|---|---|
| AutoAdaptSetting=Template Aware (TA), Technique=SFT2026.03 | 89.22 | |
| LLM2LLM% Data=100, # Seed Examples=45000, # Augmented=58052024.03 | 88.14 | |
| Baseline% Data=100, # Seed Examples=45000, # Augmented=58052024.03 | 87.94 | |
| DS-AgentSetting=Template Aware (TA), Technique=SFT2026.03 | 87.28 | |
| HF DefaultsSetting=Template Aware (TA), Technique=SFT2026.03 | 86.21 | |
| AutoMLAgentSetting=Template Aware (TA), Technique=SFT2026.03 | 84.72 | |
| LLM2LLM% Data=50, # Seed Examples=22500, # Augmented=27092024.03 | 82.92 | |
| MLCopilotSetting=Template Aware (TA), Technique=SFT2026.03 | 82.63 | |
| Baseline% Data=50, # Seed Examples=22500, # Augmented=27092024.03 | 80.39 | |
| LLM2LLM% Data=20, # Seed Examples=9000, # Augmented=11002024.03 | 78.97 | |
| LLM2LLM% Data=10, # Seed Examples=4500, # Augmented=5052024.03 | 78.21 | |
| Baseline% Data=20, # Seed Examples=9000, # Augmented=11002024.03 | 78 | |
| Baseline% Data=10, # Seed Examples=4500, # Augmented=5052024.03 | 77.03 | |
| LLM2LLM% Data=5, # Seed Examples=2250, # Augmented=4232024.03 | 76.83 | |
| LLM2LLM% Data=2, # Seed Examples=900, # Augmented=5802024.03 | 74.97 | |
| Baseline% Data=5, # Seed Examples=2250, # Augmented=4232024.03 | 74.14 | |
| LLM2LLM% Data=1, # Seed Examples=450, # Augmented=7512024.03 | 70.97 | |
| Baseline% Data=2, # Seed Examples=900, # Augmented=5802024.03 | 69.44 | |
| LLM2LLM% Data=0.5, # Seed Examples=225, # Augmented=4902024.03 | 66.5 | |
| Baseline% Data=1, # Seed Examples=450, # Augmented=7512024.03 | 46.25 | |
| Baseline% Data=0.5, # Seed Examples=225, # Augmented=4902024.03 | 33.94 | |
| Baseline% Data=0, # Seed Examples=0, # Augmented=02024.03 | 12.28 |