Definition Extraction on DefExtra (test)
0.231Avg NLIMultiStepFewShotExtractor
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MultiStepFewShotExtractorAggregation=Average (test)2026.02 | 0.231 | — | — | |
| MultiStepExtractorAggregation=Average (test)2026.02 | 0.223 | — | — | |
| DSPyPaperExtractorAggregation=Average (test)2026.02 | 0.216 | — | — | |
| OneStepExtractorAggregation=Average (test)2026.02 | 0.159 | — | — | |
| OneStepFewShotExtractorAggregation=Average (test)2026.02 | 0.15 | — | — | |
| DSPyPaperExtractorModel=Qwen3-30B-A3B-Thinking-2507, Chunking=section2026.02 | — | 69.7 | 0.397 | |
| MultiStepExtractorModel=Magistral-Small-2509, Chunking=paragraph2026.02 | — | 57.6 | 0.341 | |
| MultiStepFewShotExtractorModel=Magistral-Small-2509, Chunking=paragraph2026.02 | — | 66.7 | 0.382 | |
| OneStepExtractorModel=gpt-5-nano, Chunking=section2026.02 | — | 77.3 | 0.274 | |
| OneStepFewShotExtractorModel=deepseek-chat-v3-0324, Chunking=section2026.02 | — | 86.4 | 0.258 |