Relation Extraction on MIMICause (test)
82.9Macro F1MedLlama
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MedLlamaCategory=fine-tuning, Size=8B, Method=FT, N examples=19532026.03 | 82.9 | 84.7 | |
| MistralOrcaCategory=fine-tuning, Size=7B, Method=FT, N examples=19532026.03 | 81.7 | 84.3 | |
| gemma-2-9bCategory=fine-tuning, Size=9B, Method=FT, N examples=19532026.03 | 81 | 82.5 | |
| Mistral-7B-v0.1Category=fine-tuning, Size=7B, Method=FT, N examples=19532026.03 | 78 | 81.7 | |
| Llama-3.1-8B-InstructCategory=fine-tuning, Size=8B, Method=FT, N examples=19532026.03 | 75.1 | 78.8 | |
| Llama-2-7bCategory=fine-tuning, Size=7B, Method=FT, N examples=19532026.03 | 74.7 | 78.8 | |
| Clinical-BERT+EntCategory=BL, Size=110M, Method=FT, N examples=19532026.03 | 56 | — | |
| BERT+EntCategory=BL, Size=110M, Method=FT, N examples=19532026.03 | 54 | — | |
| DeepSeek-Qwen-DistillCategory=few-shot, Size=32B, Method=9-shot CoT, N examples=92026.03 | 48.9 | 55.7 | |
| DeepSeek-Qwen-DistillCategory=zero-shot, Size=32B, Method=2-Chain, N examples=-2026.03 | 45 | 60 | |
| DeepSeek-Qwen-DistillCategory=zero-shot, Size=32B, Method=0-shot CoT, N examples=-2026.03 | 38.7 | 54.1 | |
| MistralOrcaCategory=zero-shot, Size=7B, Method=2-Chain, N examples=-2026.03 | 34.8 | 59.1 | |
| MistralOrcaCategory=zero-shot, Size=7B, Method=SumAsk, N examples=-2026.03 | 34.3 | 44.9 | |
| MistralOrcaCategory=few-shot, Size=7B, Method=9-shot CoT, N examples=92026.03 | 27.8 | 40.5 | |
| MistralOrcaCategory=zero-shot, Size=7B, Method=0-shot CoT, N examples=-2026.03 | 27.3 | 34.2 | |
| MedLlamaCategory=zero-shot, Size=8B, Method=2-Chain, N examples=-2026.03 | 21 | 30.4 | |
| MedLlamaCategory=few-shot, Size=8B, Method=9-shot CoT, N examples=92026.03 | 20.3 | 23 | |
| MistralOrcaCategory=few-shot, Size=7B, Method=iCL, N examples=272026.03 | 17.7 | 23.7 | |
| DeepSeek-Qwen-DistillCategory=zero-shot, Size=32B, Method=InstPrompt, N examples=-2026.03 | 17.2 | 26.7 | |
| DeepSeek-Qwen-DistillCategory=zero-shot, Size=32B, Method=SumAsk, N examples=-2026.03 | 16.3 | 33.8 | |
| MistralOrcaCategory=few-shot, Size=7B, Method=iCL, N examples=92026.03 | 7.3 | 8.9 | |
| MistralOrcaCategory=zero-shot, Size=7B, Method=InstPrompt, N examples=-2026.03 | 6.4 | 9.6 | |
| MistralOrcaCategory=few-shot, Size=7B, Method=iCL, N examples=32026.03 | 4.8 | 7.4 | |
| MistralOrcaCategory=few-shot, Size=7B, Method=iCL, N examples=12026.03 | 3.3 | 5.5 |