Clinical Trial Eligibility Classification on N2C2 2018 (test)
89.05Micro F1 ScoreMedGemma 27b-it
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| MedGemma 27b-itFinetune Strategy=LoRA, Token Limit=8192, Input Strategy=RAG, Embedding Model=BAAI-large2026.04 | 89.05 | 86.02 | 88.45 | 89.22 | |
| MedGemma 27b-itFinetune Strategy=LoRA, Token Limit=8192, Input Strategy=RAG, Embedding Model=BAAI-small2026.04 | 88.8 | 85.15 | 88.89 | 89.06 | |
| MedGemma 27b-itFinetune Strategy=LoRA, Token Limit=8192, Input Strategy=ORI2026.04 | 88.49 | 83.37 | 90.63 | 89.02 | |
| Llama 3.1 8b-itFinetune Strategy=LoRA, Token Limit=8192, Input Strategy=ORI2026.04 | 86.71 | 80.51 | 89.98 | 87.4 | |
| MedGemma 27b-itFinetune Strategy=LoRA, Token Limit=8192, Input Strategy=NER2026.04 | 86.29 | 82.84 | 85.19 | 86.45 | |
| Llama 3.1 8b-itFinetune Strategy=LoRA, Token Limit=8192, Input Strategy=RAG, Embedding Model=BAAI-large2026.04 | 85.4 | 82.65 | 83.01 | 85.42 | |
| Llama 3.1 8b-itFinetune Strategy=LoRA, Token Limit=8192, Input Strategy=RAG, Embedding Model=BAAI-small2026.04 | 85.33 | 77.55 | 91.07 | 86.34 | |
| Llama 3.1 8b-itFinetune Strategy=LoRA, Token Limit=8192, Input Strategy=NER2026.04 | 82.93 | 75.91 | 86.49 | 83.69 | |
| GPT-oss 20bFinetune Strategy=LoRA, Token Limit=8192, Input Strategy=RAG, Embedding Model=BAAI-small2026.04 | 81.72 | 77.2 | 80.39 | 81.93 | |
| GPT-oss20bFinetune Strategy=LoRA, Token Limit=8192, Input Strategy=RAG, Embedding Model=BAAI-large2026.04 | 81.17 | 76.57 | 79.74 | 81.37 | |
| GPT-oss20bFinetune Strategy=LoRA, Token Limit=8192, Input Strategy=ORI2026.04 | 79.1 | 74.68 | 76.47 | 79.21 | |
| GPT-oss20bFinetune Strategy=LoRA, Token Limit=8192, Input Strategy=NER2026.04 | 74.53 | 69.2 | 71.46 | 74.65 | |
| GatorTron-2kFinetune Strategy=Full Finetune, Token Limit=2048, Input Strategy=NER2026.04 | 74.23 | 69 | 70.81 | 74.33 | |
| BERTFinetune Strategy=Full Finetune, Token Limit=512, Input Strategy=NER2026.04 | 72.4 | 66.32 | 69.93 | 72.6 |