Medical Jargon Extraction on MIMIC medical jargon extraction IV

36.6Top-3 F1 ScoreMistral7B-FT

Evaluation Results

Method
Mistral7B-FT2025.02	36.6	0.677	37	0.692	38.8	0.715
GPT-4 Turbo2025.02	36.2	0.673	39.8	0.705	42.4	0.691
Mistral7B2025.02	35.9	0.669	37.5	0.677	40.2	0.694
Llama3.1-8B2025.02	35.8	0.585	37.6	0.588	41.3	0.611
Claude 3.0 Haiku2025.02	35.7	0.668	36.4	0.715	34.9	0.693
Llama3.1-8B-MIMIC-FT2025.02	35.5	0.605	36.9	0.596	40	0.614
GPT-3.5 Turbo2025.02	35.4	0.661	36.7	0.633	39.8	0.661
Claude 3.0 Sonnet2025.02	35.4	0.683	35.7	0.684	37.1	0.692
BioMistral7B-FT2025.02	35.3	0.563	36	0.563	26.4	0.483
Mistral7B-MIMIC-FT2025.02	35.1	0.681	36.7	0.674	38.9	0.676
GPT-3.5 Turbo2025.02	34.3	0.649	35.9	0.655	37.8	0.669
Llama3.1-8B-FT2025.02	34.3	0.605	37.3	0.593	40.4	0.621
Mistral7B-FT2025.02	34.2	0.603	35	0.631	33.2	0.661
Claude 3.0 Opus2025.02	34	0.707	36.5	0.698	39	0.713
Mistral7B-MIMIC-FT2025.02	34	0.746	35.8	0.731	38	0.713
Mistral7B-MIMIC-FT2025.02	33.6	0.524	33.9	0.508	39	0.542
Llama3.1-8B-FT2025.02	33.2	0.589	35.3	0.63	38.6	0.649
GPT-4o Mini2025.02	32.9	0.664	36.9	0.7	38.3	0.708
GPT-4o Mini2025.02	32.8	0.666	36.3	0.668	38.1	0.684
Mistral7B-MIMIC-FT2025.02	32.8	0.539	33.6	0.512	34	0.488
Llama3.1-8B2025.02	32.5	0.589	29.7	0.437	35.4	0.594
GPT-4 Turbo2025.02	32.4	0.6	36.8	0.626	43.3	0.621
GPT-4o Mini2025.02	32	0.617	35.5	0.667	39.2	0.691
Claude 3.0 Haiku2025.02	32	0.672	35.8	0.69	38.7	0.683
Claude 3.0 Opus2025.02	31.9	0.646	33.1	0.551	38.7	0.68
GPT-4 Turbo2025.02	31.7	0.643	34.7	0.668	37	0.691
Claude 3.0 Opus2025.02	31.6	0.615	35.9	0.661	41.2	0.709
Llama3.1-8B-MIMIC-FT2025.02	31.6	0.591	34.6	0.597	38.1	0.637
Llama3.1-8B2025.02	31.3	0.585	37	0.644	39.9	0.652
Llama3.1-8B-FT2025.02	31.2	0.589	36	0.62	40.3	0.624
Llama3.1-8B-MIMIC-FT2025.02	31.1	0.597	36.3	0.624	39.9	0.643
Claude 3.0 Sonnet2025.02	30.8	0.596	34.5	0.598	39.9	0.639
GPT-4 Turbo2025.02	30.6	0.562	35.9	0.601	43.3	0.64
Claude 3.0 Sonnet2025.02	30.4	0.666	31.7	0.673	32.3	0.668
GPT-4o Mini2025.02	30.3	0.62	32.9	0.617	33.5	0.61
GPT-3.5 Turbo2025.02	30.3	0.575	32.9	0.573	34.3	0.572
Mistral7B2025.02	30.3	0.582	34.8	0.579	39.4	0.565
Llama3.1-8B-MIMIC-FT2025.02	29.6	0.568	30.4	0.566	29.5	0.557
Mistral7B2025.02	29.3	0.527	33.7	0.563	36.3	0.557
Mistral7B-FT2025.02	29.1	0.557	31	0.57	32.1	0.544
Claude 3.0 Sonnet2025.02	28.9	0.554	32.8	0.556	35.5	0.557
Claude 3.0 Opus2025.02	28.9	0.617	34.4	0.643	38.7	0.664
Llama3.1-8B-FT2025.02	28.9	0.558	30.1	0.571	29.4	0.575
Mistral7B-FT2025.02	28.7	0.418	32.5	0.536	35.5	0.603
BioMistral7B-FT2025.02	28.7	0.436	36.1	0.542	30.6	0.594
Biomistral7B-MIMIC-FT2025.02	28.7	0.549	30.6	0.568	33.2	0.569
Claude 3.0 Haiku2025.02	28.4	0.613	32	0.629	34.5	0.62
Biomistral7B-MIMIC-FT2025.02	28.3	0.551	30.4	0.544	34.8	0.565
Llama3.1-8B2025.02	28.2	0.572	29.6	0.569	29.9	0.576
Biomistral7B-MIMIC-FT2025.02	27.9	0.499	30.6	0.49	35.1	0.5
BioMistral7B-FT2025.02	27.3	0.46	28.6	0.462	31.1	0.468
BioMistral7B2025.02	26.2	0.5	32.1	0.567	28.8	0.529
GPT-3.5 Turbo2025.02	26.1	0.478	30.8	0.582	32.4	0.602
Biomistral7B-MIMIC-FT2025.02	26	0.48	25.9	0.481	27.7	0.449
BioMistral7B-FT2025.02	25.4	0.439	27.2	0.455	16.5	0.231
Mistral7B2025.02	25.2	0.572	26.8	0.552	27.6	0.539
Claude 3.0 Haiku2025.02	24.8	0.584	27.1	0.591	28.1	0.614
BioMistral7B2025.02	24.5	0.452	29.7	0.586	26.7	0.548
BioMistral7B2025.02	22.8	0.529	22.7	0.51	23.2	0.51
BioMistral7B2025.02	16.5	0.25	21.2	0.433	23.2	0.423