Clinical Trial Matching on TREC 2023
76.5AccuracyαNeSy-CTM
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| αNeSy-CTMModel=olmo-3-32b-think2026.06 | 76.5 | 71 | 82 | 76.4 | 53.3 | |
| αNeSy-CTMModel=gpt-oss-120b2026.06 | 76 | 77 | 75 | 76 | 52 | |
| NeSy-CTMModel=olmo-3-32b-think2026.06 | 76 | 74 | 78 | 76 | 52 | |
| αNeSy-CTMModel=Average2026.06 | 75.7 | 75.7 | 75.7 | 75.7 | 51.4 | |
| αNeSy-CTMModel=qwen3-vl-32b-instruct2026.06 | 74.6 | 79 | 70.3 | 74.6 | 49.5 | |
| NeSy-CTMModel=gpt-oss-120b2026.06 | 74.5 | 79 | 70 | 74.5 | 49.2 | |
| NeSy-CTMModel=Average2026.06 | 74.3 | 79.3 | 69.3 | 74.3 | 48.9 | |
| NeSy-CTMModel=qwen3-vl-32b-instruct2026.06 | 72.5 | 85 | 60 | 72.1 | 46.5 | |
| QuaSARModel=gpt-oss-120b2026.06 | 70.5 | 84 | 57 | 70 | 42.6 | |
| QuaSARModel=qwen3-vl-32b-instruct2026.06 | 69.5 | 90 | 49 | 68.2 | 42.8 | |
| QuaSARModel=Average2026.06 | 67.2 | 88 | 46.3 | 65.7 | 37.8 | |
| CoTModel=qwen3-vl-32b-instruct2026.06 | 63 | 94 | 32 | 59.1 | 33.1 | |
| CoTModel=gpt-oss-120b2026.06 | 62 | 96 | 28 | 57 | 32.7 | |
| QuaSARModel=olmo-3-32b-think2026.06 | 61.5 | 90 | 33 | 58.1 | 28 | |
| CoTModel=Average2026.06 | 60.2 | 82.3 | 38 | 58.1 | 22.7 | |
| Zero-shotModel=gpt-oss-120b2026.06 | 59.5 | 94 | 25 | 54 | 26.3 | |
| Zero-shotModel=qwen3-vl-32b-instruct2026.06 | 59.5 | 96 | 23 | 53.3 | 27.8 | |
| Zero-shotModel=Average2026.06 | 58 | 91.3 | 24.7 | 52.8 | 21.5 | |
| CoTModel=olmo-3-32b-think2026.06 | 55.5 | 57 | 54 | 55.5 | 11 | |
| Zero-shotModel=olmo-3-32b-think2026.06 | 55 | 84 | 26 | 50.9 | 12.3 |