Multi-label Abnormality Analysis on CT-RATE (test)
0.8401AUROCCOLIPRI
Evaluation results
| Method | Links | |||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| COLIPRISetup=Foundation Probing2025.10 | 0.8401 | — | — | 0.5738 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CT-SSGArchitecture=2.5D2025.10 | 0.8364 | 0.5718 | 0.8103 | 0.5824 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CT-SSGSetup=Supervised2025.10 | 0.8364 | — | — | 0.5824 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CT-ScrollArchitecture=2.5D2025.10 | 0.8218 | 0.543 | 0.7968 | 0.5309 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CT-MvGArchitecture=2.5D2025.10 | 0.8191 | 0.5235 | 0.7825 | 0.5151 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ViViTArchitecture=3D Transformer2025.10 | 0.8003 | 0.5066 | 0.7681 | 0.4857 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CT-NetArchitecture=3D CNN2025.10 | 0.7969 | 0.5104 | 0.7778 | 0.4852 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Swin3DArchitecture=3D Transformer2025.10 | 0.7939 | 0.5025 | 0.7663 | 0.4726 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ViT3DArchitecture=3D Transformer2025.10 | 0.7897 | 0.4953 | 0.7537 | 0.4631 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HLIPSupervision source=Qwen Summarized Reports, Zero-shot evaluation=true2025.05 | 0.787 | 0.755 | 0.724 | — | 0.384 | 74.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MerlinSetup=Foundation Probing2025.10 | 0.7785 | — | — | 0.4653 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| fVLMSupervision source=Qwen Summarized Reports, Zero-shot evaluation=true2025.05 | 0.778 | 0.751 | 0.718 | — | 0.379 | 72.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HLIPSupervision source=Original Reports, Zero-shot evaluation=true2025.05 | 0.777 | 0.747 | 0.714 | — | 0.379 | 73 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ResNet3DArchitecture=3D CNN2025.10 | 0.7748 | 0.5151 | 0.8067 | 0.5081 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CT-CLIPSetup=Foundation Probing2025.10 | 0.7685 | — | — | 0.4287 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CT-FMSetup=Foundation Probing2025.10 | 0.7435 | — | — | 0.3922 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CT-CLIPSupervision source=Original Reports, Zero-shot evaluation=true2025.05 | 0.733 | 0.708 | 0.669 | — | 0.326 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MerlinSupervision source=Original Reports, Zero-shot evaluation=true2025.05 | 0.728 | 0.709 | 0.672 | — | 0.337 | 70.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BIUDSupervision source=Original Reports, Zero-shot evaluation=true2025.05 | 0.713 | 0.716 | 0.681 | — | 0.338 | 67.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CT-CLIPInput=Uncompressed2026.05 | 0.6958 | — | 0.6292 | — | — | — | — | 0.6825 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CT-LiteInput=JPEG2026.05 | 0.6646 | — | 0.6067 | — | — | — | — | 0.6242 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| COLIPRIInput=JPEG, Training Protocol=Fine-tuned on JPEG2026.05 | 0.5891 | — | 0.5283 | — | — | — | — | 0.4942 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CT-CLIPInput=JPEG, Training Protocol=Fine-tuned on JPEG2026.05 | 0.5748 | — | 0.5092 | — | — | — | — | 0.5158 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RandomDescription=Predictions sampled from a uniform distribution2025.10 | 0.4988 | 0.2778 | 0.4989 | 0.2094 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ALBERT-baseTraining Mode=non-FT (off the shelf model)2025.06 | — | — | — | — | — | — | 0.461 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ALBERT-baseTraining Mode=DP-LoRA, Privacy Budget (epsilon)=0.01, Epochs=102025.06 | — | — | — | — | — | — | 0.315 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ALBERT-baseTraining Mode=DP-LoRA, Privacy Budget (epsilon)=0.1, Epochs=102025.06 | — | — | — | — | — | — | 0.45 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ALBERT-baseTraining Mode=DP-LoRA, Privacy Budget (epsilon)=1, Epochs=102025.06 | — | — | — | — | — | — | 0.492 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ALBERT-baseTraining Mode=DP-LoRA, Privacy Budget (epsilon)=10, Epochs=102025.06 | — | — | — | — | — | — | 0.591 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ALBERT-baseTraining Mode=Non-priv LoRA, r=64, Epochs=102025.06 | — | — | — | — | — | — | 0.788 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ALBERT-baseTraining Mode=Non-priv Full FT, Epochs=102025.06 | — | — | — | — | — | — | 0.972 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BERT-mediumTraining Mode=non-FT (off the shelf model)2025.06 | — | — | — | — | — | — | 0.39 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BERT-mediumTraining Mode=DP-LoRA, Privacy Budget (epsilon)=0.01, Epochs=102025.06 | — | — | — | — | — | — | 0.315 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BERT-mediumTraining Mode=DP-LoRA, Privacy Budget (epsilon)=0.1, Epochs=102025.06 | — | — | — | — | — | — | 0.449 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BERT-mediumTraining Mode=DP-LoRA, Privacy Budget (epsilon)=1, Epochs=102025.06 | — | — | — | — | — | — | 0.477 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BERT-mediumTraining Mode=DP-LoRA, Privacy Budget (epsilon)=10, Epochs=102025.06 | — | — | — | — | — | — | 0.535 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BERT-mediumTraining Mode=Non-priv LoRA, r=64, Epochs=102025.06 | — | — | — | — | — | — | 0.792 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BERT-mediumTraining Mode=Non-priv Full FT, Epochs=102025.06 | — | — | — | — | — | — | 0.972 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BERT-smallTraining Mode=non-FT (off the shelf model)2025.06 | — | — | — | — | — | — | 0.384 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BERT-smallTraining Mode=DP-LoRA, Privacy Budget (epsilon)=0.01, Epochs=102025.06 | — | — | — | — | — | — | 0.358 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BERT-smallTraining Mode=DP-LoRA, Privacy Budget (epsilon)=0.1, Epochs=102025.06 | — | — | — | — | — | — | 0.448 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BERT-smallTraining Mode=DP-LoRA, Privacy Budget (epsilon)=1, Epochs=102025.06 | — | — | — | — | — | — | 0.479 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BERT-smallTraining Mode=DP-LoRA, Privacy Budget (epsilon)=10, Epochs=102025.06 | — | — | — | — | — | — | 0.537 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BERT-smallTraining Mode=Non-priv LoRA, r=64, Epochs=102025.06 | — | — | — | — | — | — | 0.795 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BERT-smallTraining Mode=Non-priv Full FT, Epochs=102025.06 | — | — | — | — | — | — | 0.971 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Foundation VAETrain Set=+1xOurs2026.05 | — | — | — | — | — | — | — | — | 76.99 | 61.94 | 82.06 | 72.84 | 68.64 | 64.38 | 73.81 | 79.38 | 65.7 | 76.41 | 56.99 | 70 | 66.09 | 61.72 | 72.09 | 63.78 | 82.61 | 77.25 | 70.71 | |
| GenerateCTTrain Set=+1xGenerateCT2026.05 | — | — | — | — | — | — | — | — | 70.77 | 68.31 | 73.13 | 71.24 | 67.27 | 63.92 | 74.63 | 80.24 | 67.8 | 75.24 | 53.63 | 72.59 | 61.55 | 55.46 | 74.84 | 62.64 | 82.7 | 80.47 | 69.8 | |
| MedSynTrain Set=+1xMedSyn2026.05 | — | — | — | — | — | — | — | — | 74.65 | 67.66 | 76.58 | 70.18 | 62.55 | 63.99 | 74.5 | 81.27 | 63.98 | 77.32 | 50.03 | 71.06 | 59.69 | 63.82 | 75.55 | 60.61 | 83.41 | 77.77 | 69.7 | |
| RealTrain Set=Real2026.05 | — | — | — | — | — | — | — | — | 71.53 | 55.67 | 76.73 | 69.72 | 66.23 | 58.27 | 71 | 75.44 | 61.39 | 79.42 | 52.32 | 69.06 | 65.74 | 63.38 | 66.69 | 64.14 | 79.13 | 77.25 | 67.95 |