Aggregate Model Performance on Combined Benchmark Suite
100Average ScoreBaseline
Evaluation results
| Method | Links | |
|---|---|---|
| BaselineBase Model=Qwen2.5-VL-3B-Instruct, Token Retention Rate=100%2026.02 | 100 | |
| IDPrunerBase Model=Qwen2.5-VL-3B-Instruct, Token Retention Rate=25%2026.02 | 94.42 | |
| IDPrunerBase Model=Qwen2.5-VL-3B-Instruct, Token Retention Rate=10%2026.02 | 85.71 | |
| C2LIPMethod Category=Models fine-tuned by us2026.03 | 75 | |
| SigLIPBackbone=ViT-B/16, Fine-tuning Dataset=CC3M, Method Category=Models fine-tuned by us2026.03 | 71.5 | |
| FG-CLIPMethod Category=Fine-grained models2026.03 | 70.7 | |
| SigLIPBackbone=ViT-B/162026.03 | 70 | |
| CLICMethod Category=Composition-aware models2026.03 | 68.2 | |
| NegCLIPMethod Category=Composition-aware models2026.03 | 67.7 | |
| FineCLIPMethod Category=Fine-grained models2026.03 | 65.8 | |
| SLVC-RMethod Category=Composition-aware models2026.03 | 64.1 | |
| CLIP (OpenAI)Backbone=ViT-B/162026.03 | 63.5 | |
| FLAIR-3mMethod Category=Fine-grained models2026.03 | 63.1 | |
| CoN-CLIPMethod Category=Composition-aware models2026.03 | 63.1 | |
| CLIP (OpenAI)Backbone=ViT-B/322026.03 | 62.8 | |
| SLVC-RLMethod Category=Composition-aware models2026.03 | 62.5 | |
| TripletCLIPMethod Category=Composition-aware models2026.03 | 61.6 | |
| LLIPMethod Category=Fine-grained models2026.03 | 61.4 | |
| CE-CLIPMethod Category=Composition-aware models2026.03 | 60.5 | |
| DreamLIP-3mMethod Category=Fine-grained models2026.03 | 59.5 | |
| DAC-SAMMethod Category=Composition-aware models2026.03 | 57.5 | |
| DAC-LLMMethod Category=Composition-aware models2026.03 | 57.2 | |
| GatedNormArchitecture=24B-A3B, Precision=BF16, Training Tokens=500B2026.01 | 54.79 | |
| GatedNormArchitecture=24B-A3B, Precision=W4A4+SQ, Training Tokens=500B2026.01 | 53.56 | |
| GatedNormArchitecture=24B-A3B, Precision=W4A4, Training Tokens=500B2026.01 | 53.43 | |
| PreAffineArchitecture=24B-A3B, Precision=BF16, Training Tokens=500B2026.01 | 52.99 | |
| Gated AttentionArchitecture=24B-A3B, Precision=BF16, Training Tokens=500B2026.01 | 52.71 | |
| Gated AttentionArchitecture=24B-A3B, Precision=W4A4+SQ, Training Tokens=500B2026.01 | 51.23 | |
| Gated AttentionArchitecture=24B-A3B, Precision=W4A4, Training Tokens=500B2026.01 | 50.89 | |
| PreAffineArchitecture=24B-A3B, Precision=W4A4+SQ, Training Tokens=500B2026.01 | 50.23 | |
| PreAffineArchitecture=24B-A3B, Precision=W4A4, Training Tokens=500B2026.01 | 49.84 | |
| SCR (Ours)Backbone=Qwen2.5-7B-Instruct2026.01 | 48.03 | |
| SFT + GRPOBackbone=Qwen2.5-7B-Instruct2026.01 | 46.72 | |
| GRPOBackbone=Qwen2.5-7B-Instruct2026.01 | 46.6 | |
| SCR-Stage IBackbone=Qwen2.5-7B-Instruct2026.01 | 43.92 | |
| BaseBackbone=Qwen2.5-7B-Instruct2026.01 | 43.55 | |
| GatedNormArchitecture=MoE-7B-A2B, Precision=BF16, Training Tokens=1.2T2026.01 | 43.11 | |
| PreAffineArchitecture=MoE-7B-A2B, Precision=BF16, Training Tokens=1.2T2026.01 | 42.85 | |
| Gated AttentionArchitecture=MoE-7B-A2B, Precision=BF16, Training Tokens=1.2T2026.01 | 41.73 | |
| SCR (Ours)Backbone=Qwen2.5-3B-Instruct2026.01 | 39.75 | |
| SCR-SFTBackbone=Qwen2.5-7B-Instruct2026.01 | 39.16 | |
| GRPOBackbone=Qwen2.5-3B-Instruct2026.01 | 39.03 | |
| SFT + GRPOBackbone=Qwen2.5-3B-Instruct2026.01 | 38.98 | |
| Self-RefineBackbone=Qwen2.5-7B-Instruct2026.01 | 38.35 | |
| SCR-Stage IBackbone=Qwen2.5-3B-Instruct2026.01 | 34.72 | |
| Self-RefineBackbone=Qwen2.5-3B-Instruct2026.01 | 34.25 | |
| SCR (Ours)Backbone=Llama3.1-8B-Instruct2026.01 | 34.25 | |
| BaseBackbone=Qwen2.5-3B-Instruct2026.01 | 34.05 | |
| SCR-Stage IBackbone=Llama3.1-8B-Instruct2026.01 | 33.45 | |
| SFT + GRPOBackbone=Llama3.1-8B-Instruct2026.01 | 33.43 | |
| GRPOBackbone=Llama3.1-8B-Instruct2026.01 | 31.63 | |
| SCR-SFTBackbone=Llama3.1-8B-Instruct2026.01 | 29.22 | |
| SCR-SFTBackbone=Qwen2.5-3B-Instruct2026.01 | 26.53 | |
| BaseBackbone=Llama3.1-8B-Instruct2026.01 | 24.71 | |
| IL-CLIPMethod Category=Codebook-based models2026.03 | 23.2 | |
| Codebook-CLIPMethod Category=Codebook-based models2026.03 | 20 | |
| Self-RefineBackbone=Llama3.1-8B-Instruct2026.01 | 8.64 |