Long-context language modeling evaluation on FDA (test)
0.8004ScoreGA-S2
Evaluation results
| Method | Links | |
|---|---|---|
| GA-S2Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=50%2025.12 | 0.8004 | |
| GA-S2Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=33%2025.12 | 0.7822 | |
| AVG-S2Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=50%2025.12 | 0.7641 | |
| LM-PPLBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 0.7613 | |
| GA-S2Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 0.7595 | |
| GA-S2Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=25%2025.12 | 0.7559 | |
| AVG-S2Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=25%2025.12 | 0.7532 | |
| GA-S2Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=25%2025.12 | 0.7514 | |
| GA-S2Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=33%2025.12 | 0.7514 | |
| ARBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=33%2025.12 | 0.7505 | |
| AR-MHBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 0.7505 | |
| GR-S2Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=50%2025.12 | 0.7468 | |
| GR-S2Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=33%2025.12 | 0.7459 | |
| ARBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=50%2025.12 | 0.7432 | |
| SMARTBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=33%2025.12 | 0.7359 | |
| GA-S2Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=12.5%2025.12 | 0.7314 | |
| AVG-S2Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=33%2025.12 | 0.7241 | |
| AVG-S2Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=33%2025.12 | 0.7196 | |
| UNIFORMBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=50%2025.12 | 0.7178 | |
| SMARTBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 0.7169 | |
| KVBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=50%2025.12 | 0.7096 | |
| AR-MHBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=50%2025.12 | 0.7069 | |
| AVG-S2Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 0.706 | |
| ARBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 0.7051 | |
| ARBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=25%2025.12 | 0.7042 | |
| SMARTBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=25%2025.12 | 0.7033 | |
| VTBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 0.696 | |
| KVBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 0.6942 | |
| GR-S2Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=33%2025.12 | 0.6933 | |
| GR-S2Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=25%2025.12 | 0.686 | |
| CWEBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=50%2025.12 | 0.6842 | |
| AR-MHBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=33%2025.12 | 0.6833 | |
| AVG-S2Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=25%2025.12 | 0.6806 | |
| KVBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=25%2025.12 | 0.6788 | |
| CWEBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 0.6779 | |
| KVBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=33%2025.12 | 0.676 | |
| AVG-S1Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=50%2025.12 | 0.6751 | |
| GR-S1Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=50%2025.12 | 0.6697 | |
| SMARTBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=50%2025.12 | 0.6679 | |
| AR-MHBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=33%2025.12 | 0.6679 | |
| UNIFORMBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 0.6642 | |
| GR-S2Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 0.6633 | |
| AVG-S2Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=12.5%2025.12 | 0.6588 | |
| KVBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=25%2025.12 | 0.6461 | |
| AVG-S1Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 0.6443 | |
| AR-MHBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=25%2025.12 | 0.6416 | |
| ARBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=33%2025.12 | 0.6379 | |
| UNIFORMBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=25%2025.12 | 0.6334 | |
| VTBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=50%2025.12 | 0.6279 | |
| KVBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=33%2025.12 | 0.6261 | |
| GA-S1Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=50%2025.12 | 0.6234 | |
| CWEBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=33%2025.12 | 0.6207 | |
| GA-S2Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=12.5%2025.12 | 0.6025 | |
| ARBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=25%2025.12 | 0.6016 | |
| GR-S1Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 0.6007 | |
| AVG-S2Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=12.5%2025.12 | 0.588 | |
| ACT-MSEBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=50%2025.12 | 0.5771 | |
| ACT-MSEBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=33%2025.12 | 0.5726 | |
| ACT-MSEBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 0.5672 | |
| GR-S1Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=33%2025.12 | 0.5644 | |
| AR-MHBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=25%2025.12 | 0.5572 | |
| GA-S1Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=50%2025.12 | 0.5563 | |
| LM-PPLBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=50%2025.12 | 0.5508 | |
| CWEBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=33%2025.12 | 0.5345 | |
| LM-PPLBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=33%2025.12 | 0.5299 | |
| AVG-S1Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=33%2025.12 | 0.5227 | |
| GA-S1Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=33%2025.12 | 0.5163 | |
| GR-S1Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=33%2025.12 | 0.51 | |
| GR-S2Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=25%2025.12 | 0.5054 | |
| ACT-MSEBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=25%2025.12 | 0.5045 | |
| ARBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=12.5%2025.12 | 0.5 | |
| LM-PPLBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=25%2025.12 | 0.5 | |
| VTBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=33%2025.12 | 0.4809 | |
| LM-PPLBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=33%2025.12 | 0.4809 | |
| AR-MHBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=12.5%2025.12 | 0.4773 | |
| GA-S1Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=33%2025.12 | 0.4746 | |
| UNIFORMBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=33%2025.12 | 0.4728 | |
| AVG-S1Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=33%2025.12 | 0.4628 | |
| ARBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=12.5%2025.12 | 0.4601 | |
| GR-S1Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=25%2025.12 | 0.4465 | |
| SMARTBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=33%2025.12 | 0.4374 | |
| GA-S1Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=25%2025.12 | 0.4365 | |
| GA-S1Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=25%2025.12 | 0.4247 | |
| LM-PPLBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=12.5%2025.12 | 0.4201 | |
| VTBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=25%2025.12 | 0.4183 | |
| AVG-S1Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=25%2025.12 | 0.4165 | |
| AVG-S1Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=25%2025.12 | 0.4074 | |
| UNIFORMBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=25%2025.12 | 0.4011 | |
| UNIFORMBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=33%2025.12 | 0.3684 | |
| UNIFORMBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=12.5%2025.12 | 0.3648 | |
| VTBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=33%2025.12 | 0.3648 | |
| GR-S2Base Model=Qwen2.5-3B-Instruct, Softmax Ratio=12.5%2025.12 | 0.3612 | |
| GR-S1Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=25%2025.12 | 0.3548 | |
| SMARTBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=25%2025.12 | 0.3448 | |
| VTBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=25%2025.12 | 0.3385 | |
| ACT-MSEBase Model=Qwen2.5-3B-Instruct, Softmax Ratio=33%2025.12 | 0.3385 | |
| GR-S2Base Model=Llama-3.2-3B-Instruct, Softmax Ratio=12.5%2025.12 | 0.3303 | |
| SMARTBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=12.5%2025.12 | 0.3276 | |
| CWEBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=25%2025.12 | 0.3258 | |
| CWEBase Model=Llama-3.2-3B-Instruct, Softmax Ratio=12.5%2025.12 | 0.3149 |