Question Answering on SQuAD (ACE and AUROC Metrics)
0.112ACE (General)ATS
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ATSBackbone=Mistral, SC Measure=E-SC2026.04 | 0.112 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.68 | |
| TSBackbone=Mistral, SC Measure=L-SC2026.04 | 0.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.78 | |
| ATSBackbone=Mistral, SC Measure=G-SC2026.04 | 0.107 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.7 | |
| TSBackbone=Mistral, SC Measure=E-SC2026.04 | 0.104 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.777 | |
| ATSBackbone=Mistral, SC Measure=ML-SC2026.04 | 0.102 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.73 | |
| TSBackbone=Mistral, SC Measure=G-SC2026.04 | 0.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.791 | |
| PlattBackbone=Mistral, SC Measure=L-SC2026.04 | 0.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.77 | |
| SEBackbone=Mistral, SC Measure=L-SC2026.04 | 0.099 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.77 | |
| ATSBackbone=Mistral, SC Measure=L-SC2026.04 | 0.099 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.77 | |
| TSBackbone=Mistral, SC Measure=B-SC2026.04 | 0.096 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.8 | |
| TSBackbone=Mistral, SC Measure=T-SC2026.04 | 0.095 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.798 | |
| PlattBackbone=Mistral, SC Measure=E-SC2026.04 | 0.09 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.766 | |
| TSBackbone=Mistral, SC Measure=ML-SC2026.04 | 0.086 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.815 | |
| PlattBackbone=Mistral, SC Measure=G-SC2026.04 | 0.086 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.782 | |
| ATSBackbone=Mistral, SC Measure=T-SC2026.04 | 0.082 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.74 | |
| SEBackbone=Mistral, SC Measure=E-SC2026.04 | 0.081 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.789 | |
| SEBackbone=Mistral, SC Measure=B-SC2026.04 | 0.08 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.806 | |
| PlattBackbone=Mistral, SC Measure=ML-SC2026.04 | 0.08 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.815 | |
| SEBackbone=Mistral, SC Measure=T-SC2026.04 | 0.079 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.81 | |
| ATSBackbone=Mistral, SC Measure=B-SC2026.04 | 0.079 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.74 | |
| SEBackbone=Mistral, SC Measure=G-SC2026.04 | 0.077 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.801 | |
| PlattBackbone=Mistral, SC Measure=T-SC2026.04 | 0.077 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.8 | |
| PlattBackbone=Mistral, SC Measure=B-SC2026.04 | 0.075 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.804 | |
| TSBackbone=Mistral, SC Measure=IC-SC2026.04 | 0.073 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.8 | |
| ATSBackbone=Mistral, SC Measure=IC-SC2026.04 | 0.072 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.73 | |
| SEBackbone=Mistral, SC Measure=ML-SC2026.04 | 0.069 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.821 | |
| SEBackbone=Mistral, SC Measure=IC-SC2026.04 | 0.066 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.799 | |
| PlattBackbone=Mistral, SC Measure=IC-SC2026.04 | 0.064 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.801 | |
| TSModel=Qwen, SC Measure=L-SC2026.04 | 0.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.702 | |
| BaseBackbone=Mistral, SC Measure=E-SC2026.04 | 0.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.676 | |
| BaseBackbone=Mistral, SC Measure=L-SC2026.04 | 0.059 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.697 | |
| TSModel=Qwen, SC Measure=E-SC2026.04 | 0.058 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.726 | |
| BaseBackbone=Mistral, SC Measure=G-SC2026.04 | 0.057 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.69 | |
| ATSModel=Qwen, SC Measure=E-SC2026.04 | 0.056 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.523 | |
| ATSModel=Qwen, SC Measure=G-SC2026.04 | 0.056 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.527 | |
| TSModel=Qwen, SC Measure=G-SC2026.04 | 0.055 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.743 | |
| ATSModel=Qwen, SC Measure=ML-SC2026.04 | 0.055 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.544 | |
| BaseBackbone=Mistral, SC Measure=B-SC2026.04 | 0.055 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.733 | |
| BaseBackbone=Mistral, SC Measure=T-SC2026.04 | 0.054 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.735 | |
| TSModel=Qwen, SC Measure=ML-SC2026.04 | 0.053 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.76 | |
| ATSModel=Qwen, SC Measure=B-SC2026.04 | 0.053 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.593 | |
| ATSModel=Qwen, SC Measure=IC-SC2026.04 | 0.053 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.596 | |
| ATSModel=Qwen, SC Measure=T-SC2026.04 | 0.053 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.547 | |
| BaseBackbone=Mistral, SC Measure=ML-SC2026.04 | 0.053 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.716 | |
| TSModel=Qwen, SC Measure=T-SC2026.04 | 0.052 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.729 | |
| BaseBackbone=Mistral, SC Measure=IC-SC2026.04 | 0.052 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.725 | |
| BaseModel=Qwen, SC Measure=E-SC2026.04 | 0.051 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.534 | |
| BaseModel=Qwen, SC Measure=ML-SC2026.04 | 0.051 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.554 | |
| BaseModel=Qwen, SC Measure=G-SC2026.04 | 0.051 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.537 | |
| TSModel=Qwen, SC Measure=B-SC2026.04 | 0.051 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.748 | |
| SEModel=Qwen, SC Measure=E-SC2026.04 | 0.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.618 | |
| PlattModel=Qwen, SC Measure=E-SC2026.04 | 0.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.619 | |
| SEModel=Qwen, SC Measure=G-SC2026.04 | 0.049 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.627 | |
| TSModel=Qwen, SC Measure=IC-SC2026.04 | 0.049 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.767 | |
| BaseModel=Qwen, SC Measure=B-SC2026.04 | 0.048 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.597 | |
| BaseModel=Qwen, SC Measure=IC-SC2026.04 | 0.048 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.598 | |
| BaseModel=Qwen, SC Measure=T-SC2026.04 | 0.048 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.597 | |
| SEModel=Qwen, SC Measure=L-SC2026.04 | 0.048 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.654 | |
| SEModel=Qwen, SC Measure=IC-SC2026.04 | 0.048 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.669 | |
| PlattModel=Qwen, SC Measure=G-SC2026.04 | 0.048 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.627 | |
| BaseModel=Qwen, SC Measure=L-SC2026.04 | 0.047 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.605 | |
| PlattModel=Qwen, SC Measure=L-SC2026.04 | 0.047 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.649 | |
| PlattModel=Qwen, SC Measure=IC-SC2026.04 | 0.047 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.66 | |
| ATSModel=Qwen, SC Measure=L-SC2026.04 | 0.047 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.649 | |
| SEModel=Qwen, SC Measure=B-SC2026.04 | 0.046 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.669 | |
| SEModel=Qwen, SC Measure=T-SC2026.04 | 0.046 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.667 | |
| PlattModel=Qwen, SC Measure=B-SC2026.04 | 0.046 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.668 | |
| PlattModel=Qwen, SC Measure=T-SC2026.04 | 0.046 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.667 | |
| SEModel=Qwen, SC Measure=ML-SC2026.04 | 0.044 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.643 | |
| PlattModel=Qwen, SC Measure=ML-SC2026.04 | 0.044 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.642 | |
| ATSModel=Llama2026.04 | — | 0.054 | 0.065 | 0.047 | 0.039 | 0.038 | 0.041 | 0.052 | 0.659 | 0.731 | 0.679 | 0.705 | 0.702 | 0.709 | 0.665 | — | |
| BaseModel=Llama2026.04 | — | 0.047 | 0.047 | 0.041 | 0.046 | 0.043 | 0.045 | 0.046 | 0.594 | 0.662 | 0.632 | 0.679 | 0.677 | 0.685 | 0.607 | — | |
| PlattModel=Llama2026.04 | — | 0.043 | 0.063 | 0.043 | 0.063 | 0.052 | 0.064 | 0.043 | 0.739 | 0.73 | 0.786 | 0.765 | 0.748 | 0.772 | 0.757 | — | |
| SEModel=Llama2026.04 | — | 0.042 | 0.065 | 0.042 | 0.064 | 0.052 | 0.064 | 0.043 | 0.737 | 0.737 | 0.788 | 0.775 | 0.748 | 0.776 | 0.754 | — | |
| TSModel=Llama2026.04 | — | 0.049 | 0.086 | 0.059 | 0.09 | 0.066 | 0.09 | 0.051 | 0.804 | 0.773 | 0.827 | 0.807 | 0.783 | 0.806 | 0.806 | — |