Question Answering on SQuAD (ROUGE-Lsum)
83.06ROUGE-LsumSCALENET (Layer-wise)
Evaluation results
| Method | Links | |
|---|---|---|
| SCALENET (Layer-wise)LLM Backbone=Qwen4B-Instruct, TTA Steps=5, TTA Method=Layer-wise2026.02 | 83.06 | |
| SCALENET (Layer-wise)LLM Backbone=Qwen4B-Instruct, TTA Steps=1, TTA Method=Layer-wise2026.02 | 79.58 | |
| Base ModelLLM Backbone=Qwen4B-Instruct, TTA Steps=0, TTA Method=None2026.02 | 77.22 | |
| Naive TTALLM Backbone=Qwen4B-Instruct, TTA Steps=1, TTA Method=Fixed2026.02 | 76.57 | |
| Naive TTALLM Backbone=Qwen4B-Instruct, TTA Steps=5, TTA Method=Fixed2026.02 | 76.34 | |
| SCALENET (Layer-wise)LLM Backbone=Llama3B-Instruct, TTA Steps=5, TTA Method=Layer-wise2026.02 | 73.53 | |
| SCALENET (Layer-wise)LLM Backbone=Llama3B-Instruct, TTA Steps=1, TTA Method=Layer-wise2026.02 | 72.38 | |
| SCALENET (Step-wise)LLM Backbone=Llama3B-Instruct, TTA Steps=1, TTA Method=Step-wise2026.02 | 71.33 | |
| SCALENET (Step-wise)LLM Backbone=Llama3B-Instruct, TTA Steps=5, TTA Method=Step-wise2026.02 | 71.14 | |
| Base ModelLLM Backbone=Llama3B-Instruct, TTA Steps=0, TTA Method=None2026.02 | 70.8 | |
| Naive TTALLM Backbone=Llama3B-Instruct, TTA Steps=1, TTA Method=Fixed2026.02 | 70.58 | |
| Naive TTALLM Backbone=Llama3B-Instruct, TTA Steps=5, TTA Method=Fixed2026.02 | 65.32 | |
| SCALENET (Step-wise)LLM Backbone=Qwen4B-Instruct, TTA Steps=5, TTA Method=Step-wise2026.02 | 54.02 | |
| SCALENET (Step-wise)LLM Backbone=Qwen4B-Instruct, TTA Steps=1, TTA Method=Step-wise2026.02 | 53.61 |