List your company
ResearchFormerly WizWand

Multilingual Question Answering on TydiQA

81.9AccuracyByT5
18.66835.08451.567.916Oct 20, 2022May 11, 2023Nov 30, 2023Jun 20, 2024Jan 9, 2025Jul 31, 2025Feb 20, 2026
Updated 7mo ago

Evaluation results

MethodLinks
ByT5protocol=finetuned2022.1081.9
GISTBackbone=Llama3.2-3B, Selection Ratio=5%2026.0269.2
Flan-PaLM 540Bprompting=direct prompting2022.1067.8
LESSBackbone=Llama3.2-3B, Selection Ratio=5%2026.0267.1
FullBackbone=Llama3.2-3B, Selection Ratio=100%2026.0266.6
Rand.Backbone=Llama3.2-3B, Selection Ratio=5%2026.0264.1
LengthBackbone=Llama3.2-3B, Selection Ratio=5%2026.0263.9
SELF-EVOLVING CLEANINGBase Model=LLaMA-3.1-8B, Selection Proportion=0.62025.0263.58
Embed.Backbone=Llama3.2-3B, Selection Ratio=5%2026.0262.8
RDS+Backbone=Llama3.2-3B, Selection Ratio=5%2026.0262.6
FIXED-MODEL CLEANINGBase Model=LLaMA-3.1-8B, Selection Proportion=0.62025.0262.38
RHOBase Model=LLaMA-3.1-8B, Selection Proportion=0.62025.0261.9
LESSBackbone=Qwen2.5-1.5B, Selection Ratio=5%2026.0261.4
GISTBackbone=Qwen2.5-1.5B, Selection Ratio=5%2026.0261.2
BaseBackbone=Llama3.2-3B, Selection Ratio=0%2026.0260.4
FullBackbone=Qwen2.5-1.5B, Selection Ratio=100%2026.0259.9
FIXED-MODEL CLEANINGBase Model=Mistral-7B-v0.3, Selection Proportion=0.62025.0259.03
PPL.Backbone=Qwen2.5-1.5B, Selection Ratio=5%2026.0258.4
FULL TOKENS (50K)Base Model=LLaMA-3.1-8B, Token Budget=50K2025.0258.1
BaseBackbone=Qwen2.5-1.5B, Selection Ratio=0%2026.0257.8
PPL.Backbone=Llama3.2-3B, Selection Ratio=5%2026.0257.1
Flan-PaLM 540Bprompting=CoT prompting2022.1057
LengthBackbone=Qwen2.5-1.5B, Selection Ratio=5%2026.0256.7
UNIFORM RANDOM (50K×0.6)Base Model=LLaMA-3.1-8B, Token Budget=50K, Selection Proportion=0.62025.0256.6
SELF-EVOLVING CLEANINGBase Model=LLaMA-3.2-3B, Selection Proportion=0.62025.0256.38
LESSBackbone=Llama2-7B, Selection Ratio=5%2026.0256.2
RDS+Backbone=Qwen2.5-1.5B, Selection Ratio=5%2026.0256.2
SELF-EVOLVING CLEANINGBase Model=Mistral-7B-v0.3, Selection Proportion=0.62025.0256.17
GISTBackbone=Llama2-7B, Selection Ratio=5%2026.0255.8
DS² (10K)Base Model=Mistral-7B-v0.3, Token Budget=10K2025.0255.7
UNIFORM RANDOM (50K×0.6)Base Model=Mistral-7B-v0.3, Token Budget=50K, Selection Proportion=0.62025.0255.7
Rand.Backbone=Qwen2.5-1.5B, Selection Ratio=5%2026.0255.6
FULL TOKENS (50K)Base Model=Mistral-7B-v0.3, Token Budget=50K2025.0255.6
BASEBase Model=Mistral-7B-v0.32025.0254.7
U-PaLMSize=540B, Few-shot=true, Chain-of-thought prompting=true2022.1054.6
RHOBase Model=Mistral-7B-v0.3, Selection Proportion=0.62025.0254.5
Embed.Backbone=Qwen2.5-1.5B, Selection Ratio=5%2026.0254.1
FullBackbone=Llama2-7B, Selection Ratio=100%2026.0254
RHOBase Model=LLaMA-3.2-3B, Selection Proportion=0.62025.0253.6
PaLMSize=540B, Few-shot=true, Chain-of-thought prompting=true2022.1052.9
PaLM 540Bprompting=direct prompting2022.1052.9
Rand.Backbone=Llama2-7B, Selection Ratio=5%2026.0252.7
FIXED-MODEL CLEANINGBase Model=LLaMA-3.2-3B, Selection Proportion=0.62025.0252.6
LengthBackbone=Llama2-7B, Selection Ratio=5%2026.0251.1
RDS+Backbone=Llama2-7B, Selection Ratio=5%2026.0251
Embed.Backbone=Llama2-7B, Selection Ratio=5%2026.0249.8
FULL TOKENS (50K)Base Model=LLaMA-3.2-3B, Token Budget=50K2025.0249.6
IProXCandidate Training Data=CoT, rho=0.32026.0248.72
Llama3.2-3BCandidate Training Data=CoT2026.0247.9
UNIFORM RANDOM (50K×0.6)Base Model=LLaMA-3.2-3B, Token Budget=50K, Selection Proportion=0.62025.0247
IProXCandidate Training Data=CoT, rho=0.52026.0246.73
BaseBackbone=Llama2-7B, Selection Ratio=0%2026.0246.4
PaLM 540Bprompting=CoT prompting2022.1045.9
DS² (10K)Base Model=LLaMA-3.1-8B, Token Budget=10K2025.0245.8
IProXCandidate Training Data=CoT, rho=0.72026.0243.18
Llama3.2-1BCandidate Training Data=CoT2026.0242.67
DS² (10K)Base Model=LLaMA-3.2-3B, Token Budget=10K2025.0241.2
IProXCandidate Training Data=BioInstruct, rho=0.32026.0239.17
Llama3.2-3BCandidate Training Data=BioInstruct2026.0238.96
IProXCandidate Training Data=BioInstruct, rho=0.52026.0238.36
IProXCandidate Training Data=BioInstruct, rho=0.72026.0236.48
Llama3.2-1BCandidate Training Data=BioInstruct2026.0233.94
PPL.Backbone=Llama2-7B, Selection Ratio=5%2026.0232.9
BASEBase Model=LLaMA-3.1-8B2025.0222.8
BASEBase Model=LLaMA-3.2-3B2025.0221.1