List your company
ResearchFormerly WizWand

Logical Reasoning on Shuffled Objects

92.1AccuracyGateKD
19.61238.43157.2576.069Jan 12, 2026Feb 1, 2026Feb 21, 2026Mar 13, 2026Apr 2, 2026Apr 22, 2026May 13, 2026
Updated 5mo ago

Evaluation results

MethodLinks
GateKDModel=FlanT5-base, #Params=250M2026.0592.1——
GateKDModel=T5-base, #Params=250M2026.0590.6——
In-Writing-BFBackbone=Qwen3-8B, Shot=02026.0189.9——
Vanilla-KDModel=FlanT5-large, #Params=780M2026.0589.7——
In-Writing-IFBackbone=Qwen3-8B, Shot=02026.0188.7——
In-Writing-BFBackbone=Qwen3-8B, Shot=42026.0188.7——
In-Writing-BFBackbone=Qwen3-8B, Shot=12026.0188.3——
Mentor-KDModel=FlanT5-base, #Params=250M2026.0588.3——
Vanilla-KDModel=T5-large, #Params=780M2026.0588.1——
In-Writing-BaseBackbone=Qwen3-8B, Shot=02026.0187.6——
In-Writing-IFBackbone=Qwen3-8B, Shot=42026.0186.7——
MCC-KDModel=FlanT5-base, #Params=250M2026.0585.1——
Mentor-KDModel=T5-base, #Params=250M2026.0584.9——
In-Writing-IFBackbone=Qwen3-8B, Shot=12026.0184.8——
ZS-CoT (teacher)Model=GPT-4o-mini, #Params=-2026.0584.3——
In-Writing-BaseBackbone=Qwen3-8B, Shot=42026.0184——
GateKDModel=FlanT5-small, #Params=80M2026.0583.7——
ZS-CoT (teacher)Model=GPT-4o-mini, #Params=-2026.0583.1——
In-Writing-BaseBackbone=Qwen3-8B, Shot=12026.0182.6——
Vanilla-KDModel=FlanT5-base, #Params=250M2026.0582.6——
MCC-KDModel=T5-base, #Params=250M2026.0581——
GateKDModel=T5-small, #Params=80M2026.0580.8——
Vanilla-KDModel=T5-base, #Params=250M2026.0578.4——
Mentor-KDModel=FlanT5-small, #Params=80M2026.0576.4——
Mentor-KDModel=T5-small, #Params=80M2026.0572.9——
MCC-KDModel=FlanT5-small, #Params=80M2026.0570——
Vanilla-KDModel=FlanT5-small, #Params=80M2026.0568.2——
MCC-KDModel=T5-small, #Params=80M2026.0566.1——
Vanilla-KDModel=T5-small, #Params=80M2026.0563.7——
In-Writing-BFBackbone=LLaMA3-8B, Shot=12026.0148.3——
In-Writing-IFBackbone=LLaMA3-8B, Shot=42026.0146.7——
In-Writing-BFBackbone=LLaMA3-8B, Shot=42026.0146.4——
In-Writing-IFBackbone=LLaMA3-8B, Shot=12026.0145.8——
In-Writing-BaseBackbone=LLaMA3-8B, Shot=02026.0139——
In-Writing-BaseBackbone=LLaMA3-8B, Shot=42026.0138.2——
In-Writing-IFBackbone=LLaMA3-8B, Shot=02026.0134.8——
NL-to FormatBackbone=LLaMA3-8B, Shot=02026.0127——
In-Writing-BFBackbone=LLaMA3-8B, Shot=02026.0124.3——
In-Writing-BaseBackbone=LLaMA3-8B, Shot=12026.0122.4——
zero-shotReasoning Method=Zero-shot2022.05—31.329.7
zero-shot-cotReasoning Method=Zero-shot-CoT2022.05—52.452.9