List your company
ResearchFormerly WizWand

Logical Reasoning on LSAT

54AccuracyGlob. non-uni
20.7229.363846.64May 16, 2023Oct 9, 2023Mar 3, 2024Jul 27, 2024Dec 20, 2024May 15, 2025Oct 8, 2025
Updated 6mo ago

Evaluation results

MethodLinks
Glob. non-uniBackbone=DS-R1-Distill-Llama-8B2025.1054
Loc. uniBackbone=DS-R1-Distill-Llama-8B2025.1053
Glob. non-uniBackbone=DS-R1-Distill-Qwen-7B2025.1053
Mean Acc.Backbone=DS-R1-Distill-Llama-8B2025.1052
Self-Cert.Backbone=DS-R1-Distill-Llama-8B2025.1052
Low Ent.Backbone=DS-R1-Distill-Llama-8B2025.1052
Low Ent.Backbone=DS-R1-Distill-Qwen-7B2025.1052
High Conf.Backbone=DS-R1-Distill-Llama-8B2025.1051
High Conf.Backbone=DS-R1-Distill-Qwen-7B2025.1051
Glob. uniBackbone=DS-R1-Distill-Llama-8B2025.1050
Mean Acc.Backbone=DS-R1-Distill-Qwen-7B2025.1050
Loc. non-uniBackbone=DS-R1-Distill-Llama-8B2025.1049
Self-Cert.Backbone=DS-R1-Distill-Qwen-7B2025.1049
Loc. uniBackbone=DS-R1-Distill-Qwen-7B2025.1049
Loc. non-uniBackbone=DS-R1-Distill-Qwen-7B2025.1046
Glob. uniBackbone=DS-R1-Distill-Qwen-7B2025.1046
SATLMLanguage Model=code-davinci-002, Decoding Method=self-consistency decoding2023.0537.4
SATLMLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.0535
COTLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.0523.5
COTLanguage Model=code-davinci-002, Decoding Method=self-consistency decoding2023.0523.1
STANDARDLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.0522