List your company
ResearchFormerly WizWand

Reasoning on Python Puzzles

57.3AccuracyRoT
29.01236.35643.751.044Oct 16, 2024
Updated 8mo ago

Evaluation results

MethodLinks
RoTBackbone=GPT-4, Shots=2-shot2024.1057.3
RoTBackbone=GPT-4, Shots=1-shot2024.1054.6
BOTBackbone=GPT-4, Shots=2-shot2024.1052.8
BOTBackbone=GPT-4, Shots=1-shot2024.1051.8
RoTBackbone=GPT-3.5-turbo, Shots=2-shot2024.1050.4
RoTBackbone=GPT-3.5-turbo, Shots=1-shot2024.1050.2
BOTBackbone=GPT-3.5-turbo, Shots=1-shot2024.1048.2
BOTBackbone=GPT-3.5-turbo, Shots=2-shot2024.1048
Meta-promptingBackbone=GPT-4, Shots=2-shot2024.1045.6
TOTBackbone=GPT-4, Shots=2-shot2024.1045.2
Meta-promptingBackbone=GPT-4, Shots=1-shot2024.1045
TOTBackbone=GPT-4, Shots=1-shot2024.1043.8
GOTBackbone=GPT-4, Shots=2-shot2024.1043.5
GOTBackbone=GPT-4, Shots=1-shot2024.1042.6
Meta-promptingBackbone=GPT-3.5-turbo, Shots=2-shot2024.1041.1
Meta-promptingBackbone=GPT-3.5-turbo, Shots=1-shot2024.1040.2
TOTBackbone=GPT-3.5-turbo, Shots=2-shot2024.1039.5
TOTBackbone=GPT-3.5-turbo, Shots=1-shot2024.1038.2
GOTBackbone=GPT-3.5-turbo, Shots=2-shot2024.1036.7
COTBackbone=GPT-4, Shots=2-shot2024.1035.9
GOTBackbone=GPT-3.5-turbo, Shots=1-shot2024.1035.3
COTBackbone=GPT-4, Shots=1-shot2024.1035.3
COTBackbone=GPT-3.5-turbo, Shots=2-shot2024.1030.6
COTBackbone=GPT-3.5-turbo, Shots=1-shot2024.1030.1