List your company
ResearchFormerly WizWand

Pragmatic Reasoning on PRAGMEGA (test)

94AccuracyGricean Prompting
57.80867.20476.685.996Oct 30, 2025
Updated 5mo ago

Evaluation results

MethodLinks
Gricean PromptingBackbone=gpt-4o, Prompt Setting=Proposed, Prompting Strategy=grice2025.1094
Relevance Theory PromptingBackbone=gpt-4o, Prompt Setting=Proposed, Prompting Strategy=relevance2025.1093.5
Gricean PromptingBackbone=gpt-4o, Prompt Setting=Short, Prompting Strategy=grice2025.1090.2
Relevance Theory PromptingBackbone=gpt-4o, Prompt Setting=Short, Prompting Strategy=relevance2025.1089.2
Chain-of-ThoughtBackbone=gpt-4o, Prompt Setting=Baseline, Prompting Strategy=cot2025.1087.7
Gricean PromptingBackbone=phi-4, Prompt Setting=Proposed, Prompting Strategy=grice2025.1087.7
Relevance Theory PromptingBackbone=phi-4, Prompt Setting=Proposed, Prompting Strategy=relevance2025.1086.7
Simple PromptingBackbone=gpt-4o, Prompt Setting=Baseline, Prompting Strategy=simple2025.1084.2
Gricean PromptingBackbone=Qwen2.5-14B-Instruct, Prompt Setting=Proposed, Prompting Strategy=grice2025.1079.6
Gricean PromptingBackbone=phi-4, Prompt Setting=Short, Prompting Strategy=grice2025.1079.6
Relevance Theory PromptingBackbone=phi-4, Prompt Setting=Short, Prompting Strategy=relevance2025.1079.2
Simple PromptingBackbone=phi-4, Prompt Setting=Baseline, Prompting Strategy=simple2025.1078.1
Relevance Theory PromptingBackbone=gpt-4o-mini, Prompt Setting=Proposed, Prompting Strategy=relevance2025.1077.9
Chain-of-ThoughtBackbone=phi-4, Prompt Setting=Baseline, Prompting Strategy=cot2025.1077.5
Gricean PromptingBackbone=gpt-4o-mini, Prompt Setting=Proposed, Prompting Strategy=grice2025.1077.1
Relevance Theory PromptingBackbone=Qwen2.5-14B-Instruct, Prompt Setting=Proposed, Prompting Strategy=relevance2025.1075
Relevance Theory PromptingBackbone=gpt-4o-mini, Prompt Setting=Short, Prompting Strategy=relevance2025.1073.8
Gricean PromptingBackbone=gpt-4o-mini, Prompt Setting=Short, Prompting Strategy=grice2025.1072.3
Gricean PromptingBackbone=Qwen2.5-14B-Instruct, Prompt Setting=Short, Prompting Strategy=grice2025.1072.3
Relevance Theory PromptingBackbone=Qwen2.5-14B-Instruct, Prompt Setting=Short, Prompting Strategy=relevance2025.1072.1
Simple PromptingBackbone=Qwen2.5-14B-Instruct, Prompt Setting=Baseline, Prompting Strategy=simple2025.1071.7
Simple PromptingBackbone=gpt-4o-mini, Prompt Setting=Baseline, Prompting Strategy=simple2025.1069.6
Chain-of-ThoughtBackbone=Qwen2.5-14B-Instruct, Prompt Setting=Baseline, Prompting Strategy=cot2025.1069.6
Chain-of-ThoughtBackbone=gpt-4o-mini, Prompt Setting=Baseline, Prompting Strategy=cot2025.1069.4
Relevance Theory PromptingBackbone=Qwen2.5-7B-Instruct, Prompt Setting=Proposed, Prompting Strategy=relevance2025.1068.3
Gricean PromptingBackbone=Qwen2.5-7B-Instruct, Prompt Setting=Proposed, Prompting Strategy=grice2025.1066
Gricean PromptingBackbone=Llama-3.1-8B-Instruct, Prompt Setting=Proposed, Prompting Strategy=grice2025.1065.6
Gricean PromptingBackbone=Llama-3.1-8B-Instruct, Prompt Setting=Short, Prompting Strategy=grice2025.1063.5
Relevance Theory PromptingBackbone=Llama-3.1-8B-Instruct, Prompt Setting=Short, Prompting Strategy=relevance2025.1062.3
Simple PromptingBackbone=Llama-3.1-8B-Instruct, Prompt Setting=Baseline, Prompting Strategy=simple2025.1062.1
Chain-of-ThoughtBackbone=Qwen2.5-7B-Instruct, Prompt Setting=Baseline, Prompting Strategy=cot2025.1061.2
Simple PromptingBackbone=Qwen2.5-7B-Instruct, Prompt Setting=Baseline, Prompting Strategy=simple2025.1061
Relevance Theory PromptingBackbone=Llama-3.1-8B-Instruct, Prompt Setting=Proposed, Prompting Strategy=relevance2025.1060.4
Relevance Theory PromptingBackbone=Qwen2.5-7B-Instruct, Prompt Setting=Short, Prompting Strategy=relevance2025.1060.4
Chain-of-ThoughtBackbone=Llama-3.1-8B-Instruct, Prompt Setting=Baseline, Prompting Strategy=cot2025.1059.2
Gricean PromptingBackbone=Qwen2.5-7B-Instruct, Prompt Setting=Short, Prompting Strategy=grice2025.1059.2