# Logical Fallacy Detection on LOGIC-Climate 10% representative subset (107 instances)

Source: [Logical Fallacy Detection on LOGIC-Climate 10% representative subset (107 instances)](<https://www.sota2.com/research/sota/logical-fallacy-detection-on-logic-climate-10-percent-representative-subset-107-instances>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-23T09:43:06.269Z

Evaluation of model performance on detecting logical fallacies using the 10% representative subset (107 instances) of the LOGIC-Climate dataset.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Logical Fallacy Detection |
| Dataset (including any stated split) | LOGIC-Climate 10% representative subset (107 instances) |
| Metric | Count |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Count](<https://www.sota2.com/research/sota/logical-fallacy-detection-on-logic-climate-10-percent-representative-subset-107-instances.md?metric=Count>): Higher is better; unit: SCALAR.

- [CC+CA](<https://www.sota2.com/research/sota/logical-fallacy-detection-on-logic-climate-10-percent-representative-subset-107-instances.md?metric=CC%2BCA>): Higher is better; unit: PERCENT.

- [CC](<https://www.sota2.com/research/sota/logical-fallacy-detection-on-logic-climate-10-percent-representative-subset-107-instances.md?metric=CC>): Higher is better; unit: PERCENT.

- [CA](<https://www.sota2.com/research/sota/logical-fallacy-detection-on-logic-climate-10-percent-representative-subset-107-instances.md?metric=CA>): Higher is better; unit: PERCENT.

- [UC](<https://www.sota2.com/research/sota/logical-fallacy-detection-on-logic-climate-10-percent-representative-subset-107-instances.md?metric=UC>): Lower is better; unit: PERCENT.

- [UI](<https://www.sota2.com/research/sota/logical-fallacy-detection-on-logic-climate-10-percent-representative-subset-107-instances.md?metric=UI>): Lower is better; unit: PERCENT.

- [VF](<https://www.sota2.com/research/sota/logical-fallacy-detection-on-logic-climate-10-percent-representative-subset-107-instances.md?metric=VF>): Lower is better; unit: PERCENT.

- [NF](<https://www.sota2.com/research/sota/logical-fallacy-detection-on-logic-climate-10-percent-representative-subset-107-instances.md?metric=NF>): Lower is better; unit: PERCENT.

- [SF](<https://www.sota2.com/research/sota/logical-fallacy-detection-on-logic-climate-10-percent-representative-subset-107-instances.md?metric=SF>): Lower is better; unit: PERCENT.

## Recorded results

| Method | Count | Result date | Report paper |
| --- | --- | --- | --- |
| Gemini 2.5 pro | 302 | Not recorded | 6a3a1326206542251d7c70da |
| GPT-5 | 290 | Not recorded | 6a3a1326206542251d7c70da |
| Grok 4 fast | 279 | Not recorded | 6a3a1326206542251d7c70da |
| Kimi-k2 | 276 | Not recorded | 6a3a1326206542251d7c70da |
| Qwen3 235B | 259 | Not recorded | 6a3a1326206542251d7c70da |
| Gemini 2.5 pro | 248 | Not recorded | 6a3a1326206542251d7c70da |
| Qwen3 235B | 237 | Not recorded | 6a3a1326206542251d7c70da |
| GPT-4o | 236 | Not recorded | 6a3a1326206542251d7c70da |
| GPT-oss-120b | 234 | Not recorded | 6a3a1326206542251d7c70da |
| Gemini 2.5 flash | 229 | Not recorded | 6a3a1326206542251d7c70da |
| Claude-sonnet-4.5 | 227 | Not recorded | 6a3a1326206542251d7c70da |
| Grok 4 fast | 215 | Not recorded | 6a3a1326206542251d7c70da |
| Deepseek-r1 | 201 | Not recorded | 6a3a1326206542251d7c70da |
| Gemini 2.5 flash | 195 | Not recorded | 6a3a1326206542251d7c70da |
| Deepseek-chat-v3.1 | 193 | Not recorded | 6a3a1326206542251d7c70da |

### Gemini 2.5 pro

Count: 302. Result ID: 6a3a551afbc373592c36092c.

[Reporting paper](<https://www.sota2.com/research/paper/6a3a1326206542251d7c70da.md>)

Recorded evaluation configuration: {"Mode":"Thinking (T)","Executor":"Claude-sonnet-4.5","Max repair iterations":"4"}. This record does not assert independent reproduction.

### GPT-5

Count: 290. Result ID: 6a3a551afbc373592c360925.

[Reporting paper](<https://www.sota2.com/research/paper/6a3a1326206542251d7c70da.md>)

Recorded evaluation configuration: {"Mode":"Thinking (T)","Executor":"Claude-sonnet-4.5","Max repair iterations":"4"}. This record does not assert independent reproduction.

### Grok 4 fast

Count: 279. Result ID: 6a3a551afbc373592c36092f.

[Reporting paper](<https://www.sota2.com/research/paper/6a3a1326206542251d7c70da.md>)

Recorded evaluation configuration: {"Mode":"Non-Thinking (NT)","Executor":"Claude-sonnet-4.5","Max repair iterations":"4"}. This record does not assert independent reproduction.

### Kimi-k2

Count: 276. Result ID: 6a3a551afbc373592c360929.

[Reporting paper](<https://www.sota2.com/research/paper/6a3a1326206542251d7c70da.md>)

Recorded evaluation configuration: {"Mode":"Thinking (T)","Executor":"Claude-sonnet-4.5","Max repair iterations":"4"}. This record does not assert independent reproduction.

### Qwen3 235B

Count: 259. Result ID: 6a3a551afbc373592c360931.

[Reporting paper](<https://www.sota2.com/research/paper/6a3a1326206542251d7c70da.md>)

Recorded evaluation configuration: {"Mode":"Non-Thinking (NT)","Executor":"Claude-sonnet-4.5","Max repair iterations":"4"}. This record does not assert independent reproduction.

### Gemini 2.5 pro

Count: 248. Result ID: 6a3a551afbc373592c36092d.

[Reporting paper](<https://www.sota2.com/research/paper/6a3a1326206542251d7c70da.md>)

Recorded evaluation configuration: {"Mode":"Non-Thinking (NT)","Executor":"Claude-sonnet-4.5","Max repair iterations":"4"}. This record does not assert independent reproduction.

### Qwen3 235B

Count: 237. Result ID: 6a3a551afbc373592c360930.

[Reporting paper](<https://www.sota2.com/research/paper/6a3a1326206542251d7c70da.md>)

Recorded evaluation configuration: {"Mode":"Thinking (T)","Executor":"Claude-sonnet-4.5","Max repair iterations":"4"}. This record does not assert independent reproduction.

### GPT-4o

Count: 236. Result ID: 6a3a551afbc373592c360932.

[Reporting paper](<https://www.sota2.com/research/paper/6a3a1326206542251d7c70da.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75681e209131cea67a98.md>)

Recorded evaluation configuration: {"Mode":"Non-Thinking (NT)","Executor":"Claude-sonnet-4.5","Max repair iterations":"4"}. This record does not assert independent reproduction.

### GPT-oss-120b

Count: 234. Result ID: 6a3a551afbc373592c360926.

[Reporting paper](<https://www.sota2.com/research/paper/6a3a1326206542251d7c70da.md>)

Recorded evaluation configuration: {"Mode":"Thinking (T)","Executor":"Claude-sonnet-4.5","Max repair iterations":"4"}. This record does not assert independent reproduction.

### Gemini 2.5 flash

Count: 229. Result ID: 6a3a551afbc373592c36092a.

[Reporting paper](<https://www.sota2.com/research/paper/6a3a1326206542251d7c70da.md>)

Recorded evaluation configuration: {"Mode":"Thinking (T)","Executor":"Claude-sonnet-4.5","Max repair iterations":"4"}. This record does not assert independent reproduction.

### Claude-sonnet-4.5

Count: 227. Result ID: 6a3a551afbc373592c360924.

[Reporting paper](<https://www.sota2.com/research/paper/6a3a1326206542251d7c70da.md>)

Recorded evaluation configuration: {"Mode":"Thinking (T)","Executor":"Claude-sonnet-4.5","Max repair iterations":"4"}. This record does not assert independent reproduction.

### Grok 4 fast

Count: 215. Result ID: 6a3a551afbc373592c36092e.

[Reporting paper](<https://www.sota2.com/research/paper/6a3a1326206542251d7c70da.md>)

Recorded evaluation configuration: {"Mode":"Thinking (T)","Executor":"Claude-sonnet-4.5","Max repair iterations":"4"}. This record does not assert independent reproduction.

### Deepseek-r1

Count: 201. Result ID: 6a3a551afbc373592c360928.

[Reporting paper](<https://www.sota2.com/research/paper/6a3a1326206542251d7c70da.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

Recorded evaluation configuration: {"Mode":"Thinking (T)","Executor":"Claude-sonnet-4.5","Max repair iterations":"4"}. This record does not assert independent reproduction.

### Gemini 2.5 flash

Count: 195. Result ID: 6a3a551afbc373592c36092b.

[Reporting paper](<https://www.sota2.com/research/paper/6a3a1326206542251d7c70da.md>)

Recorded evaluation configuration: {"Mode":"Non-Thinking (NT)","Executor":"Claude-sonnet-4.5","Max repair iterations":"4"}. This record does not assert independent reproduction.

### Deepseek-chat-v3.1

Count: 193. Result ID: 6a3a551afbc373592c360927.

[Reporting paper](<https://www.sota2.com/research/paper/6a3a1326206542251d7c70da.md>)

Recorded evaluation configuration: {"Mode":"Thinking (T)","Executor":"Claude-sonnet-4.5","Max repair iterations":"4"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
