Objective-C Type Recovery on MOTIF-Bench benchmark leaderboard · SOTA2 Research

Objective-C Type Recovery on MOTIF-Bench

94.7Avg AccuracyClaude Sonnet 4

Evaluation Results

Method	Links
Claude Sonnet 42026.01		94.7	86.7	72.3	77.4	0.1	100	0	100	83.2	100	95.5
Gemini-2.0 Flash2026.01		88.7	82.9	60.8	66.8	6.5	100	0	100	79.2	100	75.5
Claude Sonnet 42026.01		78.7	80.1	62.8	—	—	—	—	99	65.9	73.8	76
ChatGPT 4o2026.01		63	74.5	51.8	—	—	—	—	100	50.7	52.3	49
Gemini 2.0 flash2026.01		57.3	71.5	48.6	—	—	—	—	100	39.6	43.1	46.6
XAI Grok2026.01		56.4	82.1	73	99.7	5.6	100	0	82.1	50.7	48.1	44.6
Deepseek R12026.01		54.8	79.5	61.1	99.9	53.7	97.7	2.3	96	42	35.8	45.5
GPT-4o2026.01		50	86.3	68.9	99.9	14.4	100	0	99.4	44.3	17	39.2
Deepseek R12026.01		48.1	58.9	43.6	—	—	—	—	71.7	47.9	36.9	39.4
Qwen3 (8B) finetuned2026.01		42.4	83.5	64.7	99.5	16.3	100	0	99.4	32.2	9.6	28.6
gpt-oss-120b2026.01		41.5	68.3	61.6	85.2	40.9	98.3	1.7	81.1	31.1	23.8	29.9
Qwen3 (14B)2026.01		39.1	73.8	49.5	97.7	46.6	100	0	99.4	31.4	1.9	23.8
Llama 3.1 (8B-Instruct) finetuned2026.01		37.6	80.3	58.1	97.9	9.1	100	0	93.6	33	3.8	20.1
gpt-oss-20b2026.01		28.5	57.1	40.2	82.4	30.4	82.6	17.4	68.8	16.7	0	28.6
Phi-3-medium2026.01		26.8	46.1	22.7	0	22.6	—	—	82.2	9.1	3.8	11.9
Static Analysis2026.01		24.5	14.9	0	—	—	—	—	98	0	0	0
Llama 3.1 8B Instruct2026.01		19.4	35.9	19	91.5	10.3	100	0	58.5	7.6	2	9.4