Thematic fit estimation on McRae
0.66Spearman CorrelationStep-by-Step Prompting
Evaluation Results
| Method | Links | |
|---|---|---|
| Step-by-Step PromptingBackbone=GPT4-Turbo, Experiment ID=Exp.3.2, Input Form=Lemma Tuples, Output Form=Categorical2024.10 | 0.66 | |
| Step-by-Step PromptingBackbone=GPT4-Turbo, Experiment ID=Exp.3.1, Input Form=Lemma Tuples, Output Form=Numeric2024.10 | 0.65 | |
| GPT4.1 (Exp.1.1)Reasoning=Standard Prompting, Input=Lemma Tuples, Output=Numeric2024.10 | 0.64 | |
| GPT4.1 (Exp.3.1)Reasoning=Step-by-Step Prompting, Input=Lemma Tuples, Output=Numeric2024.10 | 0.64 | |
| GPT4.1 (Exp.3.2)Reasoning=Step-by-Step Prompting, Input=Lemma Tuples, Output=Categorical2024.10 | 0.63 | |
| Step-by-Step PromptingBackbone=GPT4-Turbo, Experiment ID=Exp.4.2, Input Form=Generated Sentences, Output Form=Categorical2024.10 | 0.59 | |
| GPT4.1 (Exp.1.2)Reasoning=Standard Prompting, Input=Lemma Tuples, Output=Categorical2024.10 | 0.58 | |
| Simple PromptingBackbone=GPT4-Turbo, Experiment ID=Exp.1.2, Input Form=Lemma Tuples, Output Form=Categorical2024.10 | 0.58 | |
| Step-by-Step PromptingBackbone=GPT4-Turbo, Experiment ID=Exp.4.1, Input Form=Generated Sentences, Output Form=Numeric2024.10 | 0.58 | |
| Simple PromptingBackbone=GPT4-Turbo, Experiment ID=Exp.2.2, Input Form=Generated Sentences, Output Form=Categorical2024.10 | 0.56 | |
| Simple PromptingBackbone=GPT4-Turbo, Experiment ID=Exp.2.1, Input Form=Generated Sentences, Output Form=Numeric2024.10 | 0.55 | |
| Simple PromptingBackbone=GPT4-Turbo, Experiment ID=Exp.1.1, Input Form=Lemma Tuples, Output Form=Numeric2024.10 | 0.54 | |
| Exp.4.1Backbone=Qwen2024.10 | 0.51 | |
| Exp.2.1Backbone=Qwen2024.10 | 0.48 | |
| Exp.4.2Backbone=Qwen2024.10 | 0.47 | |
| GPT4.1 (Exp.4.2)Reasoning=Step-by-Step Prompting, Input=Generated Sentences, Output=Categorical2024.10 | 0.46 | |
| 20% subset v22024.10 | 0.44 | |
| GPT4.1 (Exp.4.1)Reasoning=Step-by-Step Prompting, Input=Generated Sentences, Output=Numeric2024.10 | 0.44 | |
| 20% subset v2Method ID=B22024.10 | 0.44 | |
| 20% subset v2Label=B22024.10 | 0.44 | |
| ResRoFA-MT2024.10 | 0.43 | |
| Glove-sharedtuning=tuned, data=20% subset2024.10 | 0.43 | |
| ResRoFA-MTMethod ID=B12024.10 | 0.43 | |
| Glove-shared tuned with 20%MMethod ID=B3c2024.10 | 0.43 | |
| ResRoFA-MTLabel=B12024.10 | 0.43 | |
| Glove-sharedLabel=B3c, Tuning Status=tuned, Training Subset=20%M2024.10 | 0.43 | |
| Exp.1.1Backbone=Qwen2024.10 | 0.42 | |
| GPT4.1 (Exp.2.1)Reasoning=Standard Prompting, Input=Generated Sentences, Output=Numeric2024.10 | 0.4 | |
| Exp.3.1Backbone=Qwen2024.10 | 0.4 | |
| Exp.2.2Backbone=Qwen2024.10 | 0.39 | |
| NN RF2024.10 | 0.38 | |
| NN RFMethod ID=B02024.10 | 0.38 | |
| NN RFLabel=B02024.10 | 0.38 | |
| GPT4.1 (Exp.2.2)Reasoning=Standard Prompting, Input=Generated Sentences, Output=Categorical2024.10 | 0.37 | |
| GSD20152024.10 | 0.36 | |
| GSD2015Method ID=BG2024.10 | 0.36 | |
| GSD2015Label=BG2024.10 | 0.36 | |
| Exp.3.2Backbone=Qwen2024.10 | 0.35 | |
| Exp.1.2Backbone=Qwen2024.10 | 0.34 | |
| Glove-sharedtuning=tuned2024.10 | 0.33 | |
| Glove-shared tunedMethod ID=B3b2024.10 | 0.33 | |
| Glove-sharedLabel=B3b, Tuning Status=tuned2024.10 | 0.33 | |
| Glove-sharedtuning=not-tuned2024.10 | 0.31 | |
| Glove-shared not-tunedMethod ID=B3a2024.10 | 0.31 | |
| Glove-sharedLabel=B3a, Tuning Status=not-tuned2024.10 | 0.31 |