Formal Conformity Evaluation on Songci 20 Cipai and 6 themes
0.8238Formal Conformity ScoreERNIE 4.5 Turbo
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| ERNIE 4.5 TurboPrompting Strategy=Completion2025.08 | 0.8238 | — | — | |
| ERNIE 4.5 TurboPrompting Strategy=Instruction2025.08 | 0.8154 | — | — | |
| ERNIE 4.5 TurboPrompting Strategy=One-shot2025.08 | 0.8072 | — | — | |
| GPT-4oPrompting Strategy=Zero-shot2025.08 | 0.7862 | — | — | |
| ERNIE 4.5 TurboPrompting Strategy=Chain-of-Thought2025.08 | 0.7857 | — | — | |
| Gemini 2.5 ProPrompting Strategy=Zero-shot2025.08 | 0.7821 | — | — | |
| ERNIE 4.5 TurboPrompting Strategy=Zero-shot2025.08 | 0.7736 | — | — | |
| Gemini 2.5 ProPrompting Strategy=One-shot2025.08 | 0.7715 | — | — | |
| GPT-4oPrompting Strategy=One-shot2025.08 | 0.7659 | — | — | |
| Gemini 2.5 ProPrompting Strategy=Completion2025.08 | 0.765 | — | — | |
| GPT-4oPrompting Strategy=Completion2025.08 | 0.7579 | — | — | |
| Gemini 2.5 ProPrompting Strategy=Instruction2025.08 | 0.749 | — | — | |
| GPT-4oPrompting Strategy=Instruction2025.08 | 0.737 | — | — | |
| Gemini 2.5 ProPrompting Strategy=Chain-of-Thought2025.08 | 0.7234 | — | — | |
| GPT-4oPrompting Strategy=Chain-of-Thought2025.08 | 0.7111 | — | — | |
| Qwen3-32BPrompting Strategy=One-shot2025.08 | 0.6855 | — | — | |
| Qwen3-32BPrompting Strategy=Zero-shot2025.08 | 0.6493 | — | — | |
| Qwen3-8BPrompting Strategy=One-shot2025.08 | 0.6453 | — | — | |
| Qwen3-8BPrompting Strategy=Completion2025.08 | 0.6375 | — | — | |
| Llama-3.3-70B-InstructPrompting Strategy=One-shot2025.08 | 0.6307 | — | — | |
| Qwen3-8BPrompting Strategy=Chain-of-Thought2025.08 | 0.6258 | — | — | |
| Qwen3-32BPrompting Strategy=Chain-of-Thought2025.08 | 0.6203 | — | — | |
| Qwen3-32BPrompting Strategy=Completion2025.08 | 0.6152 | — | — | |
| Qwen3-8BPrompting Strategy=Zero-shot2025.08 | 0.6019 | — | — | |
| Qwen3-4BPrompting Strategy=One-shot2025.08 | 0.5885 | — | — | |
| Llama-3-8B-InstructPrompting Strategy=One-shot2025.08 | 0.5821 | — | — | |
| Qwen3-32BPrompting Strategy=Instruction2025.08 | 0.5651 | — | — | |
| Llama-3.1-8B-InstructPrompting Strategy=One-shot2025.08 | 0.5442 | — | — | |
| Mistral-Small-Instruct-2409Prompting Strategy=One-shot2025.08 | 0.5441 | — | — | |
| Qwen3-0.6BPrompting Strategy=Completion2025.08 | 0.5411 | — | — | |
| Qwen3-8BPrompting Strategy=Instruction2025.08 | 0.5398 | — | — | |
| Qwen3-4BPrompting Strategy=Zero-shot2025.08 | 0.5361 | — | — | |
| Mistral-Small-Instruct-2409Prompting Strategy=Completion2025.08 | 0.5342 | — | — | |
| Llama-3.1-8B-InstructPrompting Strategy=Completion2025.08 | 0.534 | — | — | |
| DeepSeek-R1-0528-Qwen3-8BPrompting Strategy=Zero-shot2025.08 | 0.5134 | — | — | |
| DeepSeek-R1-0528-Qwen3-8BPrompting Strategy=Completion2025.08 | 0.5118 | — | — | |
| Llama-3.3-70B-InstructPrompting Strategy=Completion2025.08 | 0.5116 | — | — | |
| Qwen3-4BPrompting Strategy=Chain-of-Thought2025.08 | 0.5044 | — | — | |
| Qwen3-0.6BPrompting Strategy=One-shot2025.08 | 0.4974 | — | — | |
| Qwen3-4BPrompting Strategy=Completion2025.08 | 0.4954 | — | — | |
| DeepSeek-R1-0528-Qwen3-8BPrompting Strategy=One-shot2025.08 | 0.4909 | — | — | |
| Qwen3-1.7BPrompting Strategy=One-shot2025.08 | 0.4856 | — | — | |
| Llama-3-8B-InstructPrompting Strategy=Completion2025.08 | 0.4774 | — | — | |
| Qwen3-1.7BPrompting Strategy=Completion2025.08 | 0.472 | — | — | |
| Qwen3-4BPrompting Strategy=Instruction2025.08 | 0.4645 | — | — | |
| Llama-3.2-3B-InstructPrompting Strategy=One-shot2025.08 | 0.4602 | — | — | |
| Llama-3.2-3B-InstructPrompting Strategy=Completion2025.08 | 0.4579 | — | — | |
| Qwen3-1.7BPrompting Strategy=Instruction2025.08 | 0.442 | — | — | |
| Mistral-7B-Instruct-v0.3Prompting Strategy=One-shot2025.08 | 0.4313 | — | — | |
| DeepSeek-R1-0528-Qwen3-8BPrompting Strategy=Instruction2025.08 | 0.425 | — | — | |
| DeepSeek-R1-Distill-Llama-70BPrompting Strategy=One-shot2025.08 | 0.4171 | — | — | |
| Mixtral-8x7B-Instruct-v0.1Prompting Strategy=One-shot2025.08 | 0.411 | — | — | |
| Qwen3-1.7BPrompting Strategy=Chain-of-Thought2025.08 | 0.4072 | — | — | |
| Qwen3-0.6BPrompting Strategy=Instruction2025.08 | 0.4052 | — | — | |
| Mistral-Small-Instruct-2409Prompting Strategy=Instruction2025.08 | 0.4042 | — | — | |
| Llama-3.1-8B-InstructPrompting Strategy=Chain-of-Thought2025.08 | 0.4013 | — | — | |
| Qwen3-1.7BPrompting Strategy=Zero-shot2025.08 | 0.3988 | — | — | |
| Mistral-7B-Instruct-v0.3Prompting Strategy=Completion2025.08 | 0.3984 | — | — | |
| Qwen3-0.6BPrompting Strategy=Chain-of-Thought2025.08 | 0.3977 | — | — | |
| Llama-3-8B-InstructPrompting Strategy=Instruction2025.08 | 0.3969 | — | — | |
| Qwen3-0.6BPrompting Strategy=Zero-shot2025.08 | 0.3876 | — | — | |
| Llama-3.1-8B-InstructPrompting Strategy=Instruction2025.08 | 0.3872 | — | — | |
| DeepSeek-R1-Distill-Llama-70BPrompting Strategy=Completion2025.08 | 0.381 | — | — | |
| Llama-3.3-70B-InstructPrompting Strategy=Instruction2025.08 | 0.3712 | — | — | |
| Mistral-Small-Instruct-2409Prompting Strategy=Zero-shot2025.08 | 0.3711 | — | — | |
| Llama-3.2-3B-InstructPrompting Strategy=Chain-of-Thought2025.08 | 0.3647 | — | — | |
| Llama-3.2-3B-InstructPrompting Strategy=Zero-shot2025.08 | 0.3615 | — | — | |
| DeepSeek-R1-Distill-Llama-70BPrompting Strategy=Instruction2025.08 | 0.3595 | — | — | |
| Llama-3.3-70B-InstructPrompting Strategy=Zero-shot2025.08 | 0.3588 | — | — | |
| Llama-3-8B-InstructPrompting Strategy=Chain-of-Thought2025.08 | 0.357 | — | — | |
| Mixtral-8x7B-Instruct-v0.1Prompting Strategy=Completion2025.08 | 0.3563 | — | — | |
| DeepSeek-R1-Distill-Llama-70BPrompting Strategy=Chain-of-Thought2025.08 | 0.3558 | — | — | |
| Llama-3.2-3B-InstructPrompting Strategy=Instruction2025.08 | 0.3535 | — | — | |
| Llama-3.2-1B-InstructPrompting Strategy=Instruction2025.08 | 0.3532 | — | — | |
| Llama-3.3-70B-InstructPrompting Strategy=Chain-of-Thought2025.08 | 0.3531 | — | — | |
| Llama-3.2-1B-InstructPrompting Strategy=One-shot2025.08 | 0.353 | — | — | |
| Mistral-Small-Instruct-2409Prompting Strategy=Chain-of-Thought2025.08 | 0.3526 | — | — | |
| Llama-3.1-8B-InstructPrompting Strategy=Zero-shot2025.08 | 0.352 | — | — | |
| Mistral-7B-Instruct-v0.3Prompting Strategy=Chain-of-Thought2025.08 | 0.3509 | — | — | |
| Llama-3-8B-InstructPrompting Strategy=Zero-shot2025.08 | 0.3413 | — | — | |
| Llama-3.2-1B-InstructPrompting Strategy=Completion2025.08 | 0.3404 | — | — | |
| DeepSeek-R1-Distill-Llama-70BPrompting Strategy=Zero-shot2025.08 | 0.3375 | — | — | |
| Mistral-7B-Instruct-v0.3Prompting Strategy=Zero-shot2025.08 | 0.336 | — | — | |
| Llama-3.2-1B-InstructPrompting Strategy=Zero-shot2025.08 | 0.3346 | — | — | |
| Llama-3.2-1B-InstructPrompting Strategy=Chain-of-Thought2025.08 | 0.3346 | — | — | |
| Mistral-7B-Instruct-v0.3Prompting Strategy=Instruction2025.08 | 0.3187 | — | — | |
| Mixtral-8x7B-Instruct-v0.1Prompting Strategy=Chain-of-Thought2025.08 | 0.3111 | — | — | |
| Mixtral-8x7B-Instruct-v0.1Prompting Strategy=Instruction2025.08 | 0.3034 | — | — | |
| Mixtral-8x7B-Instruct-v0.1Prompting Strategy=Zero-shot2025.08 | 0.2804 | — | — | |
| DeepSeek-R1-0528-Qwen3-8BPrompting Strategy=Chain-of-Thought2025.08 | 0.2761 | — | — | |
| DeepSeek-R1-0528-Qwen3-8BPrompting Strategy=Average2025.08 | — | 44.74 | — | |
| DeepSeek-R1-0528-Qwen3-8BPrompting Strategy=Best (zero-shot)2025.08 | — | — | 51.34 | |
| DeepSeek-R1-Distill-Llama-70BPrompting Strategy=Average2025.08 | — | 37.82 | — | |
| DeepSeek-R1-Distill-Llama-70BPrompting Strategy=Best (one-shot)2025.08 | — | — | 41.71 | |
| ERNIE 4.5 TurboPrompting Strategy=Average2025.08 | — | 79.71 | — | |
| ERNIE 4.5 TurboPrompting Strategy=Best (completion)2025.08 | — | — | 82.38 | |
| Gemini 2.5 ProPrompting Strategy=Average2025.08 | — | 76.52 | — | |
| Gemini 2.5 ProPrompting Strategy=Best (zero-shot)2025.08 | — | — | 78.21 | |
| GPT-4oPrompting Strategy=Average2025.08 | — | 75.16 | — | |
| GPT-4oPrompting Strategy=Best (zero-shot)2025.08 | — | — | 78.62 |