List your company
ResearchFormerly WizWand

Formal Conformity Evaluation on Songci 20 Cipai and 6 themes

0.8238Formal Conformity ScoreERNIE 4.5 Turbo
0.2541920.4020710.549950.697829Aug 4, 2025
Updated 8mo ago

Evaluation results

MethodLinks
ERNIE 4.5 TurboPrompting Strategy=Completion2025.080.8238——
ERNIE 4.5 TurboPrompting Strategy=Instruction2025.080.8154——
ERNIE 4.5 TurboPrompting Strategy=One-shot2025.080.8072——
GPT-4oPrompting Strategy=Zero-shot2025.080.7862——
ERNIE 4.5 TurboPrompting Strategy=Chain-of-Thought2025.080.7857——
Gemini 2.5 ProPrompting Strategy=Zero-shot2025.080.7821——
ERNIE 4.5 TurboPrompting Strategy=Zero-shot2025.080.7736——
Gemini 2.5 ProPrompting Strategy=One-shot2025.080.7715——
GPT-4oPrompting Strategy=One-shot2025.080.7659——
Gemini 2.5 ProPrompting Strategy=Completion2025.080.765——
GPT-4oPrompting Strategy=Completion2025.080.7579——
Gemini 2.5 ProPrompting Strategy=Instruction2025.080.749——
GPT-4oPrompting Strategy=Instruction2025.080.737——
Gemini 2.5 ProPrompting Strategy=Chain-of-Thought2025.080.7234——
GPT-4oPrompting Strategy=Chain-of-Thought2025.080.7111——
Qwen3-32BPrompting Strategy=One-shot2025.080.6855——
Qwen3-32BPrompting Strategy=Zero-shot2025.080.6493——
Qwen3-8BPrompting Strategy=One-shot2025.080.6453——
Qwen3-8BPrompting Strategy=Completion2025.080.6375——
Llama-3.3-70B-InstructPrompting Strategy=One-shot2025.080.6307——
Qwen3-8BPrompting Strategy=Chain-of-Thought2025.080.6258——
Qwen3-32BPrompting Strategy=Chain-of-Thought2025.080.6203——
Qwen3-32BPrompting Strategy=Completion2025.080.6152——
Qwen3-8BPrompting Strategy=Zero-shot2025.080.6019——
Qwen3-4BPrompting Strategy=One-shot2025.080.5885——
Llama-3-8B-InstructPrompting Strategy=One-shot2025.080.5821——
Qwen3-32BPrompting Strategy=Instruction2025.080.5651——
Llama-3.1-8B-InstructPrompting Strategy=One-shot2025.080.5442——
Mistral-Small-Instruct-2409Prompting Strategy=One-shot2025.080.5441——
Qwen3-0.6BPrompting Strategy=Completion2025.080.5411——
Qwen3-8BPrompting Strategy=Instruction2025.080.5398——
Qwen3-4BPrompting Strategy=Zero-shot2025.080.5361——
Mistral-Small-Instruct-2409Prompting Strategy=Completion2025.080.5342——
Llama-3.1-8B-InstructPrompting Strategy=Completion2025.080.534——
DeepSeek-R1-0528-Qwen3-8BPrompting Strategy=Zero-shot2025.080.5134——
DeepSeek-R1-0528-Qwen3-8BPrompting Strategy=Completion2025.080.5118——
Llama-3.3-70B-InstructPrompting Strategy=Completion2025.080.5116——
Qwen3-4BPrompting Strategy=Chain-of-Thought2025.080.5044——
Qwen3-0.6BPrompting Strategy=One-shot2025.080.4974——
Qwen3-4BPrompting Strategy=Completion2025.080.4954——
DeepSeek-R1-0528-Qwen3-8BPrompting Strategy=One-shot2025.080.4909——
Qwen3-1.7BPrompting Strategy=One-shot2025.080.4856——
Llama-3-8B-InstructPrompting Strategy=Completion2025.080.4774——
Qwen3-1.7BPrompting Strategy=Completion2025.080.472——
Qwen3-4BPrompting Strategy=Instruction2025.080.4645——
Llama-3.2-3B-InstructPrompting Strategy=One-shot2025.080.4602——
Llama-3.2-3B-InstructPrompting Strategy=Completion2025.080.4579——
Qwen3-1.7BPrompting Strategy=Instruction2025.080.442——
Mistral-7B-Instruct-v0.3Prompting Strategy=One-shot2025.080.4313——
DeepSeek-R1-0528-Qwen3-8BPrompting Strategy=Instruction2025.080.425——
DeepSeek-R1-Distill-Llama-70BPrompting Strategy=One-shot2025.080.4171——
Mixtral-8x7B-Instruct-v0.1Prompting Strategy=One-shot2025.080.411——
Qwen3-1.7BPrompting Strategy=Chain-of-Thought2025.080.4072——
Qwen3-0.6BPrompting Strategy=Instruction2025.080.4052——
Mistral-Small-Instruct-2409Prompting Strategy=Instruction2025.080.4042——
Llama-3.1-8B-InstructPrompting Strategy=Chain-of-Thought2025.080.4013——
Qwen3-1.7BPrompting Strategy=Zero-shot2025.080.3988——
Mistral-7B-Instruct-v0.3Prompting Strategy=Completion2025.080.3984——
Qwen3-0.6BPrompting Strategy=Chain-of-Thought2025.080.3977——
Llama-3-8B-InstructPrompting Strategy=Instruction2025.080.3969——
Qwen3-0.6BPrompting Strategy=Zero-shot2025.080.3876——
Llama-3.1-8B-InstructPrompting Strategy=Instruction2025.080.3872——
DeepSeek-R1-Distill-Llama-70BPrompting Strategy=Completion2025.080.381——
Llama-3.3-70B-InstructPrompting Strategy=Instruction2025.080.3712——
Mistral-Small-Instruct-2409Prompting Strategy=Zero-shot2025.080.3711——
Llama-3.2-3B-InstructPrompting Strategy=Chain-of-Thought2025.080.3647——
Llama-3.2-3B-InstructPrompting Strategy=Zero-shot2025.080.3615——
DeepSeek-R1-Distill-Llama-70BPrompting Strategy=Instruction2025.080.3595——
Llama-3.3-70B-InstructPrompting Strategy=Zero-shot2025.080.3588——
Llama-3-8B-InstructPrompting Strategy=Chain-of-Thought2025.080.357——
Mixtral-8x7B-Instruct-v0.1Prompting Strategy=Completion2025.080.3563——
DeepSeek-R1-Distill-Llama-70BPrompting Strategy=Chain-of-Thought2025.080.3558——
Llama-3.2-3B-InstructPrompting Strategy=Instruction2025.080.3535——
Llama-3.2-1B-InstructPrompting Strategy=Instruction2025.080.3532——
Llama-3.3-70B-InstructPrompting Strategy=Chain-of-Thought2025.080.3531——
Llama-3.2-1B-InstructPrompting Strategy=One-shot2025.080.353——
Mistral-Small-Instruct-2409Prompting Strategy=Chain-of-Thought2025.080.3526——
Llama-3.1-8B-InstructPrompting Strategy=Zero-shot2025.080.352——
Mistral-7B-Instruct-v0.3Prompting Strategy=Chain-of-Thought2025.080.3509——
Llama-3-8B-InstructPrompting Strategy=Zero-shot2025.080.3413——
Llama-3.2-1B-InstructPrompting Strategy=Completion2025.080.3404——
DeepSeek-R1-Distill-Llama-70BPrompting Strategy=Zero-shot2025.080.3375——
Mistral-7B-Instruct-v0.3Prompting Strategy=Zero-shot2025.080.336——
Llama-3.2-1B-InstructPrompting Strategy=Zero-shot2025.080.3346——
Llama-3.2-1B-InstructPrompting Strategy=Chain-of-Thought2025.080.3346——
Mistral-7B-Instruct-v0.3Prompting Strategy=Instruction2025.080.3187——
Mixtral-8x7B-Instruct-v0.1Prompting Strategy=Chain-of-Thought2025.080.3111——
Mixtral-8x7B-Instruct-v0.1Prompting Strategy=Instruction2025.080.3034——
Mixtral-8x7B-Instruct-v0.1Prompting Strategy=Zero-shot2025.080.2804——
DeepSeek-R1-0528-Qwen3-8BPrompting Strategy=Chain-of-Thought2025.080.2761——
DeepSeek-R1-0528-Qwen3-8BPrompting Strategy=Average2025.08—44.74—
DeepSeek-R1-0528-Qwen3-8BPrompting Strategy=Best (zero-shot)2025.08——51.34
DeepSeek-R1-Distill-Llama-70BPrompting Strategy=Average2025.08—37.82—
DeepSeek-R1-Distill-Llama-70BPrompting Strategy=Best (one-shot)2025.08——41.71
ERNIE 4.5 TurboPrompting Strategy=Average2025.08—79.71—
ERNIE 4.5 TurboPrompting Strategy=Best (completion)2025.08——82.38
Gemini 2.5 ProPrompting Strategy=Average2025.08—76.52—
Gemini 2.5 ProPrompting Strategy=Best (zero-shot)2025.08——78.21
GPT-4oPrompting Strategy=Average2025.08—75.16—
GPT-4oPrompting Strategy=Best (zero-shot)2025.08——78.62