Natural Language Generation on MIMIC-IV-ECG (test)
0.733BLEU-1LLaMA-3-Instruct
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| LLaMA-3-InstructSIZE=8B, fine-tuned=true2024.03 | 0.733 | 0.686 | 0.648 | 0.61 | 0.799 | 0.773 | 0.795 | 0.686 | 5.78 | |
| Mistral-InstructSIZE=7B, fine-tuned=true2024.03 | 0.714 | 0.665 | 0.619 | 0.576 | 0.768 | 0.751 | 0.762 | 0.667 | 5.62 | |
| LLaMA-2-InstructSIZE=7B, fine-tuned=true2024.03 | 0.706 | 0.662 | 0.622 | 0.581 | 0.775 | 0.745 | 0.768 | 0.664 | 5.55 | |
| MistralSIZE=7B, fine-tuned=true2024.03 | 0.697 | 0.659 | 0.611 | 0.571 | 0.763 | 0.74 | 0.765 | 0.658 | 5.48 | |
| LLaMA-1SIZE=7B, fine-tuned=true2024.03 | 0.685 | 0.648 | 0.615 | 0.543 | 0.761 | 0.724 | 0.742 | 0.642 | 5.26 | |
| GPT-JSIZE=6B, fine-tuned=true2024.03 | 0.676 | 0.628 | 0.584 | 0.542 | 0.756 | 0.721 | 0.744 | 0.632 | 5.23 | |
| OPTSIZE=6.7B, fine-tuned=true2024.03 | 0.673 | 0.616 | 0.598 | 0.532 | 0.755 | 0.732 | 0.743 | 0.631 | 5.32 | |
| BLOOMSIZE=7B, fine-tuned=true2024.03 | 0.669 | 0.624 | 0.591 | 0.55 | 0.758 | 0.725 | 0.745 | 0.639 | 5.19 | |
| GPT-NeoXSIZE=20B, fine-tuned=true2024.03 | 0.645 | 0.588 | 0.539 | 0.523 | 0.719 | 0.701 | 0.712 | 0.622 | 4.92 | |
| GPT-NeoSIZE=2.7B, fine-tuned=true2024.03 | 0.631 | 0.579 | 0.534 | 0.489 | 0.727 | 0.689 | 0.715 | 0.592 | 4.81 | |
| GPT2-LargeSIZE=774M, fine-tuned=true2024.03 | 0.614 | 0.563 | 0.49 | 0.476 | 0.595 | 0.571 | 0.585 | 0.538 | 4.21 | |
| GPT2-MediumSIZE=345M, fine-tuned=true2024.03 | 0.576 | 0.527 | 0.456 | 0.425 | 0.551 | 0.523 | 0.544 | 0.512 | 3.7 |