Synthetic Transformer Shapes

Synthetic Transformer Shapes Query-Key Q ⊗ K⊤

9

Synthetic Transformer Shapes Attention-Value Att ⊗ V

9