Code Completion on RepoBench-P
0.7305SimilarityCoA
Evaluation Results
| Method | Links | |
|---|---|---|
| CoABackbone=gemini-ultra, Context Window=8k2024.06 | 0.7305 | |
| VanillaBackbone=gemini-ultra, Context Window=32k2024.06 | 0.7304 | |
| RAGBackbone=gemini-ultra, Context Window=8k2024.06 | 0.7294 | |
| CoABackbone=text-unicorn, Context Window=8k2024.06 | 0.6039 | |
| CoABackbone=text-bison, Context Window=8k2024.06 | 0.5825 | |
| VanillaBackbone=text-bison, Context Window=8k2024.06 | 0.563 | |
| RAGBackbone=text-bison, Context Window=8k2024.06 | 0.5563 | |
| VanillaBackbone=text-unicorn, Context Window=8k2024.06 | 0.5387 | |
| RAGBackbone=text-unicorn, Context Window=8k2024.06 | 0.5049 | |
| VanillaBackbone=gemini-ultra, Context Window=8k2024.06 | 0.4909 | |
| AMS-ExpectedKV Compression=Enabled2026.05 | 0.2744 | |
| StreamingLLMKV Compression=Enabled2026.05 | 0.2692 | |
| PyramidKVKV Compression=Enabled2026.05 | 0.241 | |
| Full KVKV Compression=Disabled2026.05 | 0.2404 | |
| AMS-TOVAKV Compression=Enabled2026.05 | 0.2364 | |
| TOVAKV Compression=Enabled2026.05 | 0.224 | |
| AdaKV-ExpE2KV Compression=Enabled2026.05 | 0.1912 |