# Chime-in Timing on MUIR (test) Source: [Chime-in Timing on MUIR (test)]() Collection index: [llms.txt]() Data updated: 2026-03-04T13:07:02.837Z Evaluation of Chime-in Timing performance on the test split of the MUIR dataset, reporting Accuracy, F1 score, and Weighted score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Chime-in Timing | | Dataset (including any stated split) | MUIR (test) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. - [F1 Score](): Higher is better; unit: PERCENT. - [Weighted Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | Human Evaluator | 86.42 | Not recorded | 69a66c132d603e6de3e3cd80 | | Qwen-3 | 83.4 | Not recorded | 69a66c132d603e6de3e3cd80 | | Llama-3.1-Instruct | 78.47 | Not recorded | 69a66c132d603e6de3e3cd80 | | Llama-3.2-Instruct | 77.8 | Not recorded | 69a66c132d603e6de3e3cd80 | | Gemma-2-it | 77.68 | Not recorded | 69a66c132d603e6de3e3cd80 | | Qwen-2.5-Instruct | 75.36 | Not recorded | 69a66c132d603e6de3e3cd80 | | Gte-large-en-v1.5 | 74.28 | Not recorded | 69a66c132d603e6de3e3cd80 | | Phi-4-Mini-Instruct | 74.07 | Not recorded | 69a66c132d603e6de3e3cd80 | | Gemini-2.5-Pro | 73.66 | Not recorded | 69a66c132d603e6de3e3cd80 | | Qwen-2.5-Instruct | 73.24 | Not recorded | 69a66c132d603e6de3e3cd80 | | Jina-embedding-v3 | 71.6 | Not recorded | 69a66c132d603e6de3e3cd80 | | DeepSeek-V3.2 | 67.28 | Not recorded | 69a66c132d603e6de3e3cd80 | | Bge-m3 | 66.46 | Not recorded | 69a66c132d603e6de3e3cd80 | | Qwen3-Max | 63.58 | Not recorded | 69a66c132d603e6de3e3cd80 | | Random Guess | 59.26 | Not recorded | 69a66c132d603e6de3e3cd80 | | GPT-4o | 58.02 | Not recorded | 69a66c132d603e6de3e3cd80 | | Qwen-3 | 53.8 | Not recorded | 69a66c132d603e6de3e3cd80 | ### Human Evaluator Accuracy: 86.42. Result ID: 69a82e5f6dbb7075b94e8d32. [Reporting paper]() Recorded evaluation configuration: {"Method":"Baseline","Size":"N/A"}. This record does not assert independent reproduction. ### Qwen-3 Accuracy: 83.4. Result ID: 69a82e5f6dbb7075b94e8d3e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method":"SLM + Fine-Tuning","Size":"4B"}. This record does not assert independent reproduction. ### Llama-3.1-Instruct Accuracy: 78.47. Result ID: 69a82e5f6dbb7075b94e8d40. [Reporting paper]() Recorded evaluation configuration: {"Method":"SLM + Fine-Tuning","Size":"8B"}. This record does not assert independent reproduction. ### Llama-3.2-Instruct Accuracy: 77.8. Result ID: 69a82e5f6dbb7075b94e8d3c. [Reporting paper]() Recorded evaluation configuration: {"Method":"SLM + Fine-Tuning","Size":"3B"}. This record does not assert independent reproduction. ### Gemma-2-it Accuracy: 77.68. Result ID: 69a82e5f6dbb7075b94e8d3a. [Reporting paper]() Recorded evaluation configuration: {"Method":"SLM + Fine-Tuning","Size":"2B"}. This record does not assert independent reproduction. ### Qwen-2.5-Instruct Accuracy: 75.36. Result ID: 69a82e5f6dbb7075b94e8d3b. [Reporting paper]() Recorded evaluation configuration: {"Method":"SLM + Fine-Tuning","Size":"3B"}. This record does not assert independent reproduction. ### Gte-large-en-v1.5 Accuracy: 74.28. Result ID: 69a82e5f6dbb7075b94e8d37. [Reporting paper]() Recorded evaluation configuration: {"Method":"Embedding Model + KNN","Size":"0.4B"}. This record does not assert independent reproduction. ### Phi-4-Mini-Instruct Accuracy: 74.07. Result ID: 69a82e5f6dbb7075b94e8d3d. [Reporting paper]() Recorded evaluation configuration: {"Method":"SLM + Fine-Tuning","Size":"3.8B"}. This record does not assert independent reproduction. ### Gemini-2.5-Pro Accuracy: 73.66. Result ID: 69a82e5f6dbb7075b94e8d34. [Reporting paper]() Recorded evaluation configuration: {"Method":"LLM + Prompt","Size":"N/A"}. This record does not assert independent reproduction. ### Qwen-2.5-Instruct Accuracy: 73.24. Result ID: 69a82e5f6dbb7075b94e8d3f. [Reporting paper]() Recorded evaluation configuration: {"Method":"SLM + Fine-Tuning","Size":"7B"}. This record does not assert independent reproduction. ### Jina-embedding-v3 Accuracy: 71.6. Result ID: 69a82e5f6dbb7075b94e8d39. [Reporting paper]() Recorded evaluation configuration: {"Method":"Embedding Model + KNN","Size":"0.6B"}. This record does not assert independent reproduction. ### DeepSeek-V3.2 Accuracy: 67.28. Result ID: 69a82e5f6dbb7075b94e8d35. [Reporting paper]() Recorded evaluation configuration: {"Method":"LLM + Prompt","Size":"N/A"}. This record does not assert independent reproduction. ### Bge-m3 Accuracy: 66.46. Result ID: 69a82e5f6dbb7075b94e8d38. [Reporting paper]() Recorded evaluation configuration: {"Method":"Embedding Model + KNN","Size":"0.5B"}. This record does not assert independent reproduction. ### Qwen3-Max Accuracy: 63.58. Result ID: 69a82e5f6dbb7075b94e8d33. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method":"LLM + Prompt","Size":"N/A"}. This record does not assert independent reproduction. ### Random Guess Accuracy: 59.26. Result ID: 69a82e5f6dbb7075b94e8d31. [Reporting paper]() Recorded evaluation configuration: {"Method":"Baseline","Size":"N/A"}. This record does not assert independent reproduction. ### GPT-4o Accuracy: 58.02. Result ID: 69a82e5f6dbb7075b94e8d36. [Reporting paper]() Recorded evaluation configuration: {"Method":"LLM + Prompt","Size":"N/A"}. This record does not assert independent reproduction. ### Qwen-3 Accuracy: 53.8. Result ID: 69a82e5f6dbb7075b94e8d41. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Method":"SLM + Fine-Tuning","Size":"8B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.