# Synonym Substitution Robustness on ELI5 prompts 32-bit payload Llama3.1-8B (test) Source: [Synonym Substitution Robustness on ELI5 prompts 32-bit payload Llama3.1-8B (test)]() Collection index: [llms.txt]() Data updated: 2026-05-13T09:14:07.849Z Evaluation of the robustness of the Llama 3.1 8B model, specifically when processing ELI5 prompts with a 32-bit payload, against synonym substitution attacks, measured by bit accuracy degradation. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Synonym Substitution Robustness | | Dataset (including any stated split) | ELI5 prompts 32-bit payload Llama3.1-8B (test) | | Metric | Bit Accuracy (10% Substitution) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 7 of 7 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Bit Accuracy (10% Substitution)](): Higher is better; unit: PERCENT. - [Bit Accuracy (20% Substitution)](): Higher is better; unit: PERCENT. - [Bit Accuracy (30% Substitution)](): Higher is better; unit: PERCENT. - [Bit Accuracy (40% Substitution)](): Higher is better; unit: PERCENT. - [Bit Accuracy (50% Substitution)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Bit Accuracy (10% Substitution) | Result date | Report paper | | --- | --- | --- | --- | | Ours+ | 70.7 | Not recorded | 6a0405c4dbc836b87d8277a1 | | MC2Mark | 67.1 | Not recorded | 6a0405c4dbc836b87d8277a1 | | BiMark | 66.6 | Not recorded | 6a0405c4dbc836b87d8277a1 | | MPAC | 61.2 | Not recorded | 6a0405c4dbc836b87d8277a1 | | StealthInk | 60.9 | Not recorded | 6a0405c4dbc836b87d8277a1 | | MirrorMark | 54.4 | Not recorded | 6a0405c4dbc836b87d8277a1 | | RSBH | 50.9 | Not recorded | 6a0405c4dbc836b87d8277a1 | ### Ours+ Bit Accuracy (10% Substitution): 70.7. Result ID: 6a04400d47f34ce924ea53ec. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Payload":"32-bit","Backbone":"LLAMA3.1-8B","Type":"Stateful Encoder"}. This record does not assert independent reproduction. ### MC2Mark Bit Accuracy (10% Substitution): 67.1. Result ID: 6a04400d47f34ce924ea53e7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Payload":"32-bit","Backbone":"LLAMA3.1-8B"}. This record does not assert independent reproduction. ### BiMark Bit Accuracy (10% Substitution): 66.6. Result ID: 6a04400d47f34ce924ea53e6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Payload":"32-bit","Backbone":"LLAMA3.1-8B"}. This record does not assert independent reproduction. ### MPAC Bit Accuracy (10% Substitution): 61.2. Result ID: 6a04400d47f34ce924ea53e8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Payload":"32-bit","Backbone":"LLAMA3.1-8B"}. This record does not assert independent reproduction. ### StealthInk Bit Accuracy (10% Substitution): 60.9. Result ID: 6a04400d47f34ce924ea53eb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Payload":"32-bit","Backbone":"LLAMA3.1-8B"}. This record does not assert independent reproduction. ### MirrorMark Bit Accuracy (10% Substitution): 54.4. Result ID: 6a04400d47f34ce924ea53e9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Payload":"32-bit","Backbone":"LLAMA3.1-8B"}. This record does not assert independent reproduction. ### RSBH Bit Accuracy (10% Substitution): 50.9. Result ID: 6a04400d47f34ce924ea53ea. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Payload":"32-bit","Backbone":"LLAMA3.1-8B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.