Dog Whistle Detection on FETCH! Realistic 1.0
0.94PrecisionDIRECT Pipeline
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| DIRECT PipelineModel=LLaMa 8B2024.12 | 0.94 | 60.29 | 1.17 | |
| DIRECT PipelineModel=Mistral 7B2024.12 | 0.69 | 46.32 | 0.86 | |
| BERT PREDICTFiltering Model=HateXplain BERT, Keyword Extraction Model=YAKE, Threshold=3200, Max n-grams=32024.12 | 0.1 | 1.47 | 4.63 | |
| FETCH LLM (EarShot)Filtering Model=LLaMa 13B, Keyword Extraction Model=RAKE, Threshold=200, Max n-grams=12024.12 | 0.0952 | 1.47 | 4.55 | |
| DIRECT PipelineModel=LLaMa 13B2024.12 | 0 | 0 | 0 |