We believe that great research is open research. That’s why we have open-sourced the data, code, and annotation guidelines for all of HateCheck. This means that anyone can reproduce our results, and use the HateCheck test suites to test and improve their own hate speech detection models.
We believe that great research is open research. That’s why we have open-sourced the data, code, and annotation guidelines for all of HateCheck. This means that anyone can reproduce our results, and use the HateCheck test suites to test and improve their own hate speech detection models.
The original HateCheck introduced functional tests for English hate speech detection models. It includes tests for different forms of hate, such as derogation and threatening language, as well as tests for challenging non-hate, such as counterspeech. All HateCheck test cases were generated from templates across seven target groups and then validated by a team of trained annotators.
Hate speech is a global phenomenon. But most hate speech research focuses on English language content, which makes it difficult to build more effective hate speech detection models in other languages. Even the social media giants have clear language gaps in their content moderation systems. The result? Billions of non-English speakers across the world are less protected against online hate, and more at risk of suffering from serious harm.
HateCheck is a fully open-source resource for testing hate speech detection models, built by expert researchers and presented at top academic conferences. Covering 11 languages, HateCheck provides targeted diagnostic insights into the performance of hate speech detection models. It offers 25+ functional tests in each language, and special tests for emoji-based hate in English—all selected …
