ethical-spectacle/social-bias-ner

Token Classification • Updated Oct 28, 2024 • 3.43k • 3

Note SoTA Model for Social Bias Multi-Label Token Classification

Note Dataset containing GUS entities (main dataset used). Synthetic annotations by gpt-4o agents with DSPy pipeline.

Note Underlying corpus used in GUS dataset, synthetically generated by Mistral 7B to contain a balanced distribution of demographics. This dataset also contains text sequence classes for types of bias (could be used in classifier).

Note GUS dataset formatted for LLMs in chatml format (role/content). Beware: LLMs underperformed encoder-only token classifiers (poor token/label alignment, and resource intensive inference).