| ---
|
| license: cc-by-4.0
|
| language: ti
|
| widget:
|
| - text: "<text-to-classify>"
|
| datasets:
|
| - fgaim/tigrinya-abusive-language-detection
|
| metrics:
|
| - accuracy
|
| - f1
|
| - precision
|
| - recall
|
| model-index:
|
| - name: tiroberta-tiald-all-tasks
|
| results:
|
| - task:
|
| name: Text Classification
|
| type: text-classification
|
| metrics:
|
| - name: Abu Accuracy
|
| type: accuracy
|
| value: 0.8611111111111112
|
| - name: F1
|
| type: f1
|
| value: 0.8611109396431353
|
| - name: Precision
|
| type: precision
|
| value: 0.8611128943846637
|
| - name: Recall
|
| type: recall
|
| value: 0.8611111111111112
|
| ---
|
|
|
|
|
| # TiRoBERTa Fine-tuned for Multi-task Abusiveness, Sentiment, and Topic Classification
|
|
|
| This model is a fine-tuned version of [TiRoBERTa](https://huggingface.co/fgaim/tiroberta-base) on the [TiALD](https://huggingface.co/datasets/fgaim/tigrinya-abusive-language-detection) dataset.
|
|
|
| **Tigrinya Abusive Language Detection (TiALD) Dataset** is a large-scale, multi-task benchmark dataset for abusive language detection in the Tigrinya language. It consists of **13,717 YouTube comments** annotated for **abusiveness**, **sentiment**, and **topic** tasks. The dataset includes comments written in both the **Ge’ez script** and prevalent non-standard Latin **transliterations** to mirror real-world usage.
|
|
|
| > ⚠️ The dataset contains explicit, obscene, and potentially hateful language. It should be used for research purposes only. ⚠️
|
|
|
| This work accompanies the paper ["A Multi-Task Benchmark for Abusive Language Detection in Low-Resource Settings"](https://arxiv.org/abs/2505.12116).
|
|
|
| ## Model Usage
|
|
|
| ```python
|
| from transformers import pipeline
|
|
|
| tiald_multitask = pipeline("text-classification", model="fgaim/tiroberta-tiald-all-tasks", top_k=11)
|
| tiald_multitask("<text-to-classify>")
|
| ```
|
|
|
| ### Performance Metrics
|
|
|
| This model achieves the following results on the TiALD test set:
|
|
|
| ```json
|
| "abusiveness_metrics": {
|
| "accuracy": 0.8611111111111112,
|
| "macro_f1": 0.8611109396431353,
|
| "macro_recall": 0.8611111111111112,
|
| "macro_precision": 0.8611128943846637,
|
| "weighted_f1": 0.8611109396431355,
|
| "weighted_recall": 0.8611111111111112,
|
| "weighted_precision": 0.8611128943846637
|
| },
|
| "topic_metrics": {
|
| "accuracy": 0.6155555555555555,
|
| "macro_f1": 0.5491185274678864,
|
| "macro_recall": 0.5143416011263588,
|
| "macro_precision": 0.7341640739780486,
|
| "weighted_f1": 0.5944096153417657,
|
| "weighted_recall": 0.6155555555555555,
|
| "weighted_precision": 0.6870800624645906
|
| },
|
| "sentiment_metrics": {
|
| "accuracy": 0.6533333333333333,
|
| "macro_f1": 0.5340845253007789,
|
| "macro_recall": 0.5410170159158625,
|
| "macro_precision": 0.534652401599494,
|
| "weighted_f1": 0.6620101614004723,
|
| "weighted_recall": 0.6533333333333333,
|
| "weighted_precision": 0.6750245466592532
|
| }
|
| ```
|
|
|
| ## Training Hyperparameters
|
|
|
| The following hyperparameters were used during training:
|
|
|
| - learning_rate: 3e-05
|
| - train_batch_size: 8
|
| - optimizer: Adam (betas=0.9, 0.999, epsilon=1e-08)
|
| - lr_scheduler_type: linear
|
| - num_epochs: 7.0
|
| - seed: 42
|
|
|
| ## Intended Usage
|
|
|
| The TiALD dataset and models designed to support:
|
|
|
| - Research in abusive language detection in low-resource languages
|
| - Context-aware abuse, sentiment, and topic modeling
|
| - Multi-task and transfer learning with digraphic scripts
|
| - Evaluation of multilingual and fine-tuned language models
|
|
|
| Researchers and developers should avoid using this dataset for direct moderation or enforcement tasks without human oversight.
|
|
|
| ## Ethical Considerations
|
|
|
| - **Sensitive content**: Contains toxic and offensive language. Use for research purposes only.
|
| - **Cultural sensitivity**: Abuse is context-dependent; annotations were made by native speakers to account for cultural nuance.
|
| - **Bias mitigation**: Data sampling and annotation were carefully designed to minimize reinforcement of stereotypes.
|
| - **Privacy**: All the source content for the dataset is publicly available on YouTube.
|
| - **Respect for expression**: The dataset should not be used for automated censorship without human review.
|
|
|
| This research received IRB approval (Ref: KH2022-133) and followed ethical data collection and annotation practices, including informed consent of annotators.
|
|
|
| ## Citation
|
|
|
| If you use this model or the `TiALD` dataset in your work, please cite:
|
|
|
| ```bibtex
|
| @misc{gaim-etal-2025-tiald-benchmark,
|
| title = {A Multi-Task Benchmark for Abusive Language Detection in Low-Resource Settings},
|
| author = {Fitsum Gaim and Hoyun Song and Huije Lee and Changgeon Ko and Eui Jun Hwang and Jong C. Park},
|
| year = {2025},
|
| eprint = {2505.12116},
|
| archiveprefix = {arXiv},
|
| primaryclass = {cs.CL},
|
| url = {https://arxiv.org/abs/2505.12116}
|
| }
|
| ```
|
|
|
| ## License
|
|
|
| This dataset is released under the [Creative Commons Attribution 4.0 International License (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/).
|
|
|
