AdaMLLab/mmBERT-Turkish-Quality-Classifier

mmBERT Turkish Quality Classifier

A text quality classifier for Turkish pretraining data, trained from mmBERT-small.

This model implements the FineWeb2-HQ approach (Messmer et al., 2025) but uses mmBERT as the encoder for improved Turkish understanding.

Usage

from transformers import pipeline

classifier = pipeline("text-classification", model="AdaMLLab/mmBERT-Turkish-Quality-Classifier")
result = classifier("Türkçe metin burada")

Citation

@misc{alrashed2025mixminmatch,
      title={Mix, MinHash, and Match: Cross-Source Agreement for Multilingual Pretraining Datasets}, 
      author={Sultan Alrashed and Francesco Orabona},
      year={2025},
      eprint={2512.18834v2},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2512.18834v2}, 
}