fasttext-language-detection-ms-id-en

FastText model to classify Malay {standard Malay, local Malay}, Indonesian {standard Indonesian, local Indonesian} and English {standard English, Manglish, Singlish}.

how to use

from huggingface_hub import hf_hub_download
import fasttext

filename = hf_hub_download(
    repo_id="mesolitica/fasttext-language-detection-ms-id-en", 
    filename="fasttext.ftz"
)
lang_model = fasttext.load_model(filename)
lang_model.predict('hello my name', k = 10)

Output,

(('__label__english', '__label__malay', '__label__indon'),
 array([0.95163465, 0.03559022, 0.01281445]))