fhai50032/QTK-81K

	Tokenizer	English	Hindi	Code_Python	Code_Java	c++	Math
0	fhai50032/BPE-Multilingual-72k (72k)	343695 reversible	13684 reversible	5367 reversible	2269 reversible	13072 reversible	5026 reversible
1	HuggingFaceTB/SmolLM2-1.7B-Instruct (49k)	337029 reversible	44629 reversible	5873 reversible	2548 reversible	15702 reversible	6200 reversible
2	fhai50032/QTK-81K (81k)	318160 reversible	10767 reversible	5192 reversible	2241 reversible	12730 reversible	5120 reversible
3	deepseek-ai/DeepSeek-R1 (128k)	338873 reversible	22854 reversible	5270 reversible	2223 reversible	12785 reversible	5375 reversible
4	unsloth/phi-4 (100k)	308645 reversible	40456 reversible	4844 reversible	2110 reversible	12088 reversible	5573 reversible
5	deepseek-ai/DeepSeek-R1-Distill-Llama-8B (128k)	308511 reversible	21109 reversible	4843 reversible	2110 reversible	12088 reversible	5573 reversible
6	unsloth/gemma-2-9b-it(256k)	323334 reversible	15915 reversible	5986 reversible	2568 reversible	15806 reversible	5870 reversible
7	krutrim-ai-labs/Krutrim-2-instruct (131k)	343067 reversible	15477 reversible	5246 reversible	2223 reversible	12700 reversible	5361 reversible
8	nvidia/Nemotron-4-Mini-Hindi-4B-Instruct (256k)	332271 irreversible, word count differs (212812 vs 212811)	14327 irreversible, word count differs (7570 vs 7561)	6229 irreversible, word count differs (1787 vs 1781)	2732 irreversible, word count differs (1013 vs 1002)	16306 irreversible, word count differs (5016 vs 4964)	6136 reversible
9	sarvamai/OpenHathi-7B-Hi-v0.1-Base (48k)	370132 reversible	15632 reversible	6640 reversible	2791 reversible	16756 reversible	6222 reversible
10	sarvamai/sarvam-2b-v0.5 (68k)	385383 reversible	11256 reversible	7387 reversible	3067 reversible	17586 reversible	6863 reversible