phonemetransformers
/

babble-tokenizers

Model card Files Files and versions Community

babble-tokenizers / BABYLM-TOKENIZER-BPE-TXT-SPACELESS /special_tokens_map.json

codebyzeb's picture

Upload folder using huggingface_hub

834363d verified 30 days ago

history blame contribute delete

109 Bytes

	{
	"bos_token": "UTT_BOUNDARY",
	"eos_token": "UTT_BOUNDARY",
	"pad_token": "PAD",
	"unk_token": "UNK"
	}