pranavsaranaway
/

bpe_tokenizer

Model card Files Files and versions

bpe_tokenizer / special_tokens_map.json

pranavsaranaway's picture

pranavsaranaway

Upload tokenizer

96d7939 verified 29 days ago

history blame contribute delete

67 Bytes

	{
	"bos_token": "<\|endoftext\|>",
	"eos_token": "<\|endoftext\|>"
	}