Manoj779944
/

gpt2-like-tokenizer

File size: 68 Bytes

60ec7d1

{
  "bos_token": "<|endoftext|>",
  "eos_token": "<|endoftext|>,"
}