xddd-processed / added_tokens.json
jnjj's picture
Upload tokenizer after adding special tokens and chat template
a07775b verified
{
"<|end_header_id|>": 32002,
"<|eot_id|>": 32001,
"<|start_header_id|>": 32000,
"<|start_header_id|>assistant<|end_header_id|>": 32004,
"<|start_header_id|>ipython<|end_header_id|>": 32006,
"<|start_header_id|>system<|end_header_id|>": 32003,
"<|start_header_id|>user<|end_header_id|>": 32005
}