vukrosic
/

essential-web-16k-tokenizer

Model card Files Files and versions Community

essential-web-16k-tokenizer / essential_web_500k_tokens.txt

vukrosic's picture

Add essential_web_500k_tokens.txt - Tokenized version (one token per line) of 500K chars from Essential-Web

989de3a verified about 1 month ago

history contribute delete

538 kB

File too large to display, you can check the raw version instead.