unidisc / docs /TOKENIZERS.md
aswerdlow's picture
Initial commit
131da64

For all large scale experiments, we use the vq_ds16_t2i tokenizer from LLaMaGen.

For small-scale/scaling experiments, we use the MagViTv2 tokenizer from Show-o.

For CUB200 experiments, we use the TiTok tokenizer. In experiments, we found this tokenizer to perform the best, however it was not released at the time of our earlier experiments.