How to use [REFERENCE_DOC_x] in fill-in-the-middle tasks?

#51

by JimZhang - opened Oct 11

Oct 11

Hello, I found that there are [REFERENCE_DOC_x] tokens in tokenizer_config.json.
I assume that they can be used in FIM tasks with reference codes, but I don't know how to use them.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment