Continued Pretraining Tutorial Notebook.

#1
by nirajandhakal - opened

Can you guys create a CPT notebook for .txt file dataset? Also, after the CPT is complete, I want to see if the model can generate the exact tokens present in the dataset as well.

Thanks.

Sign up or log in to comment