The model is broken, it is incorrectly merged.

#2
by Alme995 - opened

Prompt: hello

Response: Hello, welcome totoom!

Prompt: describe a car

Response: - s'-"-"-"-"-"-"-"-"-"-"-de

i have made some tests and the vocabulary is mismatched breaking it's text understanding. Google only uses 32100 tokens in its tokenizer, they are omitted, but during merging these omitted tokens are active again, causing the size of the vocabulary to grow to 32128 tokens breaking it's embedding.

Alme995 changed discussion status to closed

Sorry i downloaded the wrong checkpoint of flan.

Sign up or log in to comment