Update README.md
Browse files
README.md
CHANGED
@@ -66,10 +66,10 @@ All notebooks are **beginner friendly**! Add your dataset, click "Run All", and
|
|
66 |
## Model Summary
|
67 |
|
68 |
### Unsloth bug fixes:
|
69 |
-
1. Padding and EOS tokens are the same - fixed this
|
70 |
-
2. Chat template had extra EOS token - removed this
|
71 |
-
3. EOS token should be <|end|> not <|endoftext|>
|
72 |
-
4. Changed unk_token to �
|
73 |
|
74 |
Phi-4-mini-instruct is a lightweight open model built upon synthetic data and filtered publicly available websites - with a focus on high-quality, reasoning dense data. The model belongs to the Phi-4 model family and supports 128K token context length. The model underwent an enhancement process, incorporating both supervised fine-tuning and direct preference optimization to support precise instruction adherence and robust safety measures.
|
75 |
|
|
|
66 |
## Model Summary
|
67 |
|
68 |
### Unsloth bug fixes:
|
69 |
+
1. Padding and EOS tokens are the same - fixed this.
|
70 |
+
2. Chat template had extra EOS token - removed this. Otherwise you will be <|end|> during inference.
|
71 |
+
3. EOS token should be <|end|> not <|endoftext|>. Otherwise it'll terminate at <|endoftext|>
|
72 |
+
4. Changed unk_token to � from EOS.
|
73 |
|
74 |
Phi-4-mini-instruct is a lightweight open model built upon synthetic data and filtered publicly available websites - with a focus on high-quality, reasoning dense data. The model belongs to the Phi-4 model family and supports 128K token context length. The model underwent an enhancement process, incorporating both supervised fine-tuning and direct preference optimization to support precise instruction adherence and robust safety measures.
|
75 |
|