danielhanchen commited on
Commit
9ecd79f
·
verified ·
1 Parent(s): b894149

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -66,10 +66,10 @@ All notebooks are **beginner friendly**! Add your dataset, click "Run All", and
66
  ## Model Summary
67
 
68
  ### Unsloth bug fixes:
69
- 1. Padding and EOS tokens are the same - fixed this
70
- 2. Chat template had extra EOS token - removed this
71
- 3. EOS token should be <|end|> not <|endoftext|>
72
- 4. Changed unk_token to �
73
 
74
  Phi-4-mini-instruct is a lightweight open model built upon synthetic data and filtered publicly available websites - with a focus on high-quality, reasoning dense data. The model belongs to the Phi-4 model family and supports 128K token context length. The model underwent an enhancement process, incorporating both supervised fine-tuning and direct preference optimization to support precise instruction adherence and robust safety measures.
75
 
 
66
  ## Model Summary
67
 
68
  ### Unsloth bug fixes:
69
+ 1. Padding and EOS tokens are the same - fixed this.
70
+ 2. Chat template had extra EOS token - removed this. Otherwise you will be <|end|> during inference.
71
+ 3. EOS token should be <|end|> not <|endoftext|>. Otherwise it'll terminate at <|endoftext|>
72
+ 4. Changed unk_token to � from EOS.
73
 
74
  Phi-4-mini-instruct is a lightweight open model built upon synthetic data and filtered publicly available websites - with a focus on high-quality, reasoning dense data. The model belongs to the Phi-4 model family and supports 128K token context length. The model underwent an enhancement process, incorporating both supervised fine-tuning and direct preference optimization to support precise instruction adherence and robust safety measures.
75