yasserrmd commited on
Commit
7b4ae2f
Β·
verified Β·
1 Parent(s): 7ddedd1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +39 -0
README.md CHANGED
@@ -65,5 +65,44 @@ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
65
 
66
  ---
67
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
68
  ## πŸ“œ License
69
  apache-2.0
 
65
 
66
  ---
67
 
68
+ ## πŸ”§ Example Usage (with repetition control)
69
+
70
+ ```python
71
+ from transformers import AutoTokenizer, AutoModelForCausalLM
72
+
73
+ model_id = "yasserrmd/RSCaLM-138M-LLaMA"
74
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
75
+ model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
76
+
77
+ prompt = "when a man goes to fishing"
78
+ inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
79
+
80
+ # Generation settings to reduce repetition
81
+ outputs = model.generate(
82
+ **inputs,
83
+ max_new_tokens=100, # Limit length of output
84
+ temperature=0.7, # Lower temperature = more focused
85
+ top_p=0.9, # Nucleus sampling
86
+ top_k=50, # Top-K filtering
87
+ repetition_penalty=1.2, # Penalize repeating tokens
88
+ no_repeat_ngram_size=3, # Prevent repeating trigrams
89
+ eos_token_id=tokenizer.eos_token_id, # End generation at EOS
90
+ )
91
+
92
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
93
+ ```
94
+
95
+ ---
96
+
97
+ ### πŸ’‘ Tips for controlling repetition:
98
+
99
+ 1. **`repetition_penalty`** – Increase slightly above `1.0` (e.g., `1.2–1.5`) to discourage repeated phrases.
100
+ 2. **`no_repeat_ngram_size`** – Set to `3` or `4` to avoid repeated n-grams.
101
+ 3. **`top_k` + `top_p`** – Combine both for better randomness control.
102
+ 4. **Lower `temperature`** – Keeps outputs focused and less chaotic.
103
+ 5. **Stop sequences** – Add specific words/phrases to halt generation early if needed.
104
+
105
+ ---
106
+
107
  ## πŸ“œ License
108
  apache-2.0