togethercomputer
/

RedPajama-INCITE-7B-Chat

Text Generation

text-generation-inference

Model card Files Files and versions

juewang commited on May 5, 2023

Commit

6f2c961

·

1 Parent(s): 70e63a0

Update README.md

Files changed (1) hide show

README.md +5 -5

README.md CHANGED Viewed

@@ -120,11 +120,11 @@ Please refer to [togethercomputer/RedPajama-Data-1T](https://huggingface.co/data
 **Training Procedure**
-- **Hardware:** TODO @Dan
-- **Optimizer:**
-- **Gradient Accumulations**:
-- **Num of Tokens:** 800B Tokens
-- **Learning rate:**
 ## Community

 **Training Procedure**
+- **Hardware:** 8 A100
+- **Optimizer:** Adam
+- **Gradient Accumulations**: 1
+- **Num of Tokens:** 131M tokens
+- **Learning rate:** 1e-5
 ## Community