Commit
·
8f5cd82
1
Parent(s):
3bc5cf3
Update README.md
Browse files
README.md
CHANGED
@@ -14,6 +14,10 @@ Sheared-LLaMA-1.3B is a model pruned and further pre-trained from [meta-llama/Ll
|
|
14 |
model = AutoModelForCausalLM.from_pretrained("princeton-nlp/Sheared-LLaMA-1.3B")
|
15 |
```
|
16 |
|
|
|
|
|
|
|
|
|
17 |
## Downstream Tasks
|
18 |
|
19 |
We evaluate on an extensive set of downstream tasks including reasoning, reading comprehension, language modeling and knowledge intensive tasks. Our Sheared-LLaMA models outperform existing large language models.
|
|
|
14 |
model = AutoModelForCausalLM.from_pretrained("princeton-nlp/Sheared-LLaMA-1.3B")
|
15 |
```
|
16 |
|
17 |
+
- Smaller-scale
|
18 |
+
- Same vocabulary as LLaMA1 and LLaMA2
|
19 |
+
- Derived with 50B tokens by utilizing existing strong LLMs
|
20 |
+
|
21 |
## Downstream Tasks
|
22 |
|
23 |
We evaluate on an extensive set of downstream tasks including reasoning, reading comprehension, language modeling and knowledge intensive tasks. Our Sheared-LLaMA models outperform existing large language models.
|