Add ZipNN text
Browse files
README.md
CHANGED
@@ -38,6 +38,20 @@ zipnn_hf()
|
|
38 |
tokenizer = AutoTokenizer.from_pretrained("royleibov/Jamba-v0.1-ZipNN-Compressed", trust_remote_code=True)
|
39 |
model = AutoModelForCausalLM.from_pretrained("royleibov/Jamba-v0.1-ZipNN-Compressed", trust_remote_code=True)
|
40 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
41 |
|
42 |
# Model Card for Jamba
|
43 |
|
|
|
38 |
tokenizer = AutoTokenizer.from_pretrained("royleibov/Jamba-v0.1-ZipNN-Compressed", trust_remote_code=True)
|
39 |
model = AutoModelForCausalLM.from_pretrained("royleibov/Jamba-v0.1-ZipNN-Compressed", trust_remote_code=True)
|
40 |
```
|
41 |
+
### ZipNN
|
42 |
+
ZipNN also allows you to seemlessly save local disk space in your cache after the model is downloaded.
|
43 |
+
|
44 |
+
To compress the cached model, simply run:
|
45 |
+
```bash
|
46 |
+
python zipnn_compress_path.py safetensors --model royleibov/Jamba-v0.1-ZipNN-Compressed --hf_cache
|
47 |
+
```
|
48 |
+
|
49 |
+
The model will be decompressed automatically and safely as long as `zipnn_hf()` is added at the top of the file like in the [example above](#use-this-model).
|
50 |
+
|
51 |
+
To decompress manualy, simply run:
|
52 |
+
```bash
|
53 |
+
python zipnn_decompress_path.py --model royleibov/Jamba-v0.1-ZipNN-Compressed --hf_cache
|
54 |
+
```
|
55 |
|
56 |
# Model Card for Jamba
|
57 |
|