fbaldassarri commited on
Commit
c3964d7
·
verified ·
1 Parent(s): b95192d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -13,9 +13,9 @@ tags:
13
  - gptq
14
  - intel
15
  license: apache-2.0
16
- model_name: Minerva 1B base v1.0
17
  base_model:
18
- - sapienzanlp/Minerva-1B-base-v1.0
19
  inference: false
20
  model_creator: sapienzanlp
21
  datasets:
@@ -28,7 +28,7 @@ quantized_by: fbaldassarri
28
 
29
  ## Model Information
30
 
31
- Quantized version of [sapienzanlp/Minerva-1B-base-v1.0](https://huggingface.co/sapienzanlp/Minerva-1B-base-v1.0) using torch.float32 for quantization tuning.
32
  - 8 bits (INT8)
33
  - group size = 128
34
  - Symmetrical Quantization
@@ -38,7 +38,7 @@ Fast and low memory, 2-3X speedup (slight accuracy drop at W8G128)
38
 
39
  Quantization framework: [Intel AutoRound](https://github.com/intel/auto-round) v0.4.6
40
 
41
- Note: this INT8 version of Minerva-1B-base-v1.0 has been quantized to run inference through CPU.
42
 
43
  ## Replication Recipe
44
 
@@ -63,14 +63,14 @@ pip install -vvv --no-build-isolation -e .[cpu]
63
 
64
  ```
65
  from transformers import AutoModelForCausalLM, AutoTokenizer
66
- model_name = "sapienzanlp/Minerva-1B-base-v1.0"
67
  model = AutoModelForCausalLM.from_pretrained(model_name)
68
  tokenizer = AutoTokenizer.from_pretrained(model_name)
69
  from auto_round import AutoRound
70
  bits, group_size, sym, device, amp = 8, 128, True, 'cpu', False
71
  autoround = AutoRound(model, tokenizer, nsamples=128, iters=200, seqlen=512, batch_size=4, bits=bits, group_size=group_size, sym=sym, device=device, amp=amp)
72
  autoround.quantize()
73
- output_dir = "./AutoRound/sapienzanlp_Minerva-1B-base-v1.0-autoround-int8-gs128-sym"
74
  autoround.save_quantized(output_dir, format='auto_round', inplace=True)
75
  ```
76
 
 
13
  - gptq
14
  - intel
15
  license: apache-2.0
16
+ model_name: Minerva 350M base v1.0
17
  base_model:
18
+ - sapienzanlp/Minerva-350M-base-v1.0
19
  inference: false
20
  model_creator: sapienzanlp
21
  datasets:
 
28
 
29
  ## Model Information
30
 
31
+ Quantized version of [sapienzanlp/Minerva-350M-base-v1.0](https://huggingface.co/sapienzanlp/Minerva-350M-base-v1.0) using torch.float32 for quantization tuning.
32
  - 8 bits (INT8)
33
  - group size = 128
34
  - Symmetrical Quantization
 
38
 
39
  Quantization framework: [Intel AutoRound](https://github.com/intel/auto-round) v0.4.6
40
 
41
+ Note: this INT8 version of Minerva-350M-base-v1.0 has been quantized to run inference through CPU.
42
 
43
  ## Replication Recipe
44
 
 
63
 
64
  ```
65
  from transformers import AutoModelForCausalLM, AutoTokenizer
66
+ model_name = "sapienzanlp/Minerva-350M-base-v1.0"
67
  model = AutoModelForCausalLM.from_pretrained(model_name)
68
  tokenizer = AutoTokenizer.from_pretrained(model_name)
69
  from auto_round import AutoRound
70
  bits, group_size, sym, device, amp = 8, 128, True, 'cpu', False
71
  autoround = AutoRound(model, tokenizer, nsamples=128, iters=200, seqlen=512, batch_size=4, bits=bits, group_size=group_size, sym=sym, device=device, amp=amp)
72
  autoround.quantize()
73
+ output_dir = "./AutoRound/sapienzanlp_Minerva-350M-base-v1.0-autoround-int8-gs128-sym"
74
  autoround.save_quantized(output_dir, format='auto_round', inplace=True)
75
  ```
76