fbaldassarri commited on
Commit
d7948ff
·
verified ·
1 Parent(s): 239a4b0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -15,9 +15,9 @@ tags:
15
  - auto-gptq
16
  - intel
17
  license: apache-2.0
18
- model_name: Minerva 1B base v1.0
19
  base_model:
20
- - sapienzanlp/Minerva-1B-base-v1.0
21
  inference: false
22
  model_creator: sapienzanlp
23
  datasets:
@@ -30,7 +30,7 @@ quantized_by: fbaldassarri
30
 
31
  ## Model Information
32
 
33
- Quantized version of [sapienzanlp/Minerva-1B-base-v1.0](https://huggingface.co/sapienzanlp/Minerva-1B-base-v1.0) using torch.float32 for quantization tuning.
34
  - 8 bits (INT8)
35
  - group size = 128
36
  - Asymmetrical Quantization
@@ -38,7 +38,7 @@ Quantized version of [sapienzanlp/Minerva-1B-base-v1.0](https://huggingface.co/s
38
 
39
  Quantization framework: [Intel AutoRound](https://github.com/intel/auto-round) v0.4.6
40
 
41
- Note: this INT8 version of Minerva-1B-base-v1.0 has been quantized to run inference through CPU.
42
 
43
  ## Replication Recipe
44
 
@@ -63,14 +63,14 @@ pip install -vvv --no-build-isolation -e .[cpu]
63
 
64
  ```
65
  from transformers import AutoModelForCausalLM, AutoTokenizer
66
- model_name = "sapienzanlp/Minerva-1B-base-v1.0"
67
  model = AutoModelForCausalLM.from_pretrained(model_name)
68
  tokenizer = AutoTokenizer.from_pretrained(model_name)
69
  from auto_round import AutoRound
70
  bits, group_size, sym, device, amp = 8, 128, False, 'cpu', False
71
  autoround = AutoRound(model, tokenizer, nsamples=128, iters=200, seqlen=512, batch_size=4, bits=bits, group_size=group_size, sym=sym, device=device, amp=amp)
72
  autoround.quantize()
73
- output_dir = "./AutoRound/sapienzanlp_Minerva-1B-base-v1.0-autogptq-int8-gs128-asym"
74
  autoround.save_quantized(output_dir, format='auto_gptq', inplace=True)
75
  ```
76
 
 
15
  - auto-gptq
16
  - intel
17
  license: apache-2.0
18
+ model_name: Minerva 350M base v1.0
19
  base_model:
20
+ - sapienzanlp/Minerva-350M-base-v1.0
21
  inference: false
22
  model_creator: sapienzanlp
23
  datasets:
 
30
 
31
  ## Model Information
32
 
33
+ Quantized version of [sapienzanlp/Minerva-350M-base-v1.0](https://huggingface.co/sapienzanlp/Minerva-350M-base-v1.0) using torch.float32 for quantization tuning.
34
  - 8 bits (INT8)
35
  - group size = 128
36
  - Asymmetrical Quantization
 
38
 
39
  Quantization framework: [Intel AutoRound](https://github.com/intel/auto-round) v0.4.6
40
 
41
+ Note: this INT8 version of Minerva-350M-base-v1.0 has been quantized to run inference through CPU.
42
 
43
  ## Replication Recipe
44
 
 
63
 
64
  ```
65
  from transformers import AutoModelForCausalLM, AutoTokenizer
66
+ model_name = "sapienzanlp/Minerva-350M-base-v1.0"
67
  model = AutoModelForCausalLM.from_pretrained(model_name)
68
  tokenizer = AutoTokenizer.from_pretrained(model_name)
69
  from auto_round import AutoRound
70
  bits, group_size, sym, device, amp = 8, 128, False, 'cpu', False
71
  autoround = AutoRound(model, tokenizer, nsamples=128, iters=200, seqlen=512, batch_size=4, bits=bits, group_size=group_size, sym=sym, device=device, amp=amp)
72
  autoround.quantize()
73
+ output_dir = "./AutoRound/sapienzanlp_Minerva-350M-base-v1.0-autogptq-int8-gs128-asym"
74
  autoround.save_quantized(output_dir, format='auto_gptq', inplace=True)
75
  ```
76