bofenghuang commited on
Commit
615cfe6
1 Parent(s): 3a8dd65
Files changed (2) hide show
  1. README.md +144 -0
  2. logo_v2.jpg +0 -0
README.md ADDED
@@ -0,0 +1,144 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: llama2
3
+ language: fr
4
+ pipeline_tag: text-generation
5
+ inference: false
6
+ tags:
7
+ - LLM
8
+ - llama-2
9
+ - finetuned
10
+ ---
11
+
12
+ <p align="center" width="100%">
13
+ <img src="https://huggingface.co/bofenghuang/vigogne-2-70b-chat/resolve/main/logo_v2.jpg" alt="Vigogne" style="width: 30%; min-width: 300px; display: block; margin: auto;">
14
+ </p>
15
+
16
+ # Vigogne-2-70B-Chat: A Llama-2-based French Chat LLM
17
+
18
+ Vigogne-2-70B-Chat is a French chat LLM, based on [Llama-2-70B](https://ai.meta.com/llama), optimized to generate helpful and coherent responses in conversations with users.
19
+
20
+ Check out our [release blog](https://github.com/bofenghuang/vigogne/blob/main/blogs/2023-08-17-vigogne-chat-v2_0.md) and [GitHub repository](https://github.com/bofenghuang/vigogne) for more information.
21
+
22
+ **Usage and License Notices**: Vigogne-2-70B-Chat follows Llama-2's [usage policy](https://ai.meta.com/llama/use-policy). A significant portion of the training data is distilled from GPT-3.5-Turbo and GPT-4, kindly use it cautiously to avoid any violations of OpenAI's [terms of use](https://openai.com/policies/terms-of-use).
23
+
24
+ ## Prompt Template
25
+
26
+ We used a prompt template adapted from the chat format of Llama-2.
27
+
28
+ You can apply this formatting using the [chat template](https://huggingface.co/docs/transformers/main/chat_templating) through the `apply_chat_template()` method.
29
+
30
+ ```python
31
+ from transformers import AutoTokenizer
32
+
33
+ tokenizer = AutoTokenizer.from_pretrained("bofenghuang/vigogne-2-70b-chat")
34
+
35
+ conversation = [
36
+ {"role": "user", "content": "Bonjour ! Comment ça va aujourd'hui ?"},
37
+ {"role": "assistant", "content": "Bonjour ! Je suis une IA, donc je n'ai pas de sentiments, mais je suis prêt à vous aider. Comment puis-je vous assister aujourd'hui ?"},
38
+ {"role": "user", "content": "Quelle est la hauteur de la Tour Eiffel ?"},
39
+ {"role": "assistant", "content": "La Tour Eiffel mesure environ 330 mètres de hauteur."},
40
+ {"role": "user", "content": "Comment monter en haut ?"},
41
+ ]
42
+
43
+ print(tokenizer.apply_chat_template(conversation, tokenize=False, add_generation_prompt=True))
44
+ ```
45
+
46
+ You will get
47
+
48
+ ```
49
+ <s>[INST] <<SYS>>
50
+ Vous êtes Vigogne, un assistant IA créé par Zaion Lab. Vous suivez extrêmement bien les instructions. Aidez autant que vous le pouvez.
51
+ <</SYS>>
52
+
53
+ Bonjour ! Comment ça va aujourd'hui ? [/INST] Bonjour ! Je suis une IA, donc je n'ai pas de sentiments, mais je suis prêt à vous aider. Comment puis-je vous assister aujourd'hui ? </s>[INST] Quelle est la hauteur de la Tour Eiffel ? [/INST] La Tour Eiffel mesure environ 330 mètres de hauteur. </s>[INST] Comment monter en haut ? [/INST]
54
+ ```
55
+
56
+ ## Usage
57
+
58
+ <!-- ### Inference using the quantized versions
59
+
60
+ The quantized versions of this model are generously provided by [TheBloke](https://huggingface.co/TheBloke)!
61
+
62
+ - AWQ for GPU inference: [TheBloke/Vigogne-2-70B-Chat-AWQ](https://huggingface.co/TheBloke/Vigogne-2-70B-Chat-AWQ)
63
+ - GTPQ for GPU inference: [TheBloke/Vigogne-2-70B-Chat-GPTQ](https://huggingface.co/TheBloke/Vigogne-2-70B-Chat-GPTQ)
64
+ - GGUF for CPU+GPU inference: [TheBloke/Vigogne-2-70B-Chat-GGUF](https://huggingface.co/TheBloke/Vigogne-2-70B-Chat-GGUF)
65
+
66
+ These versions facilitate testing and development with various popular frameworks, including [AutoAWQ](https://github.com/casper-hansen/AutoAWQ), [vLLM](https://github.com/vllm-project/vllm), [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ), [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa), [llama.cpp](https://github.com/ggerganov/llama.cpp), [text-generation-webui](https://github.com/oobabooga/text-generation-webui), and more. -->
67
+
68
+ ### Inference using the unquantized model with 🤗 Transformers
69
+
70
+ ```python
71
+ from typing import Dict, List, Optional
72
+ import torch
73
+ from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig, TextStreamer
74
+
75
+ model_name_or_path = "bofenghuang/vigogne-2-70b-chat"
76
+ tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, padding_side="right", use_fast=False)
77
+ model = AutoModelForCausalLM.from_pretrained(model_name_or_path, torch_dtype=torch.float16, device_map="auto")
78
+
79
+ streamer = TextStreamer(tokenizer, timeout=10.0, skip_prompt=True, skip_special_tokens=True)
80
+
81
+
82
+ def chat(
83
+ query: str,
84
+ history: Optional[List[Dict]] = None,
85
+ temperature: float = 0.7,
86
+ top_p: float = 1.0,
87
+ top_k: float = 0,
88
+ repetition_penalty: float = 1.1,
89
+ max_new_tokens: int = 1024,
90
+ **kwargs,
91
+ ):
92
+ if history is None:
93
+ history = []
94
+
95
+ history.append({"role": "user", "content": query})
96
+
97
+ input_ids = tokenizer.apply_chat_template(history, return_tensors="pt").to(model.device)
98
+ input_length = input_ids.shape[1]
99
+
100
+ generated_outputs = model.generate(
101
+ input_ids=input_ids,
102
+ generation_config=GenerationConfig(
103
+ temperature=temperature,
104
+ do_sample=temperature > 0.0,
105
+ top_p=top_p,
106
+ top_k=top_k,
107
+ repetition_penalty=repetition_penalty,
108
+ max_new_tokens=max_new_tokens,
109
+ pad_token_id=tokenizer.eos_token_id,
110
+ **kwargs,
111
+ ),
112
+ streamer=streamer,
113
+ return_dict_in_generate=True,
114
+ )
115
+
116
+ generated_tokens = generated_outputs.sequences[0, input_length:]
117
+ generated_text = tokenizer.decode(generated_tokens, skip_special_tokens=True)
118
+
119
+ history.append({"role": "assistant", "content": generated_text})
120
+
121
+ return generated_text, history
122
+
123
+
124
+ # 1st round
125
+ response, history = chat("Un escargot parcourt 100 mètres en 5 heures. Quelle est sa vitesse ?", history=None)
126
+
127
+ # 2nd round
128
+ response, history = chat("Quand il peut dépasser le lapin ?", history=history)
129
+
130
+ # 3rd round
131
+ response, history = chat("Écris une histoire imaginative qui met en scène une compétition de course entre un escargot et un lapin.", history=history)
132
+ ```
133
+
134
+ You can also use the Google Colab Notebook provided below.
135
+
136
+ <a href="https://colab.research.google.com/github/bofenghuang/vigogne/blob/main/notebooks/infer_chat.ipynb" target="_blank"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
137
+
138
+ ## Limitations
139
+
140
+ Vigogne is still under development, and there are many limitations that have to be addressed. Please note that it is possible that the model generates harmful or biased content, incorrect information or generally unhelpful answers.
141
+
142
+ ## Acknowledgements
143
+
144
+ The model training was conducted on the [Jean-Zay supercomputer](http://www.idris.fr/eng/jean-zay/jean-zay-presentation-eng.html) at GENCI, and we extend our gratitude to the IDRIS team for their responsive support throughout the project.
logo_v2.jpg ADDED