File size: 3,666 Bytes
91f84c2 5ec4b7f 91f84c2 f657b4e 91f84c2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 |
---
datasets:
- tollefj/nor-instruct-combined
language:
- 'no'
---
# Nordavind-7B
A model trained on top of <https://huggingface.co/norallm/normistral-7b-warm> on various Norwegian instruct data with a context length of 4096. The name - *nordavind* - a cold wind from the north, is derived from Mistral. See https://en.wikipedia.org/wiki/Mistral_(wind)!
Runtime: ~20 hours on a V100 32GB. Final loss: `1.301`.
Prompt format:
```python
<s>{system_prompt} [INST] {inst} [/INST] \\n {res} </s>
```
Inference prompt:
```python
<s>{system_prompt} [INST] {inst} [/INST] \\n
```
## Examples:
*generate("Lag en skikkelig norsk setning", tokens=50, first_sent=True)*
> `Jeg liker å gå på tur i skogen.`
*generate("Lag en liste med tre forskjellige bruksområder (inkludert eksempler) for kunstig intelligens.")*
>1. Automatisering av oppgaver: AI kan brukes til å automatisere oppgaver som å svare på kundehenvendelser, behandle medisinske bilder og diagnostisere sykdommer, og til og med kjøre biler.
>2. Forbedre kundeservice: AI kan brukes for å forbedre kundeservicen ved å gi personlige anbefalinger, gi raskere svar og gi mer nøyaktige svar.
>3. Forbedre sikkerhet: AI kan brukes i sikkerhetssystemer for å oppdage og forhindre cyberangrep, oppdage svindel og oppdage mistenkelige aktiviteter.
*generate("Hvem tror du vinner valget i Bergen?", tokens=60)*
>Det er vanskelig å forutsi hvem som vinner valget i Bergen, men det er sannsynlig at enten Arbeiderpartiet eller Høyre vil vinne. Arbeiderpartiet har hatt makten i Bergen i mange år, men Høyre har hatt en sterk tilstedeværelse i byen de siste årene.
*generate("Forklar dyp læring til en femteklassing")*
>Dyp læring er en type maskinlæring som bruker nevrale nettverk for å lære komplekse mønstre i data. Det innebærer å trene en datamaskin til å gjenkjenne mønstre i data og ta beslutninger basert på disse mønstrene.
## Model loading
```python
from transformers import pipeline
pipe = pipeline("text-generation", model="tollefj/nordavind-7b-instruct-warm")
```
## Inference
```python
system_prompt = 'Du er "Nordavind", en hjelpsom assistent.'
def make_prompt(inst, res=None):
if not res:
return f"""<s>{system_prompt} [INST] {inst} [/INST] \\n"""
return f"""<s>{system_prompt} [INST] {inst} [/INST] \\n {res} </s>"""
def postprocess(output, first_sent=False):
output = output.split("\\n")[-1].strip()
# ignore hashtags as we often see #no_output
output = output.split("#")[0].strip()
# ignore incomplete sentences
if not output.endswith("."):
output = output.rsplit(".", 1)[0] + "."
if first_sent:
return output.split(".")[0] + "."
return output
def generate(prompt, tokens=100, first_sent=False, sample=False, temperature=1.0):
prompt = make_prompt(prompt)
output = pipe(
prompt,
max_length=tokens,
do_sample=sample,
temperature=temperature,
)
output = output[0]["generated_text"]
output = postprocess(output, first_sent=first_sent)
print(output)
```
# Training details
The model was fine-tuned in an 4bit BitsAndBytes config.
```python
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=getattr(torch, "float16"),
bnb_4bit_use_double_quant=False,
)
```
with the following LoRa-configuration:
```python
config = LoraConfig(
target_modules=[
"q_proj",
"k_proj",
"v_proj",
"o_proj",
"gate_proj",
"up_proj",
"down_proj",
"lm_head",
],
bias="none",
lora_dropout=0.05,
task_type="CAUSAL_LM",
)
``` |