README.md

---

license: llama3
---


- Foundation Model [Bllossom 8B](https://huggingface.co/MLP-KTLim/llama-3-Korean-Bllossom-8B)
- datasets
  -  [Koalpaca v1.1a](https://huggingface.co/datasets/beomi/KoAlpaca-v1.1a)
  -  [jojo0217/korean_safe_conversation](https://huggingface.co/datasets/jojo0217/korean_safe_conversation)

# Query
```python



import torch

from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline



BASE_MODEL = "sh2orc/llama-3-korean-8b"



model = AutoModelForCausalLM.from_pretrained(BASE_MODEL,device_map="auto")



tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)

tokenizer.pad_token = tokenizer.eos_token

tokenizer.padding_side = 'right'



instruction = "한강에는 대교가 몇 개 있어?"



pipe = pipeline("text-generation", 

                model=model, 

                tokenizer=tokenizer, 

                max_new_tokens=1024)



messages = [

    {"role": "user", "content": instruction},

]



prompt = pipe.tokenizer.apply_chat_template(

        messages, 

        tokenize=False, 

        add_generation_prompt=True

)



outputs = pipe(

    prompt,

    do_sample=True,

    temperature=0.8,

    top_k=10,

    top_p=0.9,

    add_special_tokens=True,

    eos_token_id = [ 

        pipe.tokenizer.eos_token_id,

        pipe.tokenizer.convert_tokens_to_ids("<|eot_id|>")

    ]

)



print(outputs[0]['generated_text'][len(prompt):])

```

# Result
<pre>

한강에는 총 8개의 다리(교)가 있습니다. 그 중 3개는 북쪽으로 향해 있고, 나머지 5개는 남쪽으로 향해 있습니다.

</pre>