Update README.md

README.md

@@ -2,7 +2,7 @@
 base_model: unsloth/meta-llama-3.1-8b-instruct-bnb-4bit
 language:
 - en
-license: apache-2.0
+license: mit
 tags:
 - text-generation-inference
 - transformers
@@ -10,14 +10,95 @@ tags:
 - llama
 - trl
 - sft
+datasets:
+- LimYeri/LeetCode_Python_Solutions_Data
 ---
 
-# Uploaded  model
+# CodeMind-Llama3.1-8B-unsloth
 
-- **Developed by:** LimYeri
-- **License:** apache-2.0
-- **Finetuned from model :** unsloth/meta-llama-3.1-8b-instruct-bnb-4bit
+**Codemind Project** is a language model developed to assist in solving and learning coding test problems. This model is fine-tuned using posts written by LeetCode users as training data, aiming to provide answers specialized for coding tests.
 
-This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
 
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
+## Model Information
+- **Base Model**: `meta-llama/Meta-Llama-3.1-8B-Instruct`
+- **Fine-Tuning**: Fine-tuned using the unsloth library based on the `unsloth/Meta-Llama-3.1-8B-Instruct` model
+- **Fine-Tuning Process**: Conducted with reference to the Llama 3.1 Conversational_notebook
+
+
+## Dataset Used
+- LeetCode Python Solutions Dataset: [LeetCode_Python_Solutions_Data](https://huggingface.co/datasets/LimYeri/LeetCode_Python_Solutions_Data)
+
+
+## How to Use the Model
+This model is accessible through HuggingFace's model hub and can be integrated into applications using the API. It is designed to generate explanations, code snippets, or guides for coding problems or programming-related questions.
+
+```python
+# 자세한 사항은 demo-Llama3.1.ipynb 확인
+from unsloth import FastLanguageModel
+from unsloth.chat_templates import get_chat_template
+from IPython.display import display, Markdown
+
+model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name = "LimYeri/CodeMind-Llama3.1-8B-unsloth", # YOUR MODEL YOU USED FOR TRAINING
+    max_seq_length = max_seq_length,
+    dtype = dtype,
+    load_in_4bit = load_in_4bit,
+)
+
+tokenizer = get_chat_template(
+    tokenizer,
+    chat_template = "llama-3.1",
+)
+FastLanguageModel.for_inference(model) # Enable native 2x faster inference
+
+messages = [
+    {"role": "system", "content": "You are a kind coding test teacher."},
+    {"role": "user", "content": "Enter your coding problem or question here."},
+]
+inputs = tokenizer.apply_chat_template(
+    messages,
+    tokenize = True,
+    add_generation_prompt = True, # Must add for generation
+    return_tensors = "pt",
+).to("cuda")
+
+outputs = model.generate(input_ids = inputs, max_new_tokens = 3000, use_cache = True,
+                         temperature = 0.5, min_p = 0.3) # Feel free to adjust the temperature and min_p
+text = (tokenizer.batch_decode(outputs))[0].split('assistant<|end_header_id|>\n\n')[1].strip()
+display(Markdown(text))
+```
+
+## LoRA Configuration
+- **r**: 16
+- **lora_alpha**: 16
+- **lora_dropout**: 0
+- **bias**: "none"
+- **use_gradient_checkpointing**: "unsloth"
+
+## Training Settings
+- **Per Device Train Batch Size**: 8
+- **Gradient Accumulation Steps**: 2
+- **Warmup Steps**: 200
+- **Number of Training Epochs**: 5
+- **Learning Rate**: 2e-4
+- **fp16**: not `is_bfloat16_supported()`
+- **bf16**: `is_bfloat16_supported()`
+- **Logging Steps**: 20
+- **Optimizer**: "adamw_8bit"
+- **Weight Decay**: 0.01
+- **LR Scheduler Type**: "linear"
+
+## Evaluation Results [Open LLM Leaderboard](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
+| Metric       | Value  |
+|--------------|--------|
+| Average      | 22.17  |
+| IFEval       | 64.9   |
+| BBH          | 24.19  |
+| MATH Lvl 5   | 9.97   |
+| GPQA         | 1.9    |
+| MUSR         | 6.04   |
+| MMLU-PRO     | 26     |
+
+
+## Fine-Tuning Code
+Detailed fine-tuning code and settings can be found in the [CodeMind-Extended GitHub repository](https://github.com/LimYeri/CodeMind-Extended).