Update Readme

README.md

@@ -1,62 +1,101 @@
+
 ---
-base_model: meta-llama/Llama-3.2-3B-Instruct
-library_name: peft
-model_name: results
+# Model Card metadata: https://huggingface.co/docs/hub/model-cards#model-card-metadata
+license: apache-2.0
+language:
+- en
 tags:
-- base_model:adapter:meta-llama/Llama-3.2-3B-Instruct
-- lora
-- sft
-- transformers
-- trl
-licence: license
+- llm
+- fine-tune
+- qlora
+- llama
+- bitcoin
+- finance
 pipeline_tag: text-generation
+base_model: meta-llama/Llama-3.2-3B-Instruct
+datasets:
+- tahamajs/bitcoin-llm-finetuning-dataset
 ---
+```
+
+### 📋 Overview
 
-# Model Card for results
+This model, `llama-3.2-3b-instruct-bitcoin-analyst_best`, is a fine-tuned version of the **Llama-3.2-3B-Instruct** large language model. It has been specialized for the domain of **Bitcoin analysis and cryptocurrency**. The goal of this fine-tuning was to enhance the model's ability to provide detailed, accurate, and contextually relevant information about Bitcoin, blockchain technology, market trends, and related topics, acting as a virtual Bitcoin analyst.
 
-This model is a fine-tuned version of [meta-llama/Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct).
-It has been trained using [TRL](https://github.com/huggingface/trl).
+The fine-tuning was performed using **QLoRA** on the `tahamajs/bitcoin-llm-finetuning-dataset` dataset.
 
-## Quick start
+### 🚀 Usage
+
+You can easily use this model with the `transformers` library. The fine-tuned weights are stored as a PEFT adapter.
 
 ```python
-from transformers import pipeline
+import torch
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+# Load the base model
+base_model_id = "meta-llama/Llama-3.2-3B-Instruct"
+tokenizer = AutoTokenizer.from_pretrained(base_model_id)
+base_model = AutoModelForCausalLM.from_pretrained(
+    base_model_id,
+    device_map="auto",
+    torch_dtype=torch.bfloat16,
+)
+
+# Load the fine-tuned adapter
+peft_model_id = "tahamajs/llama-3.2-3b-instruct-bitcoin-analyst_best"
+model = PeftModel.from_pretrained(base_model, peft_model_id)
 
-question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
-generator = pipeline("text-generation", model="tahamajs/results", device="cuda")
-output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
-print(output["generated_text"])
+# Example inference
+prompt = "What are the key differences between Bitcoin and Ethereum?"
+messages = [
+    {"role": "user", "content": prompt}
+]
+input_ids = tokenizer.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    return_tensors="pt"
+).to(model.device)
+
+outputs = model.generate(input_ids=input_ids, max_new_tokens=256)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 ```
 
-## Training procedure
+### 💻 Training Details
+
+This section provides an overview of the fine-tuning process.
 
- 
+  * **Base Model:** `meta-llama/Llama-3.2-3B-Instruct`
+  * **Dataset:** `tahamajs/bitcoin-llm-finetuning-dataset`
+  * **Fine-Tuning Method:** QLoRA (Quantized Low-Rank Adaptation)
+  * **Training Framework:** `trl.SFTTrainer`
+  * **Hardware:** [E.g., NVIDIA RTX 4070, 16GB VRAM]
+  * **Software Stack:** PyTorch, Transformers, TRL, PEFT, BitsAndBytes
 
+#### ⚙️ Hyperparameters
 
-This model was trained with SFT.
+The following hyperparameters were used for fine-tuning:
 
-### Framework versions
+| Hyperparameter              | Value                      |
+| :-------------------------- | :------------------------- |
+| `num_train_epochs`          | 1                          |
+| `per_device_train_batch_size` | 1                          |
+| `gradient_accumulation_steps` | 2                          |
+| `learning_rate`             | 2e-4                       |
+| `optim`                     | `paged_adamw_32bit`        |
+| `bf16`                      | `True`                     |
+| `max_grad_norm`             | 0.3                        |
+| `r` (LoRA rank)             | 16                         |
+| `lora_alpha`                | 16                         |
 
-- PEFT 0.17.0
-- TRL: 0.21.0
-- Transformers: 4.55.0
-- Pytorch: 2.6.0+cu124
-- Datasets: 4.0.0
-- Tokenizers: 0.21.4
+### ⚠️ Limitations and Biases
 
-## Citations
+As a model fine-tuned on a specific dataset, it may have the following limitations:
 
+  * **Domain Specificity:** The model's knowledge is primarily focused on Bitcoin and cryptocurrency. It may perform less effectively on general knowledge tasks.
+  * **Data Cutoff:** The model's knowledge is limited to the data it was trained on. It may not be aware of events, market changes, or new developments that occurred after the dataset's creation.
+  * **Potential Biases:** The model's responses may reflect biases present in the training data.
 
+### 📜 License
 
-Cite TRL as:
-    
-```bibtex
-@misc{vonwerra2022trl,
-	title        = {{TRL: Transformer Reinforcement Learning}},
-	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
-	year         = 2020,
-	journal      = {GitHub repository},
-	publisher    = {GitHub},
-	howpublished = {\url{https://github.com/huggingface/trl}}
-}
-```
+This model is licensed under the Apache 2.0 license, inherited from its base model.