Update README.md
Browse files
README.md
CHANGED
@@ -32,7 +32,7 @@ The following table outlines the key hyperparameters used for training Romulus.
|
|
32 |
|----------------------------------|-----------------------------------------------------------------|-----------------------------|
|
33 |
| `max_seq_length` | Maximum sequence length for the model | 4096 |
|
34 |
| `load_in_4bit` | Whether to load the model in 4-bit precision | False |
|
35 |
-
| `model_name` | Pre-trained model name from Hugging Face | meta-llama/Meta-Llama-3.1-8B|
|
36 |
| `r` | Rank of the LoRA adapter | 128 |
|
37 |
| `lora_alpha` | Alpha value for the LoRA module | 32 |
|
38 |
| `lora_dropout` | Dropout rate for LoRA layers | 0 |
|
|
|
32 |
|----------------------------------|-----------------------------------------------------------------|-----------------------------|
|
33 |
| `max_seq_length` | Maximum sequence length for the model | 4096 |
|
34 |
| `load_in_4bit` | Whether to load the model in 4-bit precision | False |
|
35 |
+
| `model_name` | Pre-trained model name from Hugging Face | meta-llama/Meta-Llama-3.1-8B-Instruct|
|
36 |
| `r` | Rank of the LoRA adapter | 128 |
|
37 |
| `lora_alpha` | Alpha value for the LoRA module | 32 |
|
38 |
| `lora_dropout` | Dropout rate for LoRA layers | 0 |
|