File size: 2,982 Bytes
c941350 254da59 68b425f 254da59 68b425f 254da59 68b425f 254da59 68b425f 254da59 68b425f 254da59 68b425f 254da59 68b425f 254da59 68b425f 254da59 68b425f 254da59 40a17c2 254da59 68b425f 254da59 40a17c2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 |
# codellama2-finetuned-nl2bash-fin
Finetuned on the [AnishJoshi/nl2bash-custom](https://huggingface.co/datasets/AnishJoshi/nl2bash-custom) dataset for generating bash code based on natural language descriptions.
## Model Details
- **Model Name:** CodeLlama2-Finetuned-NL2Bash
- **Base Model:** CodeLlama2
- **Task:** Natural Language to Bash Script Conversion
- **Framework:** PyTorch
- **Fine-tuning Dataset:** Custom dataset of natural language commands and corresponding Bash scripts, available [here](https://huggingface.co/datasets/AnishJoshi/nl2bash-custom)
### Model Description
- **Developed by:** Anish Joshi
- **Model type:** CausalLM
- **Finetuned from model:** Codellama2
## Files Included
- `adapter_config.json`: Configuration file for the adapter layers.
- `adapter_model.safetensors`: Weights of the adapter layers in the Safetensors format.
- `optimizer.pt`: State of the optimizer used during training.
- `rng_state.pth`: State of the random number generator.
- `scheduler.pt`: State of the learning rate scheduler.
- `special_tokens_map.json`: Mapping for special tokens used by the tokenizer.
- `tokenizer.json`: Tokenizer model including the vocabulary.
- `tokenizer_config.json`: Configuration settings for the tokenizer.
- `trainer_state.json`: State of the trainer including training metrics.
- `training_args.bin`: Training arguments used for fine-tuning.
- `README.md
## Usage
# Load model directly
```
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("AnishJoshi/codellama2-finetuned-nl2bash-fin")
model = AutoModelForCausalLM.from_pretrained("AnishJoshi/codellama2-finetuned-nl2bash-fin")
```
## Training Details
Training details available at [Finetuning Notebook](https://github.com/AnishJoshi13/Bash-Scripting-Assistant/blob/master/notebooks/finetuning.ipynb)
#### Training Hyperparameters
Training arguments and configuration are set using TrainingArguments and LoraConfig. The model is fine-tuned using the following parameters:
- `output_dir: codellama2-finetuned-nl2bash` - Directory to save the fine-tuned model.
- `per_device_train_batch_size`: 2 - Batch size per device.
- `gradient_accumulation_steps`: 16 - Number of gradient accumulation steps.
- `optim`: paged_adamw_32bit - Optimizer type.
- `learning_rate`: 2e-4 - Learning rate.
- `lr_scheduler_type`: cosine - Learning rate scheduler type.
- `save_strategy`: epoch - Strategy to save checkpoints.
- `logging_steps`: 10 - Number of steps between logging.
- `num_train_epochs`: 1 - Number of training epochs.
- `max_steps`: 100 - Maximum number of training steps.
- `fp16`: True - Use 16-bit floating-point precision.
- `push_to_hub`: False - Whether to push the model to Hugging Face Hub.
- `report_to`: none - Reporting destination.
## Evaluation
Evaulation metrics and calculations available at [Evaluation Notebook](https://github.com/AnishJoshi13/Bash-Scripting-Assistant/blob/master/notebooks/evaluation.ipynb) |