End of training

README.md

@@ -13,101 +13,6 @@ model-index:
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-[<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
-<details><summary>See axolotl config</summary>
-
-axolotl version: `0.4.1`
-```yaml
-adapter: lora
-base_model: facebook/opt-125m
-bf16: auto
-chat_template: llama3
-dataset_prepared_path: null
-datasets:
-- data_files:
-  - e9426b45bbc30495_train_data.json
-  ds_type: json
-  format: custom
-  path: /workspace/input_data/e9426b45bbc30495_train_data.json
-  type:
-    field_input: original_version
-    field_instruction: title
-    field_output: french_version
-    format: '{instruction} {input}'
-    no_input_format: '{instruction}'
-    system_format: '{system}'
-    system_prompt: ''
-debug: null
-deepspeed: null
-device_map: auto
-do_eval: true
-early_stopping_patience: 3
-eval_batch_size: 4
-eval_max_new_tokens: 128
-eval_steps: 300
-eval_table_size: null
-evals_per_epoch: null
-flash_attention: true
-fp16: false
-fsdp: null
-fsdp_config: null
-gradient_accumulation_steps: 4
-gradient_checkpointing: false
-group_by_length: true
-hub_model_id: JacksonBrune/646255e0-25a0-42b8-b4bd-76e703084fb9
-hub_strategy: end
-learning_rate: 0.0002
-load_in_4bit: false
-load_in_8bit: false
-local_rank: null
-logging_steps: 50
-lora_alpha: 64
-lora_dropout: 0.05
-lora_fan_in_fan_out: null
-lora_model_dir: null
-lora_r: 32
-lora_target_linear: true
-lr_scheduler: constant
-max_grad_norm: 1.0
-max_memory:
-  0: 75GB
-max_steps: 20953
-micro_batch_size: 4
-mlflow_experiment_name: /tmp/e9426b45bbc30495_train_data.json
-model_type: AutoModelForCausalLM
-num_epochs: 10
-optim_args:
-  adam_beta1: 0.9
-  adam_beta2: 0.95
-  adam_epsilon: 1e-5
-optimizer: adamw_bnb_8bit
-output_dir: miner_id_24
-pad_to_sequence_len: true
-resume_from_checkpoint: null
-s2_attention: null
-sample_packing: false
-save_steps: 300
-saves_per_epoch: null
-sequence_len: 512
-strict: false
-tf32: true
-tokenizer_type: AutoTokenizer
-train_on_inputs: false
-trust_remote_code: true
-val_set_size: 0.05
-wandb_entity: null
-wandb_mode: online
-wandb_name: 4dbfff5d-e150-40fd-bcd9-78e926ac420c
-wandb_project: SN56-18
-wandb_run: your_name
-wandb_runid: 4dbfff5d-e150-40fd-bcd9-78e926ac420c
-warmup_steps: 50
-weight_decay: 0.0
-xformers_attention: null
-
-```
-
-</details><br>
 
 # 646255e0-25a0-42b8-b4bd-76e703084fb9
 
@@ -127,87 +32,6 @@ More information needed
 
 More information needed
 
-## Training procedure
-
-### Training hyperparameters
-
-The following hyperparameters were used during training:
-- learning_rate: 0.0002
-- train_batch_size: 4
-- eval_batch_size: 4
-- seed: 42
-- gradient_accumulation_steps: 4
-- total_train_batch_size: 16
-- optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=adam_beta1=0.9,adam_beta2=0.95,adam_epsilon=1e-5
-- lr_scheduler_type: constant
-- lr_scheduler_warmup_steps: 50
-- training_steps: 20953
-
-### Training results
-
-| Training Loss | Epoch  | Step  | Validation Loss |
-|:-------------:|:------:|:-----:|:---------------:|
-| No log        | 0.0002 | 1     | 2.5731          |
-| 8.1083        | 0.0510 | 300   | 1.9096          |
-| 7.1078        | 0.1021 | 600   | 1.6839          |
-| 6.6696        | 0.1531 | 900   | 1.5693          |
-| 6.2632        | 0.2042 | 1200  | 1.5010          |
-| 6.0785        | 0.2552 | 1500  | 1.4457          |
-| 5.8829        | 0.3063 | 1800  | 1.4097          |
-| 5.6869        | 0.3573 | 2100  | 1.3755          |
-| 5.5791        | 0.4083 | 2400  | 1.3511          |
-| 5.7238        | 0.4594 | 2700  | 1.3324          |
-| 5.5444        | 0.5104 | 3000  | 1.3186          |
-| 5.648         | 0.5615 | 3300  | 1.2941          |
-| 5.5452        | 0.6125 | 3600  | 1.2857          |
-| 5.3401        | 0.6635 | 3900  | 1.2733          |
-| 5.345         | 0.7146 | 4200  | 1.2610          |
-| 5.2741        | 0.7656 | 4500  | 1.2497          |
-| 5.1186        | 0.8167 | 4800  | 1.2454          |
-| 5.2047        | 0.8677 | 5100  | 1.2338          |
-| 5.3053        | 0.9188 | 5400  | 1.2297          |
-| 5.2865        | 0.9698 | 5700  | 1.2233          |
-| 5.2367        | 1.0208 | 6000  | 1.2095          |
-| 5.2783        | 1.0719 | 6300  | 1.2095          |
-| 5.0683        | 1.1229 | 6600  | 1.2021          |
-| 5.0163        | 1.1740 | 6900  | 1.1992          |
-| 4.896         | 1.2250 | 7200  | 1.1973          |
-| 4.8037        | 1.2761 | 7500  | 1.1908          |
-| 4.9946        | 1.3271 | 7800  | 1.1821          |
-| 5.08          | 1.3781 | 8100  | 1.1812          |
-| 5.0091        | 1.4292 | 8400  | 1.1765          |
-| 4.9605        | 1.4802 | 8700  | 1.1745          |
-| 5.1488        | 1.5313 | 9000  | 1.1679          |
-| 4.7554        | 1.5823 | 9300  | 1.1685          |
-| 5.0706        | 1.6333 | 9600  | 1.1651          |
-| 4.8837        | 1.6844 | 9900  | 1.1624          |
-| 4.8728        | 1.7354 | 10200 | 1.1567          |
-| 4.8636        | 1.7865 | 10500 | 1.1549          |
-| 4.8859        | 1.8375 | 10800 | 1.1524          |
-| 4.9733        | 1.8886 | 11100 | 1.1537          |
-| 4.6105        | 1.9396 | 11400 | 1.1523          |
-| 4.9071        | 1.9906 | 11700 | 1.1477          |
-| 4.8601        | 2.0417 | 12000 | 1.1421          |
-| 4.7578        | 2.0927 | 12300 | 1.1450          |
-| 4.7045        | 2.1438 | 12600 | 1.1386          |
-| 4.7653        | 2.1948 | 12900 | 1.1393          |
-| 4.9627        | 2.2459 | 13200 | 1.1378          |
-| 4.5644        | 2.2969 | 13500 | 1.1343          |
-| 4.6682        | 2.3479 | 13800 | 1.1316          |
-| 4.8793        | 2.3990 | 14100 | 1.1303          |
-| 4.5809        | 2.4500 | 14400 | 1.1302          |
-| 4.7643        | 2.5011 | 14700 | 1.1256          |
-| 4.6261        | 2.5521 | 15000 | 1.1251          |
-| 4.7251        | 2.6031 | 15300 | 1.1243          |
-| 4.7382        | 2.6542 | 15600 | 1.1213          |
-| 4.9042        | 2.7052 | 15900 | 1.1206          |
-| 4.8754        | 2.7563 | 16200 | 1.1232          |
-| 4.552         | 2.8073 | 16500 | 1.1160          |
-| 4.6477        | 2.8584 | 16800 | 1.1167          |
-| 4.7991        | 2.9094 | 17100 | 1.1170          |
-| 4.6374        | 2.9604 | 17400 | 1.1171          |
-
-
 ### Framework versions
 
 - PEFT 0.13.2