πŸ₯ Base Model

EleutherAI/polyglot-ko-1.3b

This model is based on the Korean version of Polyglot-1.3B, an open-source language model released by EleutherAI.
It is pre-trained on a large-scale Korean corpus and designed for general-purpose Korean language understanding and generation tasks.


Training Procedure

Training Hyperparameters

The following hyperparameters were used during training:

  • output_dir: ./qlora_model_eleutherai
  • per_device_train_batch_size: 2
  • gradient_accumulation_steps: 4
  • total_batch_size: 8 (2 x 4)
  • learning_rate: 2e-5
  • num_train_epochs: 2
  • fp16: True
  • logging_dir: ./logs
  • logging_steps: 5
  • save_steps: 100
  • save_total_limit: 1
  • load_best_model_at_end: True
  • metric_for_best_model: loss
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for piboo/PiBoo

Finetuned
(18)
this model