π₯ Base Model
EleutherAI/polyglot-ko-1.3b
This model is based on the Korean version of Polyglot-1.3B, an open-source language model released by EleutherAI.
It is pre-trained on a large-scale Korean corpus and designed for general-purpose Korean language understanding and generation tasks.
Training Procedure
Training Hyperparameters
The following hyperparameters were used during training:
output_dir
:./qlora_model_eleutherai
per_device_train_batch_size
:2
gradient_accumulation_steps
:4
total_batch_size
:8 (2 x 4)
learning_rate
:2e-5
num_train_epochs
:2
fp16
:True
logging_dir
:./logs
logging_steps
:5
save_steps
:100
save_total_limit
:1
load_best_model_at_end
:True
metric_for_best_model
:loss
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
π
Ask for provider support
Model tree for piboo/PiBoo
Base model
EleutherAI/polyglot-ko-1.3b