Hugging Face's logo

REILX
/
Llama-3-8B-Instruct-750Mb-lora

Safetensors
Chinese
English
text-generation-inference
llama
chat
sft
lora
Model card Files Community

数据集

使用以下8个数据集 image/png 对Llama-3-8B-Instruct进行微调。

基础模型：

训练工具

https://github.com/hiyouga/LLaMA-Factory

测评方式：

使用opencompass(https://github.com/open-compass/OpenCompass/ )， 测试工具基于CEval和MMLU对微调之后的模型和原始模型进行测试。
测试模型分别为：

  • Llama-3-8B
  • Llama-3-8B-Instruct
  • Llama-3-8B-Instruct-750Mb-lora, 使用8DataSets数据集对Llama-3-8B-Instruct模型进行sft方式lora微调

测试机器

8*A800

8DataSets数据集：

大约750Mb的微调数据集

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 4
  • eval_batch_size: 8
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 8
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 128
  • total_eval_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 300
  • num_epochs: 1.0
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no library tag.

Model tree for REILX/Llama-3-8B-Instruct-750Mb-lora

Merges
1 model

Datasets used to train REILX/Llama-3-8B-Instruct-750Mb-lora

Collection including REILX/Llama-3-8B-Instruct-750Mb-lora