kazukifujii commited on
Commit
e7e4d37
·
verified ·
1 Parent(s): c9bf74c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -19,7 +19,7 @@ base_model:
19
  This model is a continual pre-training of [Llama-3.1-8B](https://huggingface.co/meta-llama/Llama-3.1-8B) on a mix of mathematical datasets from finemath-4+ and multilingual text datasets.
20
  The model was trained to evaluate the performance of mathematical reasoning and problem-solving as part of the SwallowMath ablation experiments (experiment 1).
21
 
22
- It was trained on **50 billion tokens** using a mix of 4.8% Finemath-4+, 13.1% Code, and 82% multilingual text, following the setup described in the [SwallowMath paper](https://arxiv.org/abs/XXXX.XXXXX).
23
  Training was performed using [Megatron-LM](https://github.com/NVIDIA/Megatron-LM/tree/core_r0.9.0).
24
 
25
  ## Use
 
19
  This model is a continual pre-training of [Llama-3.1-8B](https://huggingface.co/meta-llama/Llama-3.1-8B) on a mix of mathematical datasets from finemath-4+ and multilingual text datasets.
20
  The model was trained to evaluate the performance of mathematical reasoning and problem-solving as part of the SwallowMath ablation experiments (experiment 1).
21
 
22
+ It was trained on **50 billion tokens** using a mix of 4.8% Finemath-4+, 13.1% Code, and 82% multilingual text, following the setup described in the [SwallowMath paper](https://arxiv.org/abs/2505.02881).
23
  Training was performed using [Megatron-LM](https://github.com/NVIDIA/Megatron-LM/tree/core_r0.9.0).
24
 
25
  ## Use