README.md

metadata

datasets:
  - allenai/dolma
language:
  - en
library_name: transformers
license: apache-2.0
tags:
  - causal-lm

Model Details

Training

Models trained using litgpt and AxoNN on AMD MI250 GPUs.

Data

Train and validation data is taken from non-overlapping subsets of dolma.