README.md

metadata

language:
  - en
tags:
  - causal-lm
library_name: transformers
license: apache-2.0
datasets:
  - allenai/dolma

Model Details

Training

Models trained using litgpt and AxoNN on AMD MI250 GPUs.

Data

Train and validation data is taken from non-overlapping subsets of dolma.