---
license: openrail
datasets:
- bookcorpus/bookcorpus
language:
- en
base_model:
- meta-llama/Llama-3.1-8B
---

This is the tuned lens version of LLama 3.1 8B (i.e. all the layers of the model have been trained to minimize the KL divergence with the last layer). 

```bash
pip install tuned-lens
```
```bash
python -m tuned_lens train \
  --model.name meta-llama/Llama-3.1-8B \
  --data.name bookcorpus/bookcorpus \
  --per_gpu_batch_size=1 \
  --output my_lenses/meta-llama/Llama-3.1-8B
```
```bash
python -m tuned_lens eval \
  --data.name bookcorpus/bookcorpus \
  --model.name meta-llama/Llama-3.1-8B \
  --tokens 16400000 \
  --lens_name my_lenses/meta-llama/Llama-3.1-8B \
  --output evaluation/meta-llama/Llama-3.1-8B
```