Tuned Lens
Collection
Models fine-tuned by minimizing KL between each layer and the final unembedding matrix
•
3 items
•
Updated
This is the tuned lens version of LLama 3.1 8B (i.e. all the layers of the model have been trained to minimize the KL divergence with the last layer).
pip install tuned-lens
python -m tuned_lens train \
--model.name meta-llama/Llama-3.1-8B \
--data.name bookcorpus/bookcorpus \
--per_gpu_batch_size=1 \
--output my_lenses/meta-llama/Llama-3.1-8B
python -m tuned_lens eval \
--data.name bookcorpus/bookcorpus \
--model.name meta-llama/Llama-3.1-8B \
--tokens 16400000 \
--lens_name my_lenses/meta-llama/Llama-3.1-8B \
--output evaluation/meta-llama/Llama-3.1-8B
- Downloads last month
- 4
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
Model tree for kdhole/Llama-3.1-8B-tuned-lens
Base model
meta-llama/Llama-3.1-8B