Update README.md
Browse files
    	
        README.md
    CHANGED
    
    | @@ -8,14 +8,23 @@ base_model: | |
| 8 | 
             
            - meta-llama/Llama-3.1-8B
         | 
| 9 | 
             
            ---
         | 
| 10 |  | 
| 11 | 
            -
            This is the tuned lens version of LLama 3.1 8B (i.e. all the layers of the model have been trained to minimize the KL divergence with the last layer).
         | 
| 12 |  | 
| 13 | 
             
            ```bash
         | 
| 14 | 
             
            pip install tuned-lens
         | 
| 15 | 
             
            ```
         | 
| 16 | 
             
            ```bash
         | 
| 17 | 
            -
            python -m tuned_lens train  | 
|  | |
|  | |
|  | |
|  | |
| 18 | 
             
            ```
         | 
| 19 | 
             
            ```bash
         | 
| 20 | 
            -
            python -m tuned_lens eval  | 
|  | |
|  | |
|  | |
|  | |
|  | |
| 21 | 
             
            ```
         | 
|  | |
| 8 | 
             
            - meta-llama/Llama-3.1-8B
         | 
| 9 | 
             
            ---
         | 
| 10 |  | 
| 11 | 
            +
            This is the tuned lens version of LLama 3.1 8B (i.e. all the layers of the model have been trained to minimize the KL divergence with the last layer). 
         | 
| 12 |  | 
| 13 | 
             
            ```bash
         | 
| 14 | 
             
            pip install tuned-lens
         | 
| 15 | 
             
            ```
         | 
| 16 | 
             
            ```bash
         | 
| 17 | 
            +
            python -m tuned_lens train \
         | 
| 18 | 
            +
              --model.name meta-llama/Llama-3.1-8B \
         | 
| 19 | 
            +
              --data.name bookcorpus/bookcorpus \
         | 
| 20 | 
            +
              --per_gpu_batch_size=1 \
         | 
| 21 | 
            +
              --output my_lenses/meta-llama/Llama-3.1-8B
         | 
| 22 | 
             
            ```
         | 
| 23 | 
             
            ```bash
         | 
| 24 | 
            +
            python -m tuned_lens eval \
         | 
| 25 | 
            +
              --data.name bookcorpus/bookcorpus \
         | 
| 26 | 
            +
              --model.name meta-llama/Llama-3.1-8B \
         | 
| 27 | 
            +
              --tokens 16400000 \
         | 
| 28 | 
            +
              --lens_name my_lenses/meta-llama/Llama-3.1-8B \
         | 
| 29 | 
            +
              --output evaluation/meta-llama/Llama-3.1-8B
         | 
| 30 | 
             
            ```
         | 
