File size: 282 Bytes
9883090
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
```python
python -m moetify.mix \
   --output_dir ./moecule-3x1b-m4 \
   --model_path  unsloth/Llama-3.2-1B-Instruct \
   --modules mlp q_proj \
   --ingredients \
       unsloth/Llama-3.2-1B-Instruct \
       unsloth/Llama-3.2-1B-Instruct \
       unsloth/Llama-3.2-1B-Instruct
```