llamafy/ByteDance-Seed_Seed-OSS-36B-Instruct-llamafied

ByteDance-Seed/Seed-OSS-36B-Instruct (llamafied)

This is a version of ByteDance-Seed/Seed-OSS-36B-Instruct converted to the Llama format. It should be compatible with all programs that support Llama.

Output is token-identical to the original weights when tested with bitsandbytes:

~/AI/scripts
venv ❯ python test_byte.py
ByteDance-Seed_Seed-OSS-36B-Instruct
The `load_in_4bit` and `load_in_8bit` arguments are deprecated and will be removed in the future versions. Please, pass a `BitsAndBytesConfig` object in `quantization_config` argument instead.
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 15/15 [00:28<00:00,  1.92s/it]
The following generation flags are not valid and may be ignored: ['temperature', 'top_p']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
<seed:bos>system
You are an intelligent assistant that can answer questions in one step without the need for reasoning and thinking, that is, your thinking budget is 0. Next, please skip the thinking process and directly start answering the user's questions.
<seed:eos><seed:bos>user
How to make pasta?<seed:eos><seed:bos>assistant
<seed:think><seed:cot_budget_reflect>The current thinking budget is 0, so I will directly start answering the question.</seed:cot_budget_reflect>
</seed:think>To make pasta, follow these key steps:  


### **1. Prepare the Dough**  
- **Ingredients**: 500g (3½ cups) all-purpose or bread

venv ❯ python test_byte_llamafied.py
ByteDance-Seed_Seed-OSS-36B-Instruct-llamafied
The `load_in_4bit` and `load_in_8bit` arguments are deprecated and will be removed in the future versions. Please, pass a `BitsAndBytesConfig` object in `quantization_config` argument instead.
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 40/40 [00:19<00:00,  2.07it/s]
The following generation flags are not valid and may be ignored: ['temperature', 'top_p']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
<seed:bos>system
You are an intelligent assistant that can answer questions in one step without the need for reasoning and thinking, that is, your thinking budget is 0. Next, please skip the thinking process and directly start answering the user's questions.
<seed:eos><seed:bos>user
How to make pasta?<seed:eos><seed:bos>assistant
<seed:think><seed:cot_budget_reflect>The current thinking budget is 0, so I will directly start answering the question.</seed:cot_budget_reflect>
</seed:think>To make pasta, follow these key steps:  


### **1. Prepare the Dough**  
- **Ingredients**: 500g (3½ cups) all-purpose or bread