|
|
--- |
|
|
library_name: transformers |
|
|
license: other |
|
|
license_name: lfm1.0 |
|
|
license_link: LICENSE |
|
|
tags: |
|
|
- unsloth |
|
|
- trl |
|
|
- sft |
|
|
datasets: |
|
|
- HelpingAI/Intermediate-Thinking-130k |
|
|
language: |
|
|
- en |
|
|
base_model: |
|
|
- LiquidAI/LFM2-1.2B |
|
|
pipeline_tag: text-generation |
|
|
--- |
|
|
|
|
|
<center> |
|
|
<div style="text-align: center;"> |
|
|
<img |
|
|
src="https://cdn-uploads.huggingface.co/production/uploads/63b6f2e752c02ae8acbaa4d8/Qn87jdxhfCaqQUiAqD_60.png" |
|
|
alt="Liquid with Thinking" |
|
|
style="width: 100%; max-width: 66%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em;" |
|
|
/> |
|
|
</div> |
|
|
</center> |
|
|
|
|
|
--- |
|
|
|
|
|
# Liquid-Thinking-Preview |
|
|
|
|
|
This repository contains a full-parameter fine-tuned version of `LiquidAI/LFM2-1.2B` on the `HelpingAI/Intermediate-Thinking-130k` dataset. The goal of this fine-tune is to enhance the model's ability to perform step-by-step reasoning and solve problems that require intermediate thought processes. |
|
|
|
|
|
This model was trained with high efficiency using the [Unsloth](https://github.com/unslothai/unsloth) library. |
|
|
|
|
|
- **GGUF Quantizations:** For CPU inference, GGUF versions are available in the [GGUF repository here](https://huggingface.co/kreasof-ai/Liquid-Thinking-Preview-GGUF). |
|
|
## Model Details |
|
|
|
|
|
* **Base Model:** [LiquidAI/LFM2-1.2B](https://huggingface.co/LiquidAI/LFM2-1.2B) |
|
|
* **Dataset:** [HelpingAI/Intermediate-Thinking-130k](https://huggingface.co/datasets/HelpingAI/Intermediate-Thinking-130k) |
|
|
* **Fine-tuning Method:** Full Parameter Fine-tuning. |
|
|
* **Frameworks:** Unsloth, Transformers, PyTorch, TRL. |
|
|
|
|
|
## How to Use with Unsloth |
|
|
|
|
|
You can easily run this model for fast inference using the `unsloth` library. |
|
|
|
|
|
First, install the necessary packages: |
|
|
```bash |
|
|
pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git" |
|
|
pip install --no-deps git+https://github.com/huggingface/transformers.git |
|
|
pip install --no-deps causal-conv1d==1.5.0.post8 |
|
|
``` |
|
|
|
|
|
Next, use the following Python code to load the model and generate text: |
|
|
|
|
|
```python |
|
|
from unsloth import FastModel |
|
|
import torch |
|
|
from transformers import TextStreamer |
|
|
|
|
|
model_repo_id = "kreasof-ai/Liquid-Thinking-Preview" |
|
|
|
|
|
model, tokenizer = FastModel.from_pretrained( |
|
|
model_name = model_repo_id, |
|
|
dtype = None, # None for auto detection |
|
|
load_in_4bit = True, # Use 4bit quantization for faster inference |
|
|
) |
|
|
|
|
|
# Define the conversation |
|
|
messages = [{ |
|
|
"role": "user", |
|
|
"content": "Explain the step-by-step process of photosynthesis in simple terms.", |
|
|
}] |
|
|
|
|
|
# Apply the chat template |
|
|
inputs = tokenizer.apply_chat_template( |
|
|
messages, |
|
|
add_generation_prompt = True, # Must add for generation |
|
|
return_tensors = "pt", |
|
|
).to("cuda") |
|
|
|
|
|
# Set up the streamer for real-time output |
|
|
streamer = TextStreamer(tokenizer, skip_prompt = True) |
|
|
|
|
|
# Generate the response |
|
|
_ = model.generate( |
|
|
**inputs, |
|
|
max_new_tokens = 4096, |
|
|
# Recommended generation parameters from LiquidAI |
|
|
temperature = 0.3, |
|
|
min_p = 0.15, |
|
|
repetition_penalty = 1.05, |
|
|
use_cache = True, |
|
|
streamer = streamer, |
|
|
) |
|
|
``` |
|
|
|
|
|
## Training Procedure |
|
|
|
|
|
### Data Preprocessing |
|
|
The `HelpingAI/Intermediate-Thinking-130k` dataset, consisting of `instruction` and `output` pairs, was formatted into a conversational format using the LFM2's chat template. The template follows this structure: |
|
|
|
|
|
``` |
|
|
<|im_start|>user |
|
|
{instruction}<|im_end|> |
|
|
<|im_start|>assistant |
|
|
{output}<|im_end|> |
|
|
``` |
|
|
|
|
|
The training loss was calculated **only on the assistant's responses** (`train_on_responses_only`), encouraging the model to learn the desired output format and reasoning style without being penalized for the user's prompt. |
|
|
|
|
|
### Hyperparameters |
|
|
The model was trained with the following hyperparameters using `trl.SFTTrainer`: |
|
|
|
|
|
| Hyperparameter | Value | |
|
|
| ----------------------------- | --------------- | |
|
|
| `per_device_train_batch_size` | 8 | |
|
|
| `gradient_accumulation_steps` | 4 | |
|
|
| `Effective Batch Size` | 32 | |
|
|
| `num_train_epochs` | 3 | |
|
|
| `learning_rate` | 4e-5 | |
|
|
| `lr_scheduler_type` | `cosine` | |
|
|
| `warmup_ratio` | 0.1 | |
|
|
| `optim` | `adamw_torch` | |
|
|
| `weight_decay` | 0.01 | |
|
|
| `max_seq_length` | 8192 | |
|
|
| `seed` | 3407 | |
|
|
|
|
|
### Training Results |
|
|
The training loss curve showed a steady decrease, indicating successful learning. |
|
|
|
|
|
 |
|
|
|
|
|
|
|
|
## Intended Use & Limitations |
|
|
This model is intended for research and experimentation in multi-step reasoning, complex problem-solving, and chain-of-thought generation. |
|
|
|
|
|
* **Limitations:** Like all language models, it is prone to hallucination and may produce factually incorrect or biased information. |
|
|
* **Out-of-Scope:** This model is not designed for production use without extensive testing and safety evaluations. It should not be used for critical applications where incorrect information could cause harm. |
