---
datasets:
- ReDiX/italian-filtered-corpus
language:
- it
- en
base_model:
- Qwen/Qwen3-0.6B-Base
library_name: transformers
license: cc
---

# Qwen3 0.6B Base - Ita 🇮🇹
This model is a further-pretrained version of [Qwen3-0.6B-Base](https://huggingface.co/Qwen/Qwen3-0.6B-Base) 🚀, specifically trained on 2 billion Italian tokens. The training data includes educational content 📚 carefully filtered from multilingual pre-training datasets. This ensures the model has a strong understanding of the Italian language and its nuances. It also boasts an extended tokenizer ✍️ optimized for Italian.

⚠️ Important Note: This is an experimental model. It may generate content that is dangerous or includes personal information. Please use with caution.

## Base Model (Not Instruct) 🤖
This is not an instruct model, meaning it doesn't follow a specific chat template. Instead, it's designed to be fine-tuned for your specific use case 🎯 with the Italian language.

## Evaluation Results 📊
Here's a breakdown of the model's performance on various tasks:




|   Tasks    |Version|Filter|n-shot| Metric |   |Value |   |Stderr|
|------------|------:|------|-----:|--------|---|-----:|---|-----:|
|arc_it      |      2|none  |     0|acc     |↑  |0.2566|±  |0.0128|
|            |       |none  |     0|acc_norm|↑  |0.2840|±  |0.0132|
|hellaswag_it|      1|none  |     0|acc     |↑  |0.3363|±  |0.0049|
|            |       |none  |     0|acc_norm|↑  |0.3994|±  |0.0051|
|m_mmlu_it   |      0|none  |     5|acc     |↑  |0.2699|±  |0.0039|

## How to use this model

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "ReDiX/Qwen-0.6B-Base-ITA"

# load the tokenizer and the model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
).eval()

text = "La principale causa del raffreddore"
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=128
)
output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() 


content = tokenizer.decode(output_ids[0:], skip_special_tokens=True).strip("\n")


print("content:", content)

```