davidfred/Qwen2.5-0.5B-Instruct-Q8_0.gguf

Model Card: Multilingual Qwen2.5-0.5B-Instruct-Q8_0 Model Details Name: Qwen2.5-0.5B-Instruct-Q8_0-Multilingual Base Model: Qwen2.5-0.5B-Instruct Model Type: Instruction-tuned Language Model Size: 500MB (Quantized) Supported Languages: English, Hebrew, French Format: GGUF (Compatible with llama.cpp) Model Description This is a quantized and fine-tuned version of the Qwen2.5-0.5B-Instruct model, specifically optimized for multilingual capabilities in English, Hebrew, and French. The model represents a significant advancement in compact, efficient language models while maintaining strong performance across multiple languages. Intended Use Multilingual text generation and understanding Cross-lingual question answering Translation assistance between supported languages General instruction following in three languages How to Download and Use Download the Model: bash

huggingface-cli download / qwen2.5-0.5b-instruct-q8_0.gguf --local-dir . [[3]] Basic Usage with llama.cpp: bash

./main -m qwen2.5-0.5b-instruct-q8_0.gguf -n 512 --temp 0.7 Training Details Base Model: Qwen2.5-0.5B-Instruct Fine-tuning Data: Multilingual dataset comprising: English text corpus Hebrew text corpus French text corpus Quantization: Q8_0 quantization for optimal balance between model size and performance Performance and Limitations Strengths: Efficient 500MB size making it suitable for local deployment Balanced performance across English, Hebrew, and French Optimized for instruction-following tasksLimitations**: May show reduced performance compared to larger models Limited context window Performance may vary across languages May struggle with complex technical content Ethical Considerations The model should be used in compliance with local regulations and ethical guidelines Users should be aware of potential biases in multilingual outputs Verify critical outputs, especially for sensitive applications Example Usage python

Example code for model inference from transformers import AutoModelForCausalLM, AutoTokenizer

Load the model model = AutoModelForCausalLM.from_pretrained("path_to_model") tokenizer = AutoTokenizer.from_pretrained("path_to_model")

Multilingual example prompts = { "English": "Translate 'Hello' to French:", "Hebrew": "תרגם 'שלום' לצרפתית:", "French": "Traduisez 'Bonjour' en hébreu:" } Citation and License Based on Qwen2.5 developed by the Qwen team at Alibaba Cloud Please refer to the original Qwen2.5 license for usage terms and conditions