File size: 2,300 Bytes
4a09280 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 |
---
license: apache-2.0
language:
- en
base_model:
- yasserrmd/MedScholar-1.5B
pipeline_tag: text-generation
tags:
- Med-scholar
- medical
- text-generation-inference
- quantized
---
# Quantized MedScholar-1.5B
This repository provides quantized GGUF versions of the yasserrmd/MedScholar-1.5B. These 4-bit and 5-bit quantized variants retain the original model’s strengths in multimodal medical reasoning, while reducing memory and compute requirements—ideal for efficient inference on resource-constrained devices.
## Model Overview
- **Original Model**: yasserrmd/MedScholar-1.5B
- **Quantized Versions**:
- Q4_K_M (4-bit quantization)
- Q5_K_M (5-bit quantization)
- **Architecture**: Decoder-only transformer
- **Base Model**: Qwen2.5-1.5B-Instruct-unsloth-bnb-4bit
- **Training Framework**: Unsloth + QLoRA
- **Dataset**: MIRIAD-4.4M (1M samples) [ODC-By 1.0]
- **License**: Apache-2.0 (inherits from base model); dataset is ODC-By 1.0
- **Language**: English
## Quantization Details
### Q4_K_M Version
- Approx. ~68% size reduction
- Lower memory footprint (~940 MB)
- Best suited for deployment on edge devices or low-resource GPUs
- Slight performance degradation in complex reasoning scenarios
### Q5_K_M Version
- Approx. ~64% size reduction
- Higher fidelity (~1.04 GB)
- Better performance retention, recommended when quality is a priority
### Usage
Below, there are some code snippets on how to get quickly started with running the model.
**llama.cpp (text-only)**
```sh
./llama-cli -hf SandLogicTechnologies/MedScholar-1.5B-GGUF -p "What are the symptoms of diabetes"
```
### Note
⚠️ This model is for research, educational, and exploration purposes only. It is not a medical device and must not be used to provide clinical advice, diagnosis, or treatment.
---
## Acknowledgments
- These quantized models are based on the original work by the [https://huggingface.co/yasserrmd/MedScholar-1.5B].
- MIRIAD Dataset by Zheng et al. (2025) – [https://huggingface.co/datasets/miriad/miriad-4.4M].
- Qwen2.5 by Alibaba - [https://huggingface.co/Qwen].
- Training infrastructure: [https://github.com/unslothai/unsloth].
---
## Contact
For any inquiries or support, please contact us at [email protected] or visit our [Website](https://www.sandlogic.com/).
|