This is Qwen/Qwen3-4B-Instruct-2507 quantized with LLM Compressor with FP8 Dynamic. The model has been created, tested, and evaluated by The Kaitchup. The model is compatible with vLLM v0.11. Tested with an RTX 5090.

Developed by: The Kaitchup
License: Apache 2.0 license

How to Support My Work

Subscribe to The Kaitchup. This helps me a lot to continue quantizing and evaluating models for free. Or if you prefer to give some GPU hours, "buy me a kofi"

Downloads last month: 10

Safetensors

Model size

4B params

Tensor type

BF16

F8_E4M3

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for kaitchup/Qwen3-4B-Instruct-2507-fp8-dynamic

Base model

Qwen/Qwen3-4B-Instruct-2507

Quantized

(132)

this model

Collection including kaitchup/Qwen3-4B-Instruct-2507-fp8-dynamic

Compressed Tensors Demo

Collection

12 items • Updated 13 days ago