inference-optimization/gemma-4-1B-0.8B-tiny
1B • Updated • 1.49k • 2
Tiny Models
Tiny models used for testing
-
-
inference-optimization/DSV4-tiny-empty
3B • Updated • 4.49k • 1 -
inference-optimization/Qwen3-1.6B-A0.9B
Text Generation • 2B • Updated • 13.4k • 3 -
inference-optimization/Llama-3.2-0.5B-Instruct
Text Generation • 0.5B • Updated • 18.5k • 1 -
inference-optimization/GLM-5.2-0.8B-A0.8B
Text Generation • 0.8B • Updated • 4.58k • 4 -
inference-optimization/Phi-3.5-MoE-0.8B-A0.2B
Text Generation • 0.8B • Updated • 964 • 1 -
inference-optimization/Llama-4-Scout-1.7B-0.4B-Instruct
Image-Text-to-Text • 2B • Updated • 934 -
inference-optimization/gpt-oss-2.5B-A1.3B
3B • Updated • 900 -
inference-optimization/Qwen3-VL-1.0B-A0.4B-Instruct
Image-Text-to-Text • 1B • Updated • 946 • 1 -
inference-optimization/Inkling-0.6B-A0.6B
Image-Text-to-Text • 0.6B • Updated • 955 • 1 -
inference-optimization/DeepSeek-V4-Pro-0.5B-A0.37B
Text Generation • 0.5B • Updated • 932 -
inference-optimization/Kimi-K3-0.40B
Feature Extraction • 0.4B • Updated • 830 • 83 -
inference-optimization/Kimi-K3-0.40B-MXFP4
0.4B • Updated • 2.53k • 7 -
inference-optimization/Kimi-K3-0.40B-NVFP4
0.4B • Updated • 110 • 5 -
inference-optimization/gemma-4-unified-0.8B-tiny
0.8B • Updated • 882 -
inference-optimization/granite-vision-4.1-0.2B-tiny
0.2B • Updated • 877 -
inference-optimization/Qwen3-VL-Reranker-0.1B-tiny
98.7M • Updated • 900 -
inference-optimization/Qwen3.8-1.0B-A0.6B
Text Generation • 1.0B • Updated • 4.16k • 6 -
inference-optimization/GLM-5.3-Flash-0.1B-A0.1B
Image-Text-to-Text • 84.4M • Updated • 2.77k • 3 -
inference-optimization/Qwen3.8-Flash-Next-0.2B-A0.2B
Image-Text-to-Text • 0.2B • Updated • 1.23k • 2 -
inference-optimization/Nemotron-3.5-Lightning-1.4B-A0.1B-MTP
Text Generation • 1B • Updated • 206 • 1 -
inference-optimization/Qwen3.6-8B-A1.6B
Image-Text-to-Text • 8B • Updated • 916 -
inference-optimization/NemotronH-0.3B-A0.3B
Text Generation • 0.3B • Updated • 3.93k • 1 -
inference-optimization/GLM-5.3-0.6B-A0.4B
Text Generation • 0.6B • Updated • 1.04k • 1
inference-optimization/GLM-4.5-Air-0.72B-MTP
Text Generation • 0.7B • Updated • 11Note PR #3225. BF16 source; two-GPU quantization and vLLM MTP execution passed.
inference-optimization/GLM-4.5-Air-0.72B-MTP-FP8-Dynamic
Text Generation • 0.7B • Updated • 10Note PR #3225. FP8_DYNAMIC; two-GPU quantization and vLLM MTP execution passed.
inference-optimization/GLM-4.7-Flash-0.82B-MTP
Text Generation • 0.8B • Updated • 10Note PR #3225. BF16 source; two-GPU quantization and vLLM MTP execution passed.
inference-optimization/GLM-4.7-Flash-0.82B-MTP-FP8-Dynamic
Text Generation • 0.8B • Updated • 9Note PR #3225. FP8_DYNAMIC; two-GPU quantization and vLLM MTP execution passed.
inference-optimization/DeepSeek-V3-0.86B-MTP
Text Generation • 0.9B • Updated • 15Note PR #3225. BF16 source; two-GPU quantization and vLLM MTP execution passed.
inference-optimization/DeepSeek-V3-0.86B-MTP-FP8-Dynamic
Text Generation • 0.9B • Updated • 14Note PR #3225. FP8_DYNAMIC; two-GPU quantization and vLLM MTP execution passed.
inference-optimization/GLM-5-0.88B-MTP
Text Generation • 0.9B • Updated • 9Note PR #3225. BF16 source; backbone validated, MTP execution unverified.
inference-optimization/GLM-5-0.88B-MTP-FP8-Dynamic-MFPTQ
Text Generation • 0.9B • Updated • 10Note PR #3225. MFPTQ FP8_DYNAMIC; backbone validated, MTP execution unverified.
inference-optimization/Inkling-0.92B-MTP
Image-Text-to-Text • 0.9B • Updated • 14Note PR #3225. BF16 multimodal fixture; text backbone validated. Two-rank pipeline and MTP serving are not validated.