QKV768's picture
Calibrated FP8 KV-cache scales (mla_kv_cache_scales); card: family, FP8 KV, MTP caveat. Needs qwen-mla-vllm >= 0.5.0
572038a verified