Hugging Face's logo

hugging-quants
/
Meta-Llama-3.1-405B-Instruct-GPTQ-INT4

Text Generation
Transformers
Safetensors
llama
llama-3.1
meta
autogptq
conversational
text-generation-inference
Inference Endpoints
4-bit precision
gptq
Model card Files Community
18

Can you provide one model using `group_size=1024` to make the model smaller?

#15
by shuyuej - opened
Discussion
shuyuej

Can you provide one model using group_size=1024 to make the model smaller?

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment