Hugging Face's logo

medmekk
/
Llama-3.2-1B-BNB-int8

PyTorch
llama
8-bit precision
bitsandbytes
Model card Files
xet
Community

medmekk/Llama-3.2-1B-BNB-int8 (Quantized)

Description

This model is a quantized version of the original model medmekk/Llama-3.2-1B-BNB-int8. It has been quantized using int8 quantization with bitsandbytes.

Quantization Details

  • Quantization Type: int8
  • Threshold: 6.0
  • bnb_4bit_quant_type: None
  • bnb_4bit_use_double_quant: None

Usage

You can use this model in your applications by loading it directly from the Hugging Face Hub:

from transformers import AutoModel

model = AutoModel.from_pretrained("medmekk/Llama-3.2-1B-BNB-int8")
Downloads last month
3
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for medmekk/Llama-3.2-1B-BNB-int8

Unable to build the model tree, the base model loops to the model itself. Learn more.