The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
Paper
â˘
2402.17764
â˘
Published
â˘
624
MiJa
snapo
AI & ML interests
None yet
Recent Activity
new activity
1 day ago
deepseek-ai/DeepSeek-V3.1:This modelâs censorship is insane
liked
a model
1 day ago
deepseek-ai/DeepSeek-V3.1
reacted
to
sweatSmile's
post
with đ
13 days ago
Teaching a 7B Model to Be Just the Right Amount of Snark
Ever wondered if a language model could get sarcasm? I fine-tuned Mistral-7B using LoRA and 4-bit quantisationâon just ~720 hand-picked sarcastic promptâresponse pairs from Reddit, Twitter, and real-life conversations.
The challenge? Keeping it sarcastic but still helpful.
LoRA rank 16 to avoid overfitting
4-bit NF4 quantization to fit on limited GPU memory
10 carefully monitored epochs so it didnât turn into a full-time comedian
Result: a model that understands âOh great, another meetingâ exactly as you mean it.
Read the full journey, tech details, and lessons learned on my blog:
Fine-Tuning Mistral-7B for Sarcasm with LoRA and 4-Bit Quantisation
Try the model here on Hugging Face: sweatSmile/Mistral-7B-Instruct-v0.1-Sarcasm.
Organizations
None yet
upvoted
a
paper
over 1 year ago