For inference. CPU is enough for both quantization and inference.
ONEKQ AI
company
AI & ML interests
Benchmark, Code Generation, LLM
Organization Card
Edit this README.md
markdown file to author your organization card.
models
5
onekq-ai/starcoder2-3b-instruct-v0.1
Text Generation
•
Updated
•
34
onekq-ai/DeepSeek-Coder-V2-Lite-Base-bnb-4bit
Text Generation
•
Updated
•
120
onekq-ai/starcoder2-3b-bnb-4bit
Text Generation
•
Updated
•
58
onekq-ai/starcoder2-7b-bnb-4bit
Text Generation
•
Updated
•
20
onekq-ai/starcoder2-15b-bnb-4bit
Text Generation
•
Updated
•
43
•
1