starchat-alpha-GGML
This is GGML format quantised 4bit, 5bit and 8bit models of StarChat Alpha. This repo is the result of quantising to 4bit, 5bit and 8bit GGML for CPU inference using ggml.
Original model card
StarChat is a series of language models that are fine-tuned from StarCoder to act as helpful coding assistants. StarChat Alpha is the first of these models, and as an alpha release is only intended for educational or research purpopses. In particular, the model has not been aligned to human preferences with techniques like RLHF, so may generate problematic content (especially when prompted to do so).
Model Details
Model Description
- Model type: A 16B parameter GPT-like model fine-tuned on a blend of the
oasst1anddatabricks-dolly-15kdatasets. - Language(s) (NLP): English
- License: BigCode Open RAIL-M v1
- Finetuned from model: bigcode/starcoderbase
Model Sources [optional]
- Repository: https://github.com/bigcode-project/starcoder
- Demo: https://huggingface.co/spaces/HuggingFaceH4/starchat-playground
Uses
StarChat Alpha is intended for educational and/or research purposes and in that respect can be used to probe the programming capabilities of open-source language models.
Bias, Risks, and Limitations
StarChat Alpha has not been aligned to human preferences with techniques like RLHF or deployed with in-the-loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). Models trained primarily on code data will also have a more skewed demographic bias commensurate with the demographics of the GitHub community, for more on this see the StarCoder dataset which is derived from The Stack.
Since the base model was pretrained on a large corpus of code, it may produce code snippets that are syntactically valid but semantically incorrect. For example, it may produce code that does not compile or that produces incorrect results. It may also produce code that is vulnerable to security exploits. We have observed the model also has a tendency to produce false URLs which should be carefully inspected before clicking.
StarChat Alpha was fine-tuned from the base model StarCoder Base, please refer to its model card's Limitations Section for relevant information. In particular, the model was evaluated on some categories of gender biases, propensity for toxicity, and risk of suggesting code completions with known security flaws; these evaluations are reported in its technical report.