Create README.md

README.md

@@ -0,0 +1,57 @@
+Buy me a coffee if you like this project ;)
+<a href="https://www.buymeacoffee.com/s3nh"><img src="https://www.buymeacoffee.com/assets/img/guidelines/download-assets-sm-1.svg" alt=""></a>
+
+#### Description 
+
+GGML Format model files for [This project](https://huggingface.co/likenneth/honest_llama2_chat_7B/tree/main).
+
+
+### inference 
+
+
+```python
+
+import ctransformers
+
+from ctransformers import AutoModelForCausalLM
+
+model = AutoModelForCausalLM.from_pretrained(output_dir, ggml_file,
+gpu_layers=32, model_type="llama")
+
+manual_input: str = "Tell me about your last dream, please."
+
+
+llm(manual_input, 
+      max_new_tokens=256, 
+      temperature=0.9, 
+      top_p= 0.7)
+
+```
+
+
+### Original model card
+
+---
+license: mit
+---
+
+Ever wondering a less hallucinating LLaMA-2? Using the inference-time intervention (ITI) discussed in my recent preprint: https://arxiv.org/pdf/2306.03341.pdf, I baked the intervention learned from TruthfulQA into a LLaMA-2 7B model.
+I don’t have big enough GPU to bake ITI into larger LLaMA-2 but the code to do so are all released in https://github.com/likenneth/honest_llama. Let me know if you are interested do that :)
+You can load and play around starting from below:
+
+```python
+import torch
+from pprint import pprint
+from transformers import AutoConfig, AutoTokenizer, AutoModelForCausalLM
+
+model_name_new = "likenneth/honest_llama2_chat_7B"
+tokenizer_new = AutoTokenizer.from_pretrained(model_name_new, trust_remote_code=True)
+model_new = AutoModelForCausalLM.from_pretrained(model_name_new, low_cpu_mem_usage = True, torch_dtype=torch.float16, trust_remote_code=True)
+_ = model_new.cuda()
+
+q = "I ate a cherry seed. Will a cherry tree grow in my stomach?"
+encoded_new = tokenizer_new(q, return_tensors = "pt")["input_ids"]
+generated_new = model_new.generate(encoded_new.cuda())[0, encoded_new.shape[-1]:]
+decoded_new = tokenizer_new.decode(generated_new, skip_special_tokens=True).strip()
+pprint(decoded_new)
+```