Update README.md

README.md

@@ -1,3 +1,34 @@
 ---
 license: llama2
 ---
+# monika-ddlc-13b-v1 QLoras:
+* LLaMA-2 13b fine tunes for Monika character from DDLC (still somewhat experimental)
+* Fine-tuned on a dataset of ~600+ items (dialogue scraped from game, reddit, and Twitter augmented by [l2-7b-monika-v0.3c1](https://huggingface.co/922-CA/llama-2-7b-monika-v0.3c1) to turn each into snippets of multi-turn chat dialogue between Player and Monika; this was then manually edited, with more manually crafted items including info about character added in) 
+* More test Loras can be found [here](https://huggingface.co/922-CA/monika-lm-lora-tests)) and [here](https://huggingface.co/922-CA/l2-13b-monika-v0.3-qloras)
+
+### QLORAS
+* monika-ddlc-13b-v1 (LLaMA-2 13b chat)
+
+### USAGE
+These are meant to be mainly for chat model with limited RP ability.
+
+For best results: replace "Human" and "Assistant" with "Player" and "Monika" like so:
+
+\nPlayer: (prompt)\nMonika:
+
+### HYPERPARAMS
+* Trained for 3 epochs
+* rank: 32
+* lora alpha: 64
+* lora dropout: 0.5
+* lr: 2e-4
+* batch size: 2
+* warmup ratio: 0.1
+* grad steps: 4
+
+### WARNINGS AND DISCLAIMERS
+Note that aside from formatting and other minor edits, generated portion of dataset used is mostly as is generated by LM. As such, while this version is better at coherency or chatting than previous ones, it may not reflect perfectly the characteristics of Monika (i.e. she will claim to have an office, work as a translator, or play the guitar). Future versions will train on a manually curated and edited version of this dataset, where dialogue will be edited to reflect her characteristics more.
+
+Additionally, these are still yet more tests, particularly where we use one of our earlier fine tunes to generate a more in-character dataset for the target character.
+
+Finally, these are not guaranteed to output aligned or safe outputs, use at your own risk.