Fine-tuned Model: Merged-11268_gemma-2-2b-it-co-sft-qlora
This model is a fine-tuned version of google/gemma-2-2b-it trained on a dataset of message board conversations from 4chan's /co/ (Comics & Cartoons) board. The goal was to create a model that emulates the specific linguistic style, tone, and conversational patterns of that community.
Evaluation Results
This model was evaluated using the Open LLM Leaderboard's lm-evaluation-harness. The results show a specialization in conversational style at the cost of general knowledge and mathematical ability, which is the expected outcome for this type of fine-tuning.
| Metric | Score |
|---|---|
| Average Normalized Accuracy | 34.76% |
| Average General Accuracy | 17.62% |
| Instruction Following (IFEval) | 21.94% |
| Exact Match (GSM8K) | 1.81% |
- Downloads last month
- -
Model tree for AiAF/bf16_Merged-11268_gemma-2-2b-it-co-sft-qlora
Datasets used to train AiAF/bf16_Merged-11268_gemma-2-2b-it-co-sft-qlora
Evaluation results
- ARC (acc) on Open LLM Leaderboardself-reported0.176
- HellaSwag (acc_norm) on Open LLM Leaderboardself-reported0.348
- MMLU (acc) on Open LLM Leaderboardself-reported0.176
- TruthfulQA (mc2) on Open LLM Leaderboardself-reported0.348
- Winogrande (acc) on Open LLM Leaderboardself-reported0.176
- GSM8K (exact_match) on Open LLM Leaderboardself-reported0.018
- IFEval (inst_level_strict_acc) on Open LLM Leaderboardself-reported0.219