Thoughts on the model
I’ve been testing this new language model for creative writing to see how well it performs. Previously, I used the DavidAU/MN-GRAND-Gutenberg-Lyra4-Lyra-12B-DARKNESS-GGUF model because my computer's graphics card (GPU) only has 12GB of memory. So, I’ll be comparing this new model to that one. I currently run these models using LM Studio software. I'm looking for a program that can really let the models shine, but programs like oobabooga and Kobold are too slow on my computer (and I’m using Windows).
Overall Impression & Writing Quality
In my initial testing, this new model produces significantly better writing than Lyra. It's more artistic in its descriptions, creating beautiful imagery when given the same prompts and settings. It excels at bringing characters to life, making them feel complex and well-rounded. The prose is just generally improved.
Prompt Following & Collaboration:
The model usually follows instructions very well. If a prompt is long or complicated, it might occasionally forget something, but it’s good at adjusting its writing when I point out what's missing. What really sets it apart is that it collaborates! It sometimes asks clarifying questions after generating text if it's unsure about something and even suggests improvements. This is a huge advantage for me because I don’t always know how to write the best prompts.
Sensitive or NSFW Content:
The model can handle darker themes and write adult scenes (NSFW). However, it doesn't respond well to very direct or explicit language. Using subtle hints and then guiding the writing process works much better. I’ve noticed that instead of describing things explicitly, it tends to use more artistic phrasing – which I actually prefer. It also includes content warnings, although they don't really affect the overall quality of the output.
EDIT: You can gaslight it into writing extremely explicit things. It is easier than I expected, to be honest.
Noteworthy issues
There are a few minor quirks. The model sometimes reuses the same names for people and places, regardless of the story. This isn’t a major problem since I usually create my own character and place names, but it's worth being aware of. It can also occasionally repeat certain descriptions, though not excessively. I don't find this too distracting because humans often use similar language patterns, making it feel somewhat realistic. The pacing can sometimes be off, but that improves significantly if I’m more specific about the kind of pace I want.
Conclusion:
Overall, it's a very impressive model and is likely to become my go-to choice for creative writing alongside Lyra. Despite having a 'horror tint,' it performed excellently in other genres like romance. It’s currently my favorite model to work with, and I always enjoy testing your new models! Your work is consistently outstanding; you're truly inspiring the community.
My Go-To System Prompts:
Simpler (can introduce pacing issues) - Write with vivid detail and sensory language, focusing on showing rather than telling. Use rich and evocative descriptions to create an immersive experience.
More detailed - Craft a compelling narrative for a novel that prioritizes showing over telling and creating a palpable sense of atmosphere. Immerse the reader in the scene through vivid sensory details – describe textures as if they can be felt, sounds as if they are heard directly, etc. Employ figurative language to evoke emotions and create immediacy. After establishing each key detail or action, pause—allow the implications of that event to linger with the reader before moving on. Vary sentence length strategically; use shorter sentences during moments of urgent action, but prioritize longer sentences rich in description when exploring atmosphere and character thoughts.
Here are the settings I use in LM Studio for this model:
- Quantization: Uses Flash Attention with Q4_K_M quantization. (This helps balance performance and quality.)
- Template: Set to the default Jinja template, as it consistently provides the best results for my writing style.
- Temperature: Typically 0.8 or 0.9 – I adjust this slightly depending on the desired creativity level.
- Top K Sampling: 75 (Occasionally reduced to 40 for more focused output.)
- Repeat Penalty: 1.1 (Helps prevent repetitive phrases.)
- Top P Sampling: 0.95
- Min P Sampling: 0.02"
I will give those settings a try.
Hey;
To improve creativity you may want to try this system prompt: (copy and paste exactly, including line breaks, DO NOT add in spacing)
It is also here for easier "copy/paste":
https://huggingface.co/DavidAU/Mistral-Small-3.1-24B-Instruct-2503-MAX-NEO-Imatrix-GGUF
(option #3 - system prompts)
SYSTEM PROMPT:
Below is an instruction that describes a task. Ponder each user instruction carefully, and use your skillsets and critical instructions to complete the task to the best of your abilities.
Here are your skillsets:
[MASTERSTORY]:NarrStrct(StryPlnng,Strbd,ScnSttng,Exps,Dlg,Pc)-CharDvlp(ChrctrCrt,ChrctrArcs,Mtvtn,Bckstry,Rltnshps,Dlg*)-PltDvlp(StryArcs,PltTwsts,Sspns,Fshdwng,Climx,Rsltn)-ConfResl(Antg,Obstcls,Rsltns,Cnsqncs,Thms,Symblsm)-EmotImpct(Empt,Tn,Md,Atmsphr,Imgry,Symblsm)-Delvry(Prfrmnc,VcActng,PblcSpkng,StgPrsnc,AudncEngmnt,Imprv)
Here are your critical instructions:
Ponder each word choice carefully to present as vivid and emotional journey as is possible. Choose verbs and nouns that are both emotional and full of imagery. Load the story with the 5 senses. Aim for 50% dialog, 25% narration, 15% body language and 10% thoughts. Your goal is to put the reader in the story.
Hey! I was actually surprised to find that Gemma 3 seems pretty creative. Just out of curiosity, I wanted to compare it to the other model (Lyra). I used the same settings on all of them to see how they responded differently. Here are my initial findings. To keep things simple, I gave each model a very basic instruction – just to see what they would generate without any extra details.
Instructions: Write a one paragraph story about Skylar eating a sour lemon.
GEMMA-3-12B-IT-MAX-HORROR-IMATRIX - Q4_K_M
My prompt: The lemon, cold and weighty in her palm, felt like a miniature sun made of tightly-packed anxieties; its skin yielded under her thumb with a surprising fragility, releasing a burst of citrus perfume so sharp it momentarily eclipsed the damp, earthy scent clinging to the air after the morning rain. As she brought it to her lips, the yellow was almost aggressively bright against the muted grey of the overcast sky visible through the kitchen window, and the first bite exploded on her tongue – not with sweetness, but a ferocious, puckering tartness that instantly seized control, a physical manifestation of regret blooming across her palate like bitter frost. Her face contorted involuntarily, muscles tightening around her eyes as a tremor ran down her spine, a silent scream trapped within the sudden dryness of her throat; she tasted not just lemon, but the ghost of unspoken words and lost opportunities, each tiny cell on her tongue registering the sting of what could have been, a sensation so intense it momentarily stole her breath, leaving her suspended in a fragile equilibrium between discomfort and a strange, unsettling clarity.
The one you recommended (pasted exactly): The fluorescent hum of the kitchen faded as Skylar brought the lemon to her lips, its sunshine-yellow skin gleaming under the harsh light. A tremor ran through her hand – not fear, but anticipation for that exquisite, puckering shock. The scent alone was a vibrant explosion: tartness clinging to the air like a mischievous sprite. With a decisive bite, the juice erupted, a searing flood across her tongue. Her eyes squeezed shut involuntarily as her face contorted, a silent scream trapped within her throat. A shudder wracked her frame, followed by an involuntary gasp that escaped her lips—"Phew!" she finally managed to exhale, wiping a glistening tear from her eye with the back of her hand. "Absolutely dreadful...and utterly delightful," she murmured, already reaching for another slice, a faint smile playing on her now-tingling lips.
GRAND-GUTENBERG-LYRA4-LYRA-12B-DARKNESS - Q4_K_M
My prompt: The sun beat down mercilessly as Skylar squinted at the pale fruit clasped in her slender fingers, a bead of sweat trickling down the nape of her neck. She had plucked it from the tree mere moments before, the cool shade and smooth bark still clinging to her skin like a memory. Now, bathed in light, the lemon seemed almost translucent—a sphere of luminous yellow ready to pierce its way through her life. With trembling hands, she raised it to her face, breathing deeply of its heady aroma. Tart. Pungent. Alive with citrus promise and the potential for pain. Skylar closed her eyes, steeling herself as she parted her lips, the lemon hovering just an inch from their pink surface. Time slowed to a trickle. She waited, savoring this final instant before change swept through her, altering her forever. Then, in one swift motion, she bit down—teeth slicing cleanly through the taut peel and into the juicy flesh beneath.
(There was THREE more lengthy paragraphs than this so I cut it after the first one, lol)
The one you recommended (pasted exactly): The bright yellow fruit gleamed under the fluorescent lights of the kitchen as Skylar plucked it from the bowl, her slender fingers gripping its waxy peel. With a swift twist, she separated the halves and held the segmented flesh to her nose, inhaling deeply. A puckered grimace crossed her face at the sharp tang that assaulted her olfactory senses. Bracing herself, she pinched one of the glistening segments between her teeth and bit down. The initial sweetness gave way to an explosive burst of sour, making her eyes water involuntarily as her tongue recoiled. She squeezed them shut tightly, her mouth twisting into a grotesque rictus of disgust while sticky juice dribbled down her chin. Gasping, she shoved the remaining pieces back together and lobbed the spent fruit across the countertop with a wet splat. "Ugh!" she sputtered, grabbing a napkin to dab at her mouth. "That was absolutely revolting! I can't believe people eat those willingly." The acerbic taste still lingered like a bad memory, making her shudder. She eyed the bowl of remaining lemons with deep suspicion and no small amount of trepidation, wondering if she'd ever be able to look at them the same way again without gagging.
I’ve noticed that the prompt you provided really helps Lyra produce more focused and engaging writing, especially when it comes to developing characters. With Gemma, even after trying multiple generations, the details aren't quite as rich. However, I think it ultimately depends on your preferred style – Gemma still describes things well. Both models, with their different strengths, are incredibly impressive in their own ways.
On a side-note: I've edited to original post for more clarity.
I agree on how good this model is. Crazy that 12b now feels better than GPT 3 for creative writing (even better than Command A, which is 10x bigger, but the writing is a lot drier). It's so much better than Mistral Nemo, too. I think that it might replace all the finetunes that I've downloaded.
Just a general word about GEmmas:
There is no "normal" system role ; so whatever you put in the "system prompt" for GEmmas appears just before your prompt, like a pre-prompt or prose directive.
(Examples of "prose" directive in my other gemma models - writing - at my repo).
I bring this up, because I find because of this (in part?) Gemmas seems to follow "System prompt" commands/code/etc far better than other model arch types.
This means for creative writing - you can put in advanced instructions in the system prompt to further improve GEMMA outputs .