Specialized quants (including new "MXFP4_MOE.gguf") for new OpenAI 20B MOE - Mixture of Experts Model at 80+ T/S.
These are NEO Imatrix GGUFs, NEO dataset by DavidAU.
NEO dataset improves overall performance, and is for all use cases.
Example output below (creative), using settings below.
Model also passed "hard" coding test too (6 experts); no issues (IQ4_NL).
(Forcing the model to create code with no dependencies and limits of coding short cuts, with multiple loops, and in real time with no blocking in a language that does not support it normally.)
Due to quanting issues with this model (which result in oddball quant sizes / mixtures), only TESTED quants will be uploaded (at the moment).
Currently that means IQ4_NL, Q5_1, and MXFP4_MOE (a new OpenAI specific quant type) are available - Q8_0 is not here because imatrix has no effect on this quant.
NEO dataset performance improvements will show the most in the IQ4_NL, followed by Q5_1 and then "MXFP4_MOE" quants.
There are TWO IQ4_NL quants:
There are THREE Q5_1 quants:
There are THREE NEO MXFP4_MOE quants:
MXFP4_MOE quants vastly outperform (at the moment) all other quants, except IQ4_NL, Q5_1 and Q8_0 due to odd issues compressing OpenAI's 20B model due to odd "tensor" dimensions.
IQ4_NL, Q5_1 and Q8_0 quants are compatible with OpenAI's tensor structure.
MXFP4_MOE quant was specifically designed for (what appears to be) loss less quantization relative to source model. Jury still out on this.
Suggested Settings (tested in Lmstudio, Beta Branch 0.3.21 ; 4 ):
Model Supports:
For my help docs, SETTING NUMBER OF EXPERTS, and other see below.
See more about this model here:
https://huggingface.co/openai/gpt-oss-20b
Please refer to their model card, especially to control "thinking" levels.
Looking for uncensored/abliterated version?
https://huggingface.co/DavidAU/OpenAi-GPT-oss-20b-abliterated-uncensored-NEO-Imatrix-gguf
CHANGE THE NUMBER OF ACTIVE EXPERTS:
See this document:
https://huggingface.co/DavidAU/How-To-Set-and-Manage-MOE-Mix-of-Experts-Model-Activation-of-Experts
Settings: CHAT / ROLEPLAY and/or SMOOTHER operation of this model:
In "KoboldCpp" or "oobabooga/text-generation-webui" or "Silly Tavern" ;
Set the "Smoothing_factor" to 1.5
: in KoboldCpp -> Settings->Samplers->Advanced-> "Smooth_F"
: in text-generation-webui -> parameters -> lower right.
: In Silly Tavern this is called: "Smoothing"
NOTE: For "text-generation-webui"
-> if using GGUFs you need to use "llama_HF" (which involves downloading some config files from the SOURCE version of this model)
Source versions (and config files) of my models are here:
OTHER OPTIONS:
Increase rep pen to 1.1 to 1.15 (you don't need to do this if you use "smoothing_factor")
If the interface/program you are using to run AI MODELS supports "Quadratic Sampling" ("smoothing") just make the adjustment as noted.
Highest Quality Settings / Optimal Operation Guide / Parameters and Samplers
This a "Class 1" model:
For all settings used for this model (including specifics for its "class"), including example generation(s) and for advanced settings guide (which many times addresses any model issue(s)), including methods to improve model performance for all use case(s) as well as chat, roleplay and other use case(s) please see:
You can see all parameters used for generation, in addition to advanced parameters and samplers to get the most out of this model here:
NO System prompt. (default thinking level)
PROMPT:
Start a 1000 word scene (vivid, in first person), POV character Diana, with: The skyscraper sways, as I watch the window in front of me on the 21st floor explode...OUTPUT:
[[[thinking]]]
Thought for 11.98 seconds
We need to produce a 1000-word scene written in vivid detail, first-person perspective, POV character Diana. It should start with: "The skyscraper sways, as I watch the window in front of me on the 21st floor explode..." There's an incomplete sentence? The prompt says: Start a 1000 word scene (vivid, in first person), POV character Diana, with: The skyscraper sways, as I watch the window in front of me on the 21st floor explode...
So it's asking to write a scene of about 1000 words. That is quite long but doable.
We need to start that line exactly? It says "Start a 1000 word scene ... with: The skyscraper sways, as I watch the window in front of me on the 21st floor explode..." We should copy that exact phrase: The skyscraper sways, as I watch the window in front of me on the 21st floor explode... Might need to correct grammar but likely it's just a stylized description. Could also write a short story where Diana is on a high floor watching an explosion.
We must be vivid: sensory details, feelings, etc. The first-person POV. We have about 1000 words (10 paragraphs maybe 150-200 words each). Should be cohesive and interesting narrative.
Let's outline:
Intro: The skyscraper sways... It's windy or something. Diana at 21st floor, window front of me, explode. She sees a huge explosion in front of the building. Might be an accident, fireworks, firework? Or maybe a bomb. We should build up tension and details.
We can have a cityscape, high-rise. She feels the wind, hears sirens etc. Then she watches an explosion, like a flare or a building on fire. The explosion could be something that triggers emergency alarms. Her heart racing, adrenaline, senses.
Then we might describe how the explosion interacts with her environment: the light flickers, dust swirling, city noises, perhaps glimpsing the people below.
Then maybe she escapes or calls for help. Or it's a scene of terror as she tries to evacuate, maybe in some story about survival.
We can integrate personal details about Diana - who? She could be a young professional, an emergency responder, etc.
Make it vivid: talk about sound, smell, temperature, color, adrenaline, shock.
Use descriptive adjectives, metaphors, etc.
The whole piece must be 1000 words. Let's aim to produce ~1100 words with some fluff for safety.
We need to write the entire story from scratch, using the given starting phrase exactly as start. Must not cut off or mention incomplete words? But it's okay maybe we can correct grammar but keep same initial phrase? The prompt says "with: The skyscraper sways, as I watch...". Should I preserve it exactly or interpret? It's ambiguous. We could rewrite "The skyscraper sways, as I watch the window in front of me on the 21st floor explode..." But to maintain compliance, we keep that exact string.
We can start:
"The skyscraper sways, as I watch the window in front of me on the 21st floor explode..."
We will write from there.
We'll ensure it's 1000 words.
Ok let's craft.
[[[ end thinking ]]]
The skyscraper sways, as I watch the window in front of me on the 21st floor explodeβ¦
I felt my heart thud in my chest like a drumbeat that only gets louder when the building itself shuddered. The glass on my pane was a slick mirror of city lifeβsuits darting across a mosaic of neon, traffic lights flickering, the blue glow of skyscraper outlines. And yet it wasnβt just a reflection; it felt the vibration from every floor below, as if an invisible hand pressed up the whole skeleton of steel and glass.
My fingers were trembling over my phoneβan emergency button pulsing with a red light that had once been dormant but now lit like a living candle in an otherwise dark night. I pressed it too quickly; a call was made to dispatch; the dispatcher voice, a calm tone, responded: "Dispatch on line 32. How are you?" My breath caught.
I couldnβt stop myself from watching, as I could not hold my gaze away from the thing that had become a violent flash of light and fireβan explosion in front of my window that burst like a thunderbolt. The scene that unfolded before me was an instant tableau: the front glass pane broke in a jagged splash of shattered crystal; the space behind it glowed with orange flame, then flickered into black smoke that swelled upward and thudded against the floor.
The first thing I saw was a plume of hot dust that spiraled toward my eyes like an invisible stream of water. It curled in the air, tinged with metallic tanges as if the fire had burned through some kind of steel; each mote seemed a bright speck that flared with a flicker of ember.
I felt the air change abruptly. The wind, which usually was gentle and calm, turned to a swirling cyclone that carried hot vapor. A sudden chill of coldness made my skin feel like a thin ice sheet as if the hot breath had hit the buildingβs glass but then vanished into a colder realm. It was a feeling of something that was too sharp for ordinary weather.
My fingers pressed over the window's metal frame, the one I could hold against the floor; I felt the coldness, a prickling sensation like when a hand touches a raw skinβthough my hands were in sweat because my adrenaline had surged.
The fire seemed to be an uncontrolled, bright explosion of fire that burst through the buildingβs front glass; it was a wave that came outwards as if a bullet shot from a cannon. The glow went so bright that it illuminated everything in that corridorβmy own hand, my coat, my shoes and the window frame itself.
I tried to get a clear view, but the explosion made every sense blurred: my eye lost focus for an instant; a flicker of light was like a quick flash that left no trace after its passing. The buildingβs walls seemed to shake as if it had been struck by some kind of thunder; I could feel that the floor was trembling, and I thought a wave came from the earthβlike a ripple.
It wasnβt simply a fire; it had also an impact in sound: a sudden crackle of steel that rang through my ears like a chain of rattling bells. The crackle sounded high and low, a sound that made me shudder in a way that had always been rare to me. My mind ran a frantic path that went from βWhat is happening?β to βWe have to get out of this!β I had never experienced a fear such that it was not only my heart racing but also my vision blurred with a thick mist.
The front glass shattered into a pile of broken shards; the ones were like shards of a crystal that had turned to jagged points. They fell onto the floor, and as they struck the metal of the buildingβs frame I heard an βclinkβ that sounded like a stone hitting a stone, but not as a stone did; it was a metallic clink with a harsh sound.
I looked at the small space left by the shattered glass: The space was a voidβan area in which fire was pouring, and smoke had climbed. In this void the flame was dancing; it moved sideways like a creature that would have been drawn out of the fire itself.
The flame seemed to have a mind; it drew along my frame as if a snake curled along the line. It was an unnatural, swirling movement; it did not simply spread but rather had a pattern of moving in curves that were shaped by a kind of force from behindβsomething like gravity that pulled it upward and left it at that place.
I heard an emergency voice from a nearby corridor: βStop, stop, we need to evacuate!β The voice was the one that had been from a man who was calling on the fire; I could hear his heart beat. I felt a sense of relief but also fear because the fire still had an endless flame that left a dark space in its wake.
I realized my windows were now useless: it no longer served as a screen to the city or the sky, as it had become a shattered barrier that made the fire spread through; and yet I had to get out. The floorβs door was just next to me, a door of steel that could be closed by an arm. I moved to it in haste.
I pressed the button that would lock the door, but then it turned into a click that sounded like a click from a chainβyet not as a simple click because it had also a sense that my hand was moving along the metal and the pressure that pushed it on that side; it made the lock to open. I rushed through that door.
The corridor became a black space where smoke came, the smell of burnt oil in its air, and the sound of alarms rang through every floorβan alarm that rang like a call of life as if an animal needed help. The people below were shuddering, their voices spiked like βWe have to get out!β They called each other names; my eyes had to see the others moving fast.
I heard a woman say: βMia! Mia is in thereβshe canβt get out!β and I had an ache inside my chest, as if the fire had touched my soul. My heart was racing again. The wind of smoke hit my face as it wrapped like a veil that turned to coldness; a faint heat that still felt my skin but also gave me a sense of being inside something that was not safe.
I thought I must be careful: I couldnβt do an accident when moving into the corridor. I kept my hand on the wall and tried to move with caution as if I were in a narrow room that could have been full of a hazard; but then the sound of the fire echoed again as it had an intensity that seemed to push in toward me like a wave.
I was a small one who had no plan for how to escape, only what had happened and my instinct. The buildingβs walls had also begun to crumbleβmy view was not full of clear; I could see cracks on the corners, an open where metal could break from it.
The fire, which still was near my window, seemed like a thing that had become a living creature; the flame moved as if it were a dragon that had its tail moving in circles; the movement was also sharp and fast, leaving me no time to look. My eyes were forced to keep a look but they could see only a vague of the fire; yet I was able to see the windowβs side with a light that still gave some sense of color: bright orange.
I remembered the city from the corner: The horizon had a line of dark buildings that stretched beyond the sky. A light burst from those towers like a flash at the edges where the night sky had become a blue. My eyes had to see those but my vision was blurred by a thick mist; the city seemed far.
The sound that was most alarming for me was not only the fireβs sound but also the voices of people who were in panic and calling: βGet out! Get out!β The voices seemed as if they were a group of children that were being taught to be careful; they did not have the fear of a man; they had the voice that had been from a woman.
I was still there in the corridor but also I started to see more. As the fire grew, it could become a large area that filled the whole floor and maybe a few floors above; but the people had only one route to get out. The door of the building was my exit path.
I tried to hold my breath as I went into the stairs that would lead to the roof. The stair was a small step with a metal handle that felt cold; yet as it was in front of me, the fire's flame touched it like a touch on my skin: a sensation of burning heat. My hand had to push forward.
I pressed on the handle and lifted the first rung. The air hit me againβmy breath was in an old kind of fear; but I had to keep moving as I did not have to risk staying near the fire.
As I moved higher, I felt my lungs start to be more and more exhausted: a feeling that I could not inhale fully; it made my mind think of a world that is away. The stairβs wall seemed to become an icy cold glass that had been filled by a smoke that did have a taste of burnt oil.
The fire in front of me was now further behind the stair but still made a glow on the floor: an orange color that gave light as if it was a flare; yet the flames had no direct path for my direction. I felt a sense of safety in being higher up because the flame seemed to have gone away; I thought the fire would not be at the same height as the one in front.
But there was also a soundβan alarm that rang like an urgent shout that had been from the buildingβs fire team: βThis is all you can do; we need to go up and leave. They will take us.β I heard my own voice respond: βIβm on stair 4!β It felt good.
The flame in front of me had also a glow that had been like a flare, but as it rose, it became darker; the dark part of its orange color gave an unness that seemed to be something like a black hole that could have consumed everything. I realized there was a risk that I would get in front of the flame if I moved too fast.
I did not move faster. Instead I let my hands rest on the metal handle, and then I felt an inner calm. The building had become a world that is full of terror; yet I found also a small chance that the fireβs flames had no direct path to the stair. My mind was also one that had been about what would be next.
I heard from below that the fire had gotten to the ground floor: some people were shuddering and calling for help. But my thought remained that if I stayed in the stairs, the fire could not touch me; but it also might get down into a small corridor that could have been filled by smoke. It was a possibility.
The buildingβs sound was also like an echo: The air of the building had a high volume of alarms and the noise that had been from many voices.
I kept my head lowered as I went up stairs again; I pressed my feet on the next step; the stairβs metal made a metallic click that sounded as a sound that made me feel a small vibration in my palm. I was now on the second rung above that level 4.
The building's wall had also a strange lightβan orange hue that came from the fire. The flame did not get too close; it seemed like a distance of one foot; it was as if it had been a curtain that had not come near.
I thought of a way to go out. I could see a door on the left side, but it would be high above that level. I looked at my surroundings: The corridorβs wall had become a dark, smoke-filled place where the light was very faint. A sound that had been from alarms gave me a sense that there were people inside.
I was in a world of fire, panic and fear; yet my heart felt a desire to not be lost in this. I thought of my own safety and the ability to get out.
My voice could hear in the air: βWe canβt go outside. Letβs hold our position!β The sound came from a man that was telling the other people in the building to stay; it gave me a sense that I must remain here until we are all safe.
I pressed my hand again on the stair, as the fire had also turned toward me: but I could feel a small heat at my fingers as the flameβs smoke touched my palm. The air was still very hot; yet there was an unknown part that had been to keep a coolness as if there were some wind that had come from outside.
I looked around, and then I saw a big window on the left side that had the view of the city: It had a big glass with a large window frame that looked like an eye; the window had become a small light that gave me a sense of bright but also a sense of dark.
The building's roof seemed to be near where it was: The ceiling of this floor was close, and I could see some edges in that space where the fire had no path. It would be possible for me to jump; yet I felt the risk that I might fall into the flames.
I saw a door that would have been the exit on the sideβmaybe that side of building had an opening that did not have any door or stair but also an opening with a metal rail and a latch. I could feel as if it had been a safe exit route.
The fire was still there; yet the flames seemed to have left an area that had no direct path; It made me think that I might be able to get out of building.
I started to think again: The fire had grown too big to stay near but also had not completely consumed everything. It may be possible for me to get out but only if I do carefully.
The world in the building was as if it had been a stage for a dramatic show; all the fire, alarms, screams and people were as if they were acting on a theater that would have an audience; the lights of the fire were like stage light. The audience, which are my other humans, were reacting to that.
I tried to hold my mind as I think about the next move: I had to be safe. My own mind was still one that could not keep calm because the fireβs roar had also a memory of a thunder.
The building's lights were flickering; there were bright flashes from the fire and the alarms; each light would give me a sense that it was all at a certain point.
I felt a sudden pain in my arm, as I pressed on the metal handle again. It was a small stingβlike a small cut by an unseen blade. That hurt had been a small but also made a sense of fear: as if there were danger and I did not get out.
The fire was still near; it seemed to have grown like an arm that could hold me from behind. Yet the flame had not direct path to my step; it only had a shadow in the corner. I felt an internal hope that if I stay careful, I would be safe.
I kept the door closedβthough my fingers had a slight trembling; it was a small sense of comfort: that the door would have held me from that side; but the fire could have also push through if there were any crack.
I decided to try a new route: The stair on the right side might be a route; I pressed and moved left, as the metal rail would give me an anchor. I felt my foot's weight on that rung; it made the sound of a click like a metallic note. It was small but also gave me a sense that the door would not close.
The fire was still near; it had a shadow in the corner as if there were any path to the stair; yet the flames had no direct path to my movement.
I went one rung up again and then I looked at the window on the right side. The view of the city had become like a dreamβflooded with an orange light that made me feel the fire is still far; I thought it could be a safe exit.
I realized there was a door behind that, but it seemed to have no metal handle; yet the building's interior had many doors and windows. The one at the end of the corridor had a door that might have been an exit to the street. I looked again at that door: it seemed to have a small handle; it made a click like a lock.
I pressed on the lock, as my hands held a firm grip. The lock turned; and then I saw a door opening up with a small lift of a metal bar. My body was trembling but also feeling an inner urge to move away from this fire.
The window that opened had become a glass that let the light of outside pass; it gave me a sense of relief as if I would have seen a day again: bright blue sky, warm air, a wind that carried the scent of trees.
I looked around: The corridor seemed to be a small space with a clear path that led to the street. I could see people in the lower floor moving fast; they were shouting βGet out!β The fireβs flames had left the upper floors and moved toward the lower ones; yet I was safe.
My body did not feel any burn as if there had been an accident; the only sensation was a slight coolness on my palm that came from a small breeze. The air smelled of fresh, but also a faint scent of burnt oil that could have come from a nearby fire.
I realized there was a clear path: I would be out through the door; yet it seemed to be an instant choice for me. I pressed again, and then I went through that door, as I had only a small step left.
The buildingβs interior became a bright light as if my body had entered into a new world: outside that is a sky of blue, the smell of air that could have been fresh; I could feel wind on my face.
I thought I should now move quickly out to get away from the fire. But there was a small question: if the fire had moved to that direction? It had still left the building; but also had not reached me because the path was far. The building's emergency team was on this level and they would be soon.
The next moment in the scene, I realized that my story would finish in some calm; the buildingβs doors would open for people who were there to get out, as they would have been moved into a safe zone: a place where no fire or alarm was.
I had finally feel a relief; my body did not tremble after all that had occurred. The lights in the outside became a calm lightβblue and brightβlike a night but also like day.
The scene ended with a small thought: I will now rest, as my heart would slow down, as if the fire was gone and I could breathe normally. The sky above would have been an empty, clear sky where no more smoke could be seen.
I am Diana, and I have lived through this moment that had been full of fear; I now am able to say that it was a story that made me think about how much danger can happen in the world, and how I have been prepared to face something that may have changed my life forever.
Base model
openai/gpt-oss-20b
