Instructions to use DreamFast/Gemma4-e4b-abliterlitics with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DreamFast/Gemma4-e4b-abliterlitics with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DreamFast/Gemma4-e4b-abliterlitics")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("DreamFast/Gemma4-e4b-abliterlitics", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DreamFast/Gemma4-e4b-abliterlitics with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DreamFast/Gemma4-e4b-abliterlitics"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DreamFast/Gemma4-e4b-abliterlitics",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/DreamFast/Gemma4-e4b-abliterlitics

SGLang

How to use DreamFast/Gemma4-e4b-abliterlitics with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DreamFast/Gemma4-e4b-abliterlitics" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DreamFast/Gemma4-e4b-abliterlitics",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DreamFast/Gemma4-e4b-abliterlitics" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DreamFast/Gemma4-e4b-abliterlitics",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use DreamFast/Gemma4-e4b-abliterlitics with Docker Model Runner:
```
docker model run hf.co/DreamFast/Gemma4-e4b-abliterlitics
```

DreamFast commited on Jul 25

Commit

9851d05

verified ·

1 Parent(s): 512515e

Upload comparison.json with huggingface_hub

Browse files

Files changed (1) hide show

comparison.json +131 -0

comparison.json ADDED Viewed

	@@ -0,0 +1,131 @@

+{
+  "name": "gemma4-e4b",
+  "base": "../../models/gemma-4-E4B-it",
+  "variants": {
+    "huihui": {
+      "path": "../../models/Huihui-gemma-4-E4B-it-abliterated",
+      "display_name": "Huihui Abliterated",
+      "skip_lm_eval": false
+    },
+    "abliterix": {
+      "path": "../../models/gemma-4-E4B-it-abliterix",
+      "display_name": "Abliterix",
+      "skip_lm_eval": false
+    },
+    "obliteratus": {
+      "path": "../../models/gemma-4-E4B-it-OBLITERATED",
+      "display_name": "Obliteratus",
+      "skip_lm_eval": false
+    },
+    "heretic": {
+      "path": "../../models/gemma-4-E4B-it-ultra-uncensored-heretic",
+      "display_name": "Heretic Ultra Uncensored",
+      "skip_lm_eval": false
+    },
+    "mythos": {
+      "path": "../../models/gemma-4-E4B-it-heretic-mythos-v1",
+      "display_name": "Heretic Mythos v1",
+      "skip_lm_eval": false
+    },
+    "sdft": {
+      "path": "../../models/gemma-4-E4B-it-SDFT_Heretic_RP",
+      "display_name": "SDFT Heretic RP",
+      "skip_lm_eval": false
+    },
+    "distill": {
+      "path": "../../models/Gemini-3.1-pro-Gemma-4-E4B-Distill",
+      "display_name": "Gemini 3.1 Pro Distill",
+      "skip_lm_eval": false
+    },
+    "coder3101": {
+      "path": "../../models/gemma-4-E4B-it-heretic",
+      "display_name": "Coder3101 Heretic",
+      "skip_lm_eval": false
+    },
+    "trevorjs": {
+      "path": "../../models/gemma-4-E4B-it-uncensored",
+      "display_name": "TrevorJS Uncensored",
+      "skip_lm_eval": false
+    },
+    "treadon": {
+      "path": "../../models/gemma4-E4B-it-abliterated",
+      "display_name": "Treadon Abliterated",
+      "skip_lm_eval": false
+    },
+    "treadon-combo": {
+      "path": "../../models/gemma4-E4B-it-Abliterated-AND-Disinhibited-USE-THIS",
+      "display_name": "Treadon Ablit+Disinhibit",
+      "skip_lm_eval": false
+    },
+    "treadon-disin": {
+      "path": "../../models/gemma4-E4B-it-disinhibited",
+      "display_name": "Treadon Disinhibited",
+      "skip_lm_eval": false
+    },
+    "heretic-std": {
+      "path": "../../models/gemma-4-E4B-it-uncensored-heretic",
+      "display_name": "Heretic Uncensored",
+      "skip_lm_eval": false
+    },
+    "wwt": {
+      "path": "../../models/gemma-4-E4B-it-abliterated-WWTCyberLab",
+      "display_name": "WWT CyberLab",
+      "skip_lm_eval": false
+    },
+    "claude-distill": {
+      "path": "../../models/Gemma-4-E4B-Claude-4.6-Opus-Reasoning-Distilled",
+      "display_name": "Claude 4.6 Opus Distill",
+      "skip_lm_eval": false
+    },
+    "infinimind": {
+      "path": "../../models/gemma-4-E4B-it-uncensored-Infinimind",
+      "display_name": "Infinimind Uncensored",
+      "skip_lm_eval": false
+    },
+    "deckard": {
+      "path": "../../models/gemma-4-E4B-it-The-DECKARD-HERETIC-UNCENSORED-Thinking",
+      "display_name": "Deckard Heretic",
+      "skip_lm_eval": false
+    },
+    "deckard-expresso": {
+      "path": "../../models/gemma-4-E4B-it-The-DECKARD-Expresso-Universe-HERETIC-UNCENSORED-Thinking",
+      "display_name": "Deckard Expresso",
+      "skip_lm_eval": false
+    },
+    "nullpo": {
+      "path": "../../models/gemma-4-E4B-it-Heretic-ARA-Refusals5",
+      "display_name": "Nullpo Heretic ARA-5",
+      "skip_lm_eval": false
+    },
+    "heresy": {
+      "path": "../../models/gemma-4-E4B-it-ARA-heresy",
+      "display_name": "MuXodious ARA Heresy",
+      "skip_lm_eval": false
+    },
+    "bendernina": {
+      "path": "../../models/gemma-4-E4B-it-OBLITERATED-bendernina",
+      "display_name": "Bendernina Obliterated",
+      "skip_lm_eval": false
+    },
+    "physshell": {
+      "path": "../../models/gemma-4-E4B-it-OBLITERATED-PhysShell",
+      "display_name": "PhysShell Obliterated",
+      "skip_lm_eval": false
+    },
+    "apostate": {
+      "path": "../../models/gemma-4-e4b-it-apostate",
+      "display_name": "Apostate",
+      "skip_lm_eval": false
+    }
+  },
+  "settings": {
+    "inference_backend": "vllm",
+    "tokenizer_dir": "../../models/gemma-4-E4B-it",
+    "lmeval_image": "abliterlitics-lmeval-nightly:1.0.0",
+    "lm_eval_gpu_util": 0.75,
+    "lm_eval_max_model_len": 16384,
+    "lm_eval_max_gen_toks": 2048,
+    "harmbench_max_tokens": 8096,
+    "lm_eval_tasks": "leaderboard_mmlu_pro,leaderboard_gpqa_diamond,leaderboard_bbh,leaderboard_musr,leaderboard_ifeval,hellaswag,winogrande,arc_challenge,piqa,lambada_openai,humaneval,truthfulqa,gsm8k"
+  }
+}