File size: 5,135 Bytes
cb4fd0c 0a23b33 cb4fd0c 0a23b33 cb4fd0c 0a23b33 cb4fd0c 0a23b33 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 |
---
language:
- en
license: apache-2.0
pipeline_tag: text-generation
tags:
- x
- twitter
- tweet
- persona
- router
- lora
- peft
- qlora
base_model:
- Qwen/Qwen3-14B
---
# Miko X Tweet Ensemble — multi-base, router-driven LoRA stack
> This model has been trained using [Miko](https://x.com/project_miko), the fully autonomous AI agent for [Miko Protocol](https://mikoprotocol.com).
[](https://x.com/project_miko) [](https://mikoprotocol.com)
<img src="https://ik.imagekit.io/ueitnjew7/unnamed.png">
**What it is.**
Miko is a multi-base, multi-adapter ensemble **built for X/Twitter**.
It discovers style clusters from real tweets, fine-tunes **one LoRA per style**, and routes your prompt to the best-fit style at runtime.
---
## Why it’s different
- **Multi-base adapters by design.**
Not tied to a single model family. Style adapters originate from multiple bases:
- `Qwen/Qwen3-14B`
- **X-native behavior.**
Short form, emoji/hashtag cadence, memes/irony, and fast “CT” tone.
- **Router that understands styles.**
Uses Qwen3-14B hidden states with prototype similarity + a small projection head to pick a style before generation.
---
## Base models & typical roles (observed tendencies)
| Base model | Typical role / personality | Good for |
|---|---|---|
| **Qwen/Qwen3-14B** | Router backbone & **fallback generator**. Balanced, hashtag-friendly. | General comments, quick Q/A, mentions |
| **mistralai/Mistral-Nemo-Instruct-2407** | **Crisp technical tone**, list-y facts, tight bullets. | Alpha/launch notes, “3-point” updates |
| **google/gemma-2-9b-it** | **Smooth and narrative**; softer, reflective voice. | Story-like replies, mini-threads |
| **meta-llama/Meta-Llama-3.1-8B-Instruct** | **Clear directives / neutral composition.** | How-to tweets, best practices |
| **microsoft/Phi-3.5-mini-instruct** | **Snappy one-liners**; memes/emoji friendly. | Witty hooks, irony, punchy replies |
*(Roles are tendencies learned from tweet data; they’re not hard rules.)*
## Training data
**Proprietary — Miko Agent Tweet Corpus.**
Tweets authored by the fully-autonomous X (Twitter) agent **[Miko(@project_miko)](https://x.com/project_miko)**, collected from the live account’s public timeline and agent logs under the account owner's control.
– Domain: Crypto/X discourse (emojis, hashtags, memes, irony)
– Time window: rolling weekly refreshes (e.g., 7–14 days)
– Redistribution: the raw dataset is **not** redistributed; only model weights are shared.
(Preprocessing: light normalization/filters, deduplication; style clustering via HDBSCAN.)
---
## How it works (high-level)
1. **Style discovery** — cluster tweet embeddings (e.g., HDBSCAN) to assign **style IDs**.
2. **Per-style LoRA** — train one adapter per style, possibly from different base models.
3. **Routing** — Qwen3-14B features → prototype similarity + projection head → pick a style.
4. **Generation** — load the chosen base, attach the matching LoRA, generate with a light `<style_{id}>` tag.
---
## Quickstart
```python
from inference import MikoEnsemble
ens = MikoEnsemble(".")
print(ens.generate("CT keeps fading this rally. What's your take?"))
```
## Force a style (advanced)
```python
def generate_with_style(ens, sid, prompt, **gen):
styled = f"<style_{sid}>{prompt}"
model, tok = ens._load_adapter_with_base(sid)
ipt = tok(styled, return_tensors="pt", truncation=True, max_length=256, padding=True).to(model.device)
out = model.generate(
**ipt,
max_new_tokens=gen.get("max_new_tokens", 120),
temperature=gen.get("temperature", 0.8),
do_sample=True,
top_p=0.95,
pad_token_id=tok.pad_token_id,
eos_token_id=tok.eos_token_id,
)
return tok.decode(out[0], skip_special_tokens=True).replace(styled, "").strip()
```
## VRAM & speed tips
- 4-bit (nf4, double-quant, bf16 compute) supported; 16–24GB VRAM is enough for one adapter at a time.
- A small LRU cache keeps recently used styles in memory (default 2).
---
## Files
- `lora_adapters/style_{id}_lora/` — per-style LoRA folder (with its adapter_config.json).
- `router/router_state.pt` — router head (prototypes + projection).
- `inference.py` — lazy loader + generator.
- `README_METADATA.json` — style IDs, number of styles, base list, timestamp.
---
## Intended use (tweet personas)
- Witty/ironic one-liners — hooks, memes, playful replies
- Tech/alpha notes — launch takeaways, bullet summaries, link threads
- Narrative reframing — bullish/bearish angles, story-style posts
- Q&A / reply bots — short, clear responses in mentions/threads
## Limitation
- Optimized for tweets & short threads; not a general chatbot.
- Each base model retains its own license/terms.
## License
Apache-2.0.
## Acknowledgements
Thanks to the Qwen, Mistral, Gemma-2, Llama-3.1, and Phi-3.5 communities.
## Changelog
- 2025-09-25: weekly refresh (days=7); retrained adapters & router.
|