projectmiko/miko-x-multibase-lora-ensemble

Miko X Tweet Ensemble — multi-base, router-driven LoRA stack

This model has been trained using Miko, the fully autonomous AI agent for Miko Protocol.

What it is. Miko is a multi-base, multi-adapter ensemble built for X/Twitter. It discovers style clusters from real tweets, fine-tunes one LoRA per style, and routes your prompt to the best-fit style at runtime.

---

Why it’s different

• Multi-base adapters by design. Not tied to a single model family. Style adapters originate from multiple bases:

• Qwen/Qwen3-14B

• X-native behavior. Short form, emoji/hashtag cadence, memes/irony, and fast “CT” tone.

• Router that understands styles. Uses Qwen3-14B hidden states with prototype similarity + a small projection head to pick a style before generation.

---

Base models & typical roles (observed tendencies)

Base model	Typical role / personality	Good for
Qwen/Qwen3-14B	Router backbone & fallback generator. Balanced, hashtag-friendly.	General comments, quick Q/A, mentions
mistralai/Mistral-Nemo-Instruct-2407	Crisp technical tone, list-y facts, tight bullets.	Alpha/launch notes, “3-point” updates
google/gemma-2-9b-it	Smooth and narrative; softer, reflective voice.	Story-like replies, mini-threads
meta-llama/Meta-Llama-3.1-8B-Instruct	Clear directives / neutral composition.	How-to tweets, best practices
microsoft/Phi-3.5-mini-instruct	Snappy one-liners; memes/emoji friendly.	Witty hooks, irony, punchy replies

(Roles are tendencies learned from tweet data; they’re not hard rules.)

Training data

Proprietary — Miko Agent Tweet Corpus. Tweets authored by the fully-autonomous X (Twitter) agent Miko(@project_miko), collected from the live account’s public timeline and agent logs under the account owner's control. – Domain: Crypto/X discourse (emojis, hashtags, memes, irony) – Time window: rolling weekly refreshes (e.g., 7–14 days) – Redistribution: the raw dataset is not redistributed; only model weights are shared. (Preprocessing: light normalization/filters, deduplication; style clustering via HDBSCAN.)

---

How it works (high-level)

1. Style discovery — cluster tweet embeddings (e.g., HDBSCAN) to assign style IDs.
2. Per-style LoRA — train one adapter per style, possibly from different base models.
3. Routing — Qwen3-14B features → prototype similarity + projection head → pick a style.
4. Generation — load the chosen base, attach the matching LoRA, generate with a light <style_{id}> tag.

---

Quickstart

from inference import MikoEnsemble

ens = MikoEnsemble(".")
print(ens.generate("CT keeps fading this rally. What's your take?"))

Force a style (advanced)

def generate_with_style(ens, sid, prompt, **gen):
    styled = f"<style_{sid}>{prompt}"
    model, tok = ens._load_adapter_with_base(sid)
    ipt = tok(styled, return_tensors="pt", truncation=True, max_length=256, padding=True).to(model.device)
    out = model.generate(
        **ipt,
        max_new_tokens=gen.get("max_new_tokens", 120),
        temperature=gen.get("temperature", 0.8),
        do_sample=True,
        top_p=0.95,
        pad_token_id=tok.pad_token_id,
        eos_token_id=tok.eos_token_id,
    )
    return tok.decode(out[0], skip_special_tokens=True).replace(styled, "").strip()

VRAM & speed tips

• 4-bit (nf4, double-quant, bf16 compute) supported; 16–24GB VRAM is enough for one adapter at a time.
• A small LRU cache keeps recently used styles in memory (default 2).

---

Files

• lora_adapters/style_{id}_lora/ — per-style LoRA folder (with its adapter_config.json).
• router/router_state.pt — router head (prototypes + projection).
• inference.py — lazy loader + generator.
• README_METADATA.json — style IDs, number of styles, base list, timestamp.

---

Intended use (tweet personas)

• Witty/ironic one-liners — hooks, memes, playful replies
• Tech/alpha notes — launch takeaways, bullet summaries, link threads
• Narrative reframing — bullish/bearish angles, story-style posts
• Q&A / reply bots — short, clear responses in mentions/threads

Limitation

• Optimized for tweets & short threads; not a general chatbot.
• Each base model retains its own license/terms.

License

Apache-2.0.

Acknowledgements

Thanks to the Qwen, Mistral, Gemma-2, Llama-3.1, and Phi-3.5 communities.

Changelog

• 2025-09-25: weekly refresh (days=7); retrained adapters & router.