|
|
--- |
|
|
library_name: transformers |
|
|
tags: |
|
|
- generated_from_trainer |
|
|
- text-generation |
|
|
- transformers |
|
|
- meta-math |
|
|
- qwen2 |
|
|
- symbolic-ai |
|
|
- symbioticlm |
|
|
model-index: |
|
|
- name: SymLM |
|
|
results: [] |
|
|
license: afl-3.0 |
|
|
datasets: |
|
|
- meta-math/MetaMathQA |
|
|
- open-thoughts/OpenThoughts2-1M |
|
|
language: |
|
|
- en |
|
|
base_model: |
|
|
- Qwen/Qwen2.5-0.5B |
|
|
pipeline_tag: text-generation |
|
|
metrics: |
|
|
- accuracy |
|
|
--- |
|
|
|
|
|
# 🧠 SymLM |
|
|
|
|
|
**SymbioticLM** is a hybrid symbolic–neural language model that integrates a frozen transformer backbone (`Qwen2ForCausalLM`) with a suite of symbolic cognitive modules for adaptive, interpretable reasoning. |
|
|
|
|
|
--- |
|
|
|
|
|
## 📐 Model Description |
|
|
|
|
|
The architecture fuses neural token-level generation with symbolic introspection and reasoning: |
|
|
|
|
|
- **Dynamic Thought Evolution with Helical Encoding and DNA-Inspired Memory (DTE-HDM)** |
|
|
Enables structured long-term memory and spiral-context encoding across tokens. |
|
|
|
|
|
- **Multi-Agent Symbiotic Response Mechanisms (M.A.S.R.M)** |
|
|
Coordinates symbolic-neural agents via gated attention and adaptive response layers. |
|
|
|
|
|
- **QwenExoCortex** |
|
|
Projects contextual hidden states from the Qwen model into a symbolic fusion space for reasoning and memory replay. |
|
|
|
|
|
- **Symbolic processors** |
|
|
Includes: |
|
|
- `ThoughtDynamicsLNN` |
|
|
- `Liquid / Crystalline Processors` |
|
|
- `Graph Reasoning with DNAConv` |
|
|
- A rolling `ThoughtMemory` |
|
|
|
|
|
This enables real-time fusion of symbolic thinking, token generation, and reasoning-aware language modeling. |
|
|
|
|
|
--- |
|
|
|
|
|
## 🎯 Intended Uses & Limitations |
|
|
|
|
|
### ✅ Intended Uses |
|
|
|
|
|
- **Mathematical reasoning and proof generation** |
|
|
Fine-tuned on *MetaMathQA*, optimized for symbolic Q&A, equation logic, and structured inference. |
|
|
|
|
|
- **Symbolic-cognitive AI research** |
|
|
Useful for studying attention modulation, memory replay, and neural-symbolic interface dynamics. |
|
|
|
|
|
- **Low-resource adaptation** |
|
|
Modular memory and projection design enables meaningful performance even with smaller datasets. |
|
|
|
|
|
- **Building adaptive cognition systems** |
|
|
Can serve as a symbolic kernel for reflective AI agents and knowledge evolution pipelines. |
|
|
|
|
|
--- |
|
|
|
|
|
### ⚠️ Limitations |
|
|
|
|
|
- **Limited training scale** |
|
|
Trained on 25,000 MetaMathQA examples. Effective for symbolic form, but not yet broad generalization. |
|
|
|
|
|
- **No RLHF or alignment** |
|
|
Outputs are not tuned for safety or instruction alignment and may hallucinate. |
|
|
|
|
|
- **Fluency ≠ correctness** |
|
|
Symbolic fluency does not imply mathematically valid proofs. Verification is recommended. |
|
|
|
|
|
- **Not optimized for open-domain generation** |
|
|
This model prioritizes logic and structure over conversational depth. |
|
|
|
|
|
--- |
|
|
|
|
|
## ⚙️ Training Procedure |
|
|
|
|
|
This checkpoint is currently in experimental phase. |
|
|
|
|
|
### 🧪 Training Hyperparameters |
|
|
|
|
|
- **learning_rate**: `3e-5` |
|
|
- **train_batch_size**: `16` |
|
|
- **eval_batch_size**: `16` |
|
|
- **gradient_accumulation_steps**: `64` |
|
|
- **total_train_batch_size**: `1024` |
|
|
- **optimizer**: `AdamW`, betas=(0.9, 0.999), epsilon=1e-08 |
|
|
- **lr_scheduler_type**: `cosine` |
|
|
- **warmup_steps**: `500` |
|
|
- **num_epochs**: `3` |
|
|
- **mixed_precision_training**: `Native AMP` |
|
|
|
|
|
--- |
|
|
|
|
|
## 🧱 Framework Versions |
|
|
|
|
|
- 🤗 Transformers: `4.51.3` |
|
|
- 🧠 PyTorch: `2.7.0+cu126` |
|
|
- 📚 Datasets: `3.5.0` |
|
|
- 🔤 Tokenizers: `0.21.1` |
|
|
|
|
|
--- |
|
|
|
|
|
## 📚 Research Foundations |
|
|
|
|
|
SymbioticLM builds upon a cohesive theoretical framework for dynamic reasoning and neuro-symbolic learning: |
|
|
|
|
|
### 🔁 Multi-Agent Symbiosis and Dynamic Thought |
|
|
|
|
|
**Rapid Adaptation via Multi-Agent Symbiotic Response Mechanisms (M.A.S.R.M)** |
|
|
> A framework where symbolic and neural agents dynamically adapt via gated feedback, memory modulation, and agent-based specialization. |
|
|
|
|
|
**Focus**: Multi-agent control, reflective learning, contextual responsiveness |
|
|
|
|
|
--- |
|
|
|
|
|
### 🧬 Dynamic Thought Evolution with Helical Encoding and DNA-Inspired Memory (DTE-HDM) |
|
|
|
|
|
> A memory structure inspired by biological helices, enabling thought persistence through spiral-layered contextual encodings across time. |
|
|
|
|
|
**Focus**: Long-term token evolution, normalized replay, thought continuity |
|
|
|
|
|
--- |
|
|
|
|
|
### 🧠 Integrating DTE-HDM + M.A.S.R.M for Adaptive AI |
|
|
|
|
|
> Combines symbolic evolution and multi-agent adaptation to construct an LLM that reflects, adapts, and deepens reasoning through internal dynamics. |
|
|
|
|
|
**Result**: A system that *learns faster*, *adapts deeper*, and *thinks symbolically* |
|
|
|
|
|
--- |
|
|
|
|
|
### 📐 Theoretical Underpinning |
|
|
|
|
|
**The Analytic Foundations Theorem (AFT)** |
|
|
> A rigorous, measure-theoretic replacement for classical calculus: replaces pointwise derivatives with discrepancy-driven integral convergence across vanishing sets. |
|
|
|
|
|
**Applies to**: |
|
|
- Symbolic gradients |
|
|
- Gradient-free optimization |
|
|
- Discrete logic approximation in function spaces |
|
|
|
|
|
--- |
|
|
|
|
|
These form the **mathematical and architectural core** of SymbioticLM, enabling: |
|
|
|
|
|
- 🧠 *Neuro-symbolic cognitive evolution* |
|
|
- 🔁 *Multi-agent dynamic feedback coordination* |
|
|
- 📏 *Formal memory through discrepancy-based logic* |
|
|
|
|
|
--- |
