File size: 1,558 Bytes
3376fe0 72e02f6 3376fe0 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
---
license: apache-2.0
datasets:
- Allanatrix/Scientific_Research_Tokenized
language:
- en
base_model:
- allenai/OLMo-7B
pipeline_tag: text-generation
library_name: peft
tags:
- Olmo
- lora
- peft
- transformers
- scientific-ml
- fine-tuned
- research-assistant
- hypothesis-generation
- scientific-writing
- scientific-reasoning
---
# Model Card for nexa-OLMo-sci7b
## Model Details
**Model Description:**
nexa-OLMo-sci7b is a fine-tuned variant of allenai/OLMo-7B, optimized for scientific research generation tasks such as hypothesis generation, abstract writing, and methodology completion. Fine-tuning was performed using PEFT with LoRA in 4-bit quantized mode via bitsandbytes.
**Developed by:** Allan (Independent Scientific Intelligence Architect)
**Shared by:** Allan (https://huggingface.co/allan-wandia)
**Model type:** Decoder-only transformer (causal language model)
**Language(s):** English (scientific domain-specific vocabulary)
**License:** Apache 2.0
**Fine-tuned from:** allenai/OLMo-7B
**Repository:** https://huggingface.co/allan-wandia/nexa-olmo-sci7b
## Training Details
**Training Data:**
- Size: 100 million tokens
- Source: Curated scientific literature (Bio, Physics, QST, Astro)
**Hyperparameters:**
- Sequence length: 1024
- Batch size: 1
- Gradient Accumulation Steps: 64
- Effective Batch Size: 64
- Learning rate: 2e-05
- Epochs: 2
- LoRA: Enabled (PEFT)
- Quantization: 4-bit
**Results:**
Robust performance in scientific prose tasks, with novelty varying by prompt diversity. |