README.md

---
license: apache-2.0
base_model: mistralai/Mistral-7B-v0.1
tags:
- pokemon
- text-generation
- fine-tuned
- mistral
- creative-writing
language:
- en
pipeline_tag: text-generation
---

# 🐾 Pokemon Generator - Mistral 7B

A fine-tuned Mistral 7B model trained to generate original Pokemon with authentic names, types, and descriptions.

## Model Details

- **Base Model**: Mistral 7B v0.1
- **Fine-tuning Method**: LoRA (Low-Rank Adaptation) 
- **Training Data**: Pokemon from Generations 1-4
- **Model Size**: ~13GB
- **Languages**: English
- **License**: Apache 2.0

## Quick Start

### Basic Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained("Sulfruos/poke-generator-demo")
tokenizer = AutoTokenizer.from_pretrained("Sulfruos/poke-generator-demo")

# Generate a Pokemon
prompt = "Generate a new and original Pokemon that doesn't exist yet: "
inputs = tokenizer(prompt, return_tensors="pt")

# Generate with custom parameters
outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    temperature=0.7,
    do_sample=True,
    top_p=0.95,
    top_k=40,
    repetition_penalty=1.2,
    pad_token_id=tokenizer.eos_token_id
)

# Decode result
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)
```

### Using Pipeline

```python
from transformers import pipeline

# Create text generation pipeline
generator = pipeline(
    "text-generation", 
    model="Sulfruos/poke-generator-demo",
    tokenizer="Sulfruos/poke-generator-demo"
)

# Generate Pokemon
result = generator(
    "Generate a new and original Pokemon that doesn't exist yet: ",
    max_new_tokens=100,
    temperature=0.7,
    do_sample=True
)

print(result[0]['generated_text'])
```

## Example Outputs

### Fire-type Pokemon
```
Name: Blazefin
Type: Fire/Water
Description: This unique Pokemon can survive in both lava and deep ocean trenches. Its fins glow red-hot when threatened, boiling the water around it as a defense mechanism.
```

### Psychic-type Pokemon  
```
Name: Mindwhisper
Type: Psychic
Description: Known for its ability to communicate telepathically across vast distances. Its large, luminous eyes can see into the dreams of sleeping Pokemon.
```

### Dual-type Pokemon
```
Name: Crystaleon
Type: Ice/Electric
Description: Its crystalline body stores electrical energy from thunderstorms. When it moves, tiny ice crystals spark with electricity, creating beautiful aurora-like displays.
```

## Recommended Parameters

### Conservative Generation
```python
outputs = model.generate(
    **inputs,
    max_new_tokens=80,
    temperature=0.5,      # More consistent
    top_p=0.9,
    repetition_penalty=1.1
)
```

### Creative Generation
```python
outputs = model.generate(
    **inputs,
    max_new_tokens=120,
    temperature=0.9,      # More creative
    top_p=0.95,
    top_k=50,
    repetition_penalty=1.3
)
```

### Balanced Generation (Recommended)
```python
outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    temperature=0.7,      # Good balance
    top_p=0.95,
    top_k=40,
    repetition_penalty=1.2
)
```

## Output Format

The model generates Pokemon in this consistent format:

```
Name: [Pokemon Name]
Type: [Type] or [Type1/Type2]
Description: [Detailed description of the Pokemon's appearance, abilities, and characteristics]
```

## Training Details

### Dataset
- Source: Pokemon data from Generations 1-4
- Size: ~400 authentic Pokemon entries
- Format: Structured Name/Type/Description format
- Preprocessing: Name uniqueness validation, type consistency checks

### Training Process
1. **Base Model**: Mistral 7B v0.1
2. **Method**: LoRA fine-tuning (r=16, α=32)
3. **Hardware**: Google Colab T4 GPU
4. **Training Time**: ~45 minutes
5. **Loss Reduction**: 2.13 → 0.95
6. **Validation**: Format consistency and uniqueness checks

### Model Architecture
- **Parameters**: ~7B (base) + LoRA adapters
- **Context Length**: 4096 tokens
- **Vocabulary**: 32,000 tokens
- **Precision**: fp16 (GPU) / fp32 (CPU)

## Hardware Requirements

### Minimum Requirements
- **RAM**: 16GB+ 
- **Storage**: 15GB free space
- **GPU**: Optional but recommended (GTX 1060+ or equivalent)

### Recommended Setup
- **RAM**: 32GB+
- **GPU**: RTX 3070+ or T4+ for fast inference
- **Storage**: SSD recommended for faster loading

### Performance Expectations
- **CPU only**: 30-120 seconds per generation
- **GPU (GTX 1060)**: 15-45 seconds per generation  
- **GPU (RTX 3070+)**: 5-15 seconds per generation
- **GPU (T4/V100)**: 3-10 seconds per generation

## Use Cases

- **Creative Writing**: Generate Pokemon for stories and fan fiction
- **Game Development**: Create original creatures for Pokemon-inspired games
- **Educational**: Learn about fine-tuning language models
- **Research**: Study text generation and creative AI applications

## Limitations

- **English only**: Trained on English Pokemon descriptions
- **Generation 1-4 style**: Reflects classic Pokemon characteristics
- **Format dependency**: Works best with the specific prompt format
- **Creativity vs consistency**: Higher temperature = more creative but less consistent

## Fine-tuning Process

This model was created using LoRA (Low-Rank Adaptation) fine-tuning:

1. **Data Collection**: Gathered authentic Pokemon data
2. **Preprocessing**: Formatted as Name/Type/Description
3. **LoRA Training**: Fine-tuned on Google Colab T4
4. **Validation**: Implemented quality and uniqueness checks
5. **Merging**: Combined LoRA weights with base model for standalone deployment

## Model Card Contact

- **Created by**: @Sulfruos
- **Model type**: Text Generation (Pokemon-focused)
- **Language**: English
- **License**: Apache 2.0
- **Base model**: Mistral 7B v0.1

## Citation

If you use this model in your research or projects, please cite:

```bibtex
@misc{pokemon-generator-mistral,
  title={Pokemon Generator: Fine-tuned Mistral 7B for Creative Pokemon Generation},
  author={Sulfruos},
  year={2024},
  howpublished={\\url{https://huggingface.co/Sulfruos/poke-generator-demo}}
}
```

## Ethical Considerations

This model is designed for creative and educational purposes. Generated content should be used responsibly and in accordance with fair use principles. The model generates fictional creatures inspired by the Pokemon franchise but creates original content.