Branis333/astro-gpt2-chatbot

Astronomy GPT-2 Chatbot

This is a fine-tuned GPT-2 model for answering astronomy questions.

Model Details

• Base Model: GPT-2
• Training Data: 2,736 astronomy Q&A pairs
• Final Perplexity: 1.61
• Training Epochs: 10
• Domain: Astronomy, Space Science

Usage

from transformers import GPT2Tokenizer, GPT2LMHeadModel

# Load model
tokenizer = GPT2Tokenizer.from_pretrained("Branis333/astro-gpt2-chatbot")
model = GPT2LMHeadModel.from_pretrained("Branis333/astro-gpt2-chatbot")

# Generate answer
question = "What is a black hole?"
prompt = f"Q: {question}\nA:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100, pad_token_id=tokenizer.eos_token_id)
answer = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(answer)

Training Details

• Batch Size: 2
• Learning Rate: 2e-5
• Max Sequence Length: 512
• Optimizer: AdamW
• GPU: Tesla T4

Performance

Epoch	Train Loss	Val Loss	Perplexity
1	0.5524	0.4702	1.60
5	0.3928	0.4498	1.57
10	0.2912	0.4740	1.61

Dataset

The model was trained on cleaned astronomy Q&A pairs covering:

• Terminology and definitions
• Multiple choice questions with explanations
• True/false questions
• Matching questions

Limitations

• Specialized for astronomy domain only
• May generate factually incorrect information
• Best used for educational purposes
• Should be fact-checked for accuracy

Citation

If you use this model, please cite:

@misc{astro-gpt2-chatbot,
  author = {Your Name},
  title = {Astronomy GPT-2 Chatbot},
  year = {2024},
  publisher = {HuggingFace},
  url = {https://huggingface.co/Branis333/astro-gpt2-chatbot}
}