---
license: mit
datasets:
  - openwebtext
  - alpacha
tags:
  - text-generation
  - gpt-2
  - openwebtext
  - alpacha
model_name: gpt2-81M
language: en
---

# GPT-2 81M Fine-tuned on OpenWebText and Alpacha

This model is a fine-tuned version of GPT-2 (81M parameters) trained on the OpenWebText dataset and further fine-tuned on the Alpacha dataset.

## Model Description

This model is based on the GPT-2 architecture and has been fine-tuned on a combination of two datasets:

1. **OpenWebText**: The model was initially trained on the OpenWebText dataset for 600,000 iterations.
2. **Alpacha**: The model was further fine-tuned on the Alpacha dataset for the remaining 50,000 iterations.

The model was trained using a **laptop with an RTX 3060 GPU** for a total of **650,000 iterations** (approximately **8 days** of training).

## Hardware Details

- **GPU**: Laptop with an **RTX 3060**
- **Training Time**: The model took **8 days** (approximately 650,000 iterations) to train.
- **Total Iterations**: 650,000 iterations (600,000 on OpenWebText + 50,000 on Alpacha).

## How to Use

You can use this model for text generation with the Hugging Face `transformers` library:

```python
from transformers import GPT2LMHeadModel, GPT2Tokenizer

model = GPT2LMHeadModel.from_pretrained("Aaltjo/gpt2-81M-openwebtext-alpacha")
tokenizer = GPT2Tokenizer.from_pretrained("Aaltjo/gpt2-81M-openwebtext-alpacha")

input_text = "Once upon a time"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```