Model Card for warren-buffett-letters-qna-r1-enhanced-1998-2024-finetuned-llama-3.2-1B-Instruct
This model is fine-tuned to answer questions based on Warren Buffett’s annual shareholder letters from 1998 to 2024. It understands the themes, vocabulary, and tone of Buffett’s writing and is capable of responding to questions about his investment philosophy, decisions, and observations.
Invoke notebook shared here, a publicly available Colab notebook for tests.
Model Details
Model Description
This is a transformer-based question-answering model fine-tuned from unsloth/Llama-3.2-1B-Instruct. It was trained on a dataset derived from Warren Buffett’s letters to Berkshire Hathaway shareholders. The dataset pairs real excerpts with corresponding questions and answers for a conversational learning experience.
- Developed by: Yiqiao Yin
- Model type: Causal Language Model (fine-tuned for Q&A)
- Language(s): English
- License: MIT
- Finetuned from model: unsloth/Llama-3.2-1B-Instruct
Uses
Direct Use
This model can be used to:
- Ask questions about specific themes or time periods in Warren Buffett’s letters
- Learn about value investing and Buffett’s decision-making
- Generate educational content based on his financial wisdom
Out-of-Scope Use
- This model is not suited for general-purpose financial advice.
- It may not generalize well outside the context of Buffett’s letters.
Bias, Risks, and Limitations
The model inherits the biases and perspectives from Warren Buffett’s letters, which reflect his personal views and investment philosophy. While these views are valuable, they do not represent all schools of financial thought. Also, since the model was fine-tuned on a niche dataset, it may not perform well on unrelated questions or general knowledge.
Recommendations
Always verify model outputs, especially if using for educational or advisory purposes.
How to Get Started with the Model
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("eagle0504/warren-buffett-letters-qna-r1-enhanced-1998-2024-finetuned-llama-3.2-1B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("eagle0504/warren-buffett-letters-qna-r1-enhanced-1998-2024-finetuned-llama-3.2-1B-Instruct")
inputs = tokenizer("Question: What is intrinsic value?\nAnswer:", return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0]))
Training Details
Training Data
- Dataset: eagle0504/warren-buffett-letters-qna-r1-enhanced-1998-2024
- Format:
{"question": "...", "answer": "..."}based on text from the letters
Training Procedure
Preprocessing
A formatting function was used to convert each entry to:
Question: <question text>
Answer: <answer text>
Training Hyperparameters
- Epochs: 50
- Batch Size: 8
- Learning Rate: 2e-5
- Gradient Accumulation: 1
- Mixed Precision: No (fp32)
- Framework: 🤗 Transformers + TRL + DeepSpeed
Final Training Metrics
- Loss: 0.0532
- Gradient Norm: 0.2451
- Learning Rate: 9.70e-08
- Mean Token Accuracy: 98.05%
- Final Epoch: 49.76
Compute Infrastructure
- Hardware: 4× NVIDIA RTX A6000, 38 vCPUs, 200 GB RAM
- Cloud Provider: Runpod
- Docker Image:
runpod/pytorch:2.1.0-py3.10-cuda11.8.0-devel-ubuntu22.04 - Package Manager:
uv
Environmental Impact
Carbon emissions can be estimated using the Machine Learning Impact calculator.
- Hardware Type: 4× NVIDIA RTX A6000 GPUs
- Hours used: 19 hours
- Cloud Provider: Runpod
- Compute Region: Unknown
- Training Cost: $1.96/hour → Total: $37.24
- Carbon Emitted: Not formally calculated
Model Card Contact
Author: Yiqiao Yin Connect with me on LinkedIn
- Downloads last month
- 134