🍷 FineData

This is the home of the 🍷 FineData team, a branch of the 🤗 Hugging Face Science Team releasing large scale pre-training datasets to accelerate open LLM development.

• 🍷 FineWeb: A 15T tokens English dataset for LLM pre-training. See the blogpost and paper.
• 📚 FineWeb-Edu: a filtered subset of the most educational content from FineWeb.
• 🥂 FineWeb2: an extension of FineWeb to over 1000 languages. See the paper.
• 📄 FinePDFs: 3T tokens of text data extracted from PDFs sourced from the Web. See the blogpost
• 🌐 FineWiki: an updated, better extracted version of Wikipedia in 300+ languages.
• 📄 FinePDFs-Edu: 350B+ highly educational tokens filtered from 📄 FinePDFs
• 💬 FineTranslations: 1+1T tokens of parallel text translated from 500+ 🥂 FineWeb2 languages