5 open-source datasets used to train LLMs

1. MineDojo (videos and text)
2. ROOTS (text)
3. NaturalInstructions-v2 (text)
4. Anthropic Helpfulness dataset (text)
5. LAION-115M(text and image)

Links 🧵

MineDojo (videos and text)

https://crfm.stanford.edu/ecosystem-graphs/index.html?asset=MineDojo

ROOTS

A 1.6TB dataset spanning 59 languages

https://crfm.stanford.edu/ecosystem-graphs/index.html?asset=ROOTS

NaturalInstructions-v2

https://crfm.stanford.edu/ecosystem-graphs/index.html?asset=NaturalInstructions-v2

Anthropic Helpfulness dataset

https://crfm.stanford.edu/ecosystem-graphs/index.html?asset=Anthropic%20Helpfulness%20dataset

LAION-115M

https://crfm.stanford.edu/ecosystem-graphs/index.html?asset=LAION-115M

5 Open-Source Datasets for LLM Training