Datasets
Public datasets and corpora for training, evaluating and grounding models.
JSON: /api/v1/directory.json (filter on category=datasets)
Common Crawl
Free, open repository of petabytes of web crawl data, useful for research and training datasets.
FreeNo authData.gov
Catalog of US federal open datasets with a CKAN-based search API.
FreeNo authHugging Face Datasets
Hub of community datasets with a viewer API for browsing and downloading them.
FreeNo authKaggle Datasets (Free API)
Browse and download hundreds of thousands of open datasets for AI research, model training, and analysis.
See siteSee docsEuropean Data Portal
Discover the single point of access to open data from European countries, EU institutions, agencies and bodies and other European countries.
See siteSee docsAwesome Public Datasets (GitHub)
A topic-centric list of HQ open datasets.
FreeNo authFineWeb
Large cleaned web text dataset.
FreeNo authDolma
Allen AI's open pretraining corpus and toolkit.
FreeNo authRedPajama
Open pretraining dataset and tools.
FreeNo authOpenWebText
Open replication of WebText.
FreeNo authLAION
Open large-scale image-text datasets.
FreeNo authImageNet
Landmark image classification dataset.
FreeNo authCOCO
Common Objects in Context image dataset.
FreeNo authOpen Images
Google's large annotated image dataset.
FreeNo authCommon Voice
Mozilla's crowdsourced speech dataset.
FreeNo authLibriSpeech
Speech corpus of read English audiobooks.
FreeNo authOpenSLR
Open speech and language resources.
FreeNo authGoogle Dataset Search
Search engine for datasets.
FreeNo authUN Data
United Nations statistical data.
FreeNo authOur World in Data
Research and data on global problems.
FreeNo authOpenML
Open platform for sharing datasets and ML experiments.
FreeAPI keyUCI ML Repository
Classic machine-learning datasets.
FreeNo authWikipedia Dumps
Full database dumps of Wikipedia.
FreeNo authProject Gutenberg
Free ebooks; Gutendex offers a JSON API.
FreeNo authInternet Archive
Digital library with APIs.
FreeNo authStack Exchange Data Dump
Stack Exchange public dumps.
FreeNo authThe Stack
Large permissively-licensed source code dataset.
FreeNo authOpenHermes
Open instruction-tuning datasets.
FreeNo authUltraFeedback
Preference dataset for alignment.
FreeNo authOpenStreetMap Planet
Full OSM data dumps.
FreeNo authNASA Open Data
NASA open datasets and APIs.
FreeAPI keyRoboflow Universe
Open computer vision datasets and models.
Free tierAPI key