ଓOpenOdia

Browser

Odia datasets.

Live from Hugging Face — every dataset tagged for Odia. Parallel corpora, speech, classification, instruction-tuning, each with its size, license, and a citation.

★ Featuredrotates every Monday
Other

taco-datasets

@saillab

This repo consists of the datasets used for the TaCo paper. There are four datasets: Multilingual Alpaca-52K GPT-4 dataset Multilingual Dolly-15K GPT-4 dataset TaCo dataset Multilingual Vicuna Benchmark dataset We translated the first three datasets using G…

326 17
Translation

Pralekha

@ai4bharat

Pralekha: Cross-Lingual Document Alignment for Indic Languages Pralekha is a large-scale parallel document dataset spanning across 11 Indic languages and English. It comprises over 3 million document pairs, with 1.

461 16
Also worth a look
Text Classification

indic_glue

2.3k 15Token Classification

universal_dependencies

5.8k 8Translation

Tatoeba-Translations

130 6
Text Generation

wikipedia

@wikimedia

Dataset Card for Wikimedia Wikipedia Dataset Summary Wikipedia dataset containing cleaned articles of all languages. The dataset is built from the Wikipedia dumps with one subset per language, each containing a single train split.

10M < n < 100M rows 299.8k 1.7k
CC-BY-SA-3.0textCite
Text Generation

wikipedia

@legacy-datasets

Wikipedia dataset containing cleaned articles of all languages. The datasets are built from the Wikipedia dump with one split per language.

n < 1K rows 58.9k 685
CC-BY-SA-3.0Cite
Other

xP3

@bigscience

xP3 (Crosslingual Public Pool of Prompts) is a collection of prompts & datasets across 46 of languages & 16 NLP tasks. It is used for the training of BLOOMZ and mT0, multilingual language models capable of following human instructions in dozens of languages…

100M < n < 1B rows 53.4k 114
Apache-2.0Cite
Text Generation

sangraha

@ai4bharat

Sangraha Sangraha is the largest high-quality, cleaned Indic language pretraining data containing 251B tokens summed up over 22 languages, extracted from curated sources, existing multilingual corpora and large scale translations.

100M < n < 1B rows 46.4k 86
CC-BY-4.0textCite
Question Answering

belebele

@facebook

The Belebele Benchmark for Massively Multilingual NLU Evaluation Belebele is a multiple-choice machine reading comprehension (MRC) dataset spanning 122 language variants. This dataset enables the evaluation of mono- and multi-lingual models in high-, medium…

100K < n < 1M rows 41.8k 142
CC-BY-SA-4.0textCite
Token Classification

wikiann

@unimelb-nlp

Dataset Card for WikiANN Dataset Summary WikiANN (sometimes called PAN-X) is a multilingual named entity recognition dataset consisting of Wikipedia articles annotated with LOC (location), PER (person), and ORG (organisation) tags in the IOB2 format. This v…

1M < n < 10M rows 36.4k 126
No licensetextCite
Automatic Speech Recognition

common_voice_16_0

@fsicoli

Dataset Card for Common Voice Corpus 16. 0 This dataset is an unofficial version of the Mozilla Common Voice Corpus 16.

100B < n < 1T rows 27.5k 4
CC0-1.0Cite
Text Generation

CulturaX

@uonlp

CulturaX Cleaned, Enormous, and Public: The Multilingual Fuel to Democratize Large Language Models for 167 Languages Dataset Summary We present CulturaX, a substantial multilingual dataset with 6. 3 trillion tokens in 167 languages, tailored for large langu…

1B < n < 10B rows 26.0k 701
No licensetextCite
Automatic Speech Recognition

common_voice_15_0

@fsicoli

Dataset Card for Common Voice Corpus 15. 0 This dataset is an unofficial version of the Mozilla Common Voice Corpus 15.

100B < n < 1T rows 25.3k 6
No licenseCite
Translation

opus-100

@Helsinki-NLP

Dataset Card for OPUS-100 Dataset Summary OPUS-100 is an English-centric multilingual corpus covering 100 languages. OPUS-100 is English-centric, meaning that all training pairs include English on either the source or target side.

10M < n < 100M rows 24.8k 255
No licensetextCite
Automatic Speech Recognition

Vaani

@ARTPARK-IISc

VAANI is an India-representative multi-modal multi-lingual dataset. The current version (phase 1- 80 districts, phase 2- 85 districts) contains ~31278 hours of spontaenous,image-prompted speech by 156K speakers across 165 districts, talking about 288K image…

1M < n < 10M rows 21.2k 173
CC-BY-4.0audioimagetextCite
Automatic Speech Recognition

bolAIndia

@kapturecx

bolAIndia A growing atlas of India’s spoken voices India does not speak in a single voice. It speaks through a vast and vibrant mosaic of languages, regional accents, dialects, and everyday expressions—across cities, towns, and rural communities.

17.2k 4
No licenseCite
Automatic Speech Recognition

common_voice_22_0

@fsicoli

Dataset Card for Common Voice Corpus 22. 0 This dataset is an unofficial version of the Mozilla Common Voice Corpus 22.

100B < n < 1T rows 16.5k 20
CC0-1.0Cite
Other

xP3all

@bigscience

xP3 (Crosslingual Public Pool of Prompts) is a collection of prompts & datasets across 46 of languages & 16 NLP tasks. It is used for the training of BLOOMZ and mT0, multilingual language models capable of following human instructions in dozens of languages…

10M < n < 100M rows 15.5k 33
Apache-2.0textCite
Other

Global-MMLU-Lite

@CohereLabs

Releases: Version 3. 0 (May 2026): GMMLU Lite 3.

10K < n < 100K rows 14.1k 45
Apache-2.0textCite
Other

Complete_Data_Source_100K_HOURS

@RidheshBhati

Multi-Language Audio Collection (100K Hours) This repository is physically reorganized for Absolute 100% Data Visibility. 🏗️ Global Consolidator Select your language subset to listen to high-quality waveform audio.

1M < n < 10M rows 13.1k 4
No licenseaudiotextCite
Other

OPUS_GlobalVoices

@wecover

10M < n < 100M rows 12.1k 0
No licensetextCite
Automatic Speech Recognition

common_voice_17_0

@fsicoli

Dataset Card for Common Voice Corpus 17. 0 This dataset is an unofficial version of the Mozilla Common Voice Corpus 17.

100B < n < 1T rows 12.0k 19
CC0-1.0Cite
Text Generation

oscar-small

@nthngdy

The Open Super-large Crawled ALMAnaCH coRpus is a huge multilingual corpus obtained by language classification and filtering of the Common Crawl corpus using the goclassy architecture.

10M < n < 100M rows 11.2k 17
CC0-1.0textCite
Text Generation

wikipedia-monthly

@omarkamali

🚀 Wikipedia Monthly Last updated: October 19, 2026, 06:34 UTC This repository provides monthly, multilingual dumps of Wikipedia, processed and prepared for easy use in NLP projects. 📊 Current Statistics Metric Current Export (October 2026) All Exports (To…

10.9k 84
CC-BY-SA-4.0Cite
Text Generation

flores_101

@gsarti

One of the biggest challenges hindering progress in low-resource and multilingual machine translation is the lack of good evaluation benchmarks. Current evaluation benchmarks either lack good coverage of low-resource languages, consider only restricted doma…

100K < n < 1M rows 10.8k 34
CC-BY-SA-4.0tabulartextCite
Audio Classification

ml_spoken_words

@MLCommons

Multilingual Spoken Words Corpus is a large and growing audio dataset of spoken words in 50 languages collectively spoken by over 5 billion people, for academic research and commercial applications in keyword spotting and spoken term search, licensed under …

10M < n < 100M rows 9.3k 37
CC-BY-4.0Cite
Other

xP3mt

@bigscience

xP3 (Crosslingual Public Pool of Prompts) is a collection of prompts & datasets across 46 of languages & 16 NLP tasks. It is used for the training of BLOOMZ and mT0, multilingual language models capable of following human instructions in dozens of languages…

10M < n < 100M rows 9.3k 27
Apache-2.0textCite
Text To Speech

Pretraining-V1

@projectkaira

Indic TTS Unified v1 A large-scale, unified collection of speech data for text-to-speech (TTS) and speech research. This dataset consolidates 17 distinct source datasets into a single, schema-normalized resource covering Indian / South Asian languages, plus…

10M < n < 100M rows 8.4k 0
CC-BY-4.0audiotextCite
Text Classification

ccnews

@stanford-oval

This dataset is the result of processing all WARC files in the CCNews Corpus, from the beginning (2016) to June of 2024. The data has been cleaned and deduplicated, and language of articles have been detected and added.

100M < n < 1B rows 8.4k 37
No licenseimagetextCite
Other

STT_MODEL

@RidheshBhati

Multilingual STT Dataset Audio and transcript pairs for 50 languages. Each language is a Dataset Viewer configuration with train, validation, and test splits.

1M < n < 10M rows 8.4k 1
No licenseaudiotextCite
Text To Speech

indicvoices_r

@ai4bharat

IndicVoices-R: Multilingual, Multi-Speaker Speech Corpus for Indian TTS Dataset Summary IndicVoices-R (IV-R) is the largest multilingual Indian text-to-speech (TTS) dataset derived from an automatic speech recognition (ASR) dataset. It contains 1,704 hours …

100K < n < 1M rows 6.8k 49
CC-BY-4.0audiotextCite
Image To Text

NayanaOCR_Corpus_2025

@Cognitive-Lab

🪷 NayanaOCR Corpus 2025 A 1M-page, 22-language fully-parallel synthetic OCR + VQA corpus for document-centric vision-language models — every page rendered in every language. NayanaOCR Corpus 2025 is one of the largest open-source multilingual, multi-task d…

1M < n < 10M rows 6.4k 18
CC-BY-NC-4.0imagetextCite
Token Classification

universal_dependencies

@universal-dependencies

Dataset Card (v2. 0) for Universal Dependencies Treebank Version 2.

1M < n < 10M rows 5.8k 8
No licensetextCite
Text To Speech

yodas2_sidon

@sarulab-speech

YODAS2-Sidon Overview This dataset is a cleansed version of YODAS-2 with Sidon speech restoration mode for Speech Synthesis and Spoken Language Modeling. YODAS-2 is a massive, multilingual YouTube-derived dataset.

1M < n < 10M rows 5.7k 65
CC-BY-3.0audiotextCite

Showing 30 of 536 datasets