Datasets
Dataset
CulturaX
by uonlp
CulturaX Cleaned, Enormous, and Public: The Multilingual Fuel to Democratize Large Language Models for 167 Languages Dataset Summary We present CulturaX, a substantial multilingual dataset with 6. 3 trillion tokens in 167 languages, tailored for large langu…
No licensetext-generation 12593 6711B < n < 10B rows
published 4 Sept 2023
Listed inAwesome-Odia-AI
Preview
The dataset does not exist, or is not accessible without authentication (private or gated). Please check the spelling of the dataset name or retry with authentication.
Cite this
No licensetextCite
Tags
task_categories:text-generationtask_categories:fill-masktask_ids:language-modelingtask_ids:masked-language-modelingannotations_creators:no-annotationlanguage_creators:foundmultilinguality:multilingualsource_datasets:originallanguage:aflanguage:alslanguage:amlanguage:anlanguage:arlanguage:arzlanguage:aslanguage:astlanguage:avlanguage:azlanguage:azblanguage:balanguage:barlanguage:bcllanguage:belanguage:bglanguage:bhlanguage:bnlanguage:bolanguage:bpylanguage:brlanguage:bslanguage:bxrlanguage:calanguage:cbklanguage:celanguage:ceblanguage:ckblanguage:cslanguage:cvlanguage:cylanguage:da