# Buy low-resource language data for AI training URL: https://sellmydatatoai.net/buy-ai-training-data/low-resource-language-data Low-resource language data is text, and text paired with translations, in languages with little usable material online, licensed to train and evaluate models beyond the major languages. Google says only a few of more than 7,000 spoken languages are well represented online. The Linguistic Data Consortium sells LORELEI packs, such as 7.2 million words of Yoruba, with fees shown only after login. ## Key takeaways - AI teams use low-resource language data for (1) pre-training and fine-tuning in under-served languages, (2) machine translation into and out of English and (3) multilingual evaluation sets. - Usually priced per corpus or per million words. Mostly not published. - Common delivery formats: UTF-8 text or JSONL with ISO 639-3 language codes, sentence-aligned parallel text in TMX or TSV and annotation files for entities and semantics. - Check before you license low-resource language data: (1) US copyright: a written work made for hire agreement or assignment from each translator, (2) rights in the source texts behind any translation and (3) written by native speakers, not machine translated. - Every seller we shortlist passes 8 vetting checks before you see a sample. ## Buyer's spec - Used for: Pre-training and fine-tuning in under-served languages; Machine translation into and out of English; Multilingual evaluation sets; Named entity and safety classifiers in local languages - Formats: UTF-8 text or JSONL with ISO 639-3 language codes; Sentence-aligned parallel text in TMX or TSV; Annotation files for entities and semantics; Dialect, region and script metadata - Usually priced: Per corpus or per million words. Mostly not published - Check before you sign: US copyright: a written work made for hire agreement or assignment from each translator; Rights in the source texts behind any translation; Written by native speakers, not machine translated; Language and script labels verified by native reviewers; Community and tribal consent for Indigenous language material; US: CCPA and other state privacy laws for personal information; UK and EU: UK GDPR or EU GDPR for personal data ## What is low-resource language data? Low-resource language data is text in a language with too little digital material to train a strong model, often with English translations attached. The label is relative. Meta counts 150 low-resource languages among the 200 its No Language Left Behind model translates. This page covers buying that data. Our [multilingual text data](https://sellmydatatoai.net/buy-ai-training-data/multilingual-text-data) page covers widely spoken languages and translation memories in general. ## Why is low-resource language data hard to find? Low-resource language data is hard to find because the open web covers few languages well. Google wrote in November 2022, launching its 1,000 Languages Initiative, that only a few of more than 7,000 spoken languages are well represented online. Meta states that reliable translation data is not available for many languages. English alone makes up about 42.6% of pages in Common Crawl's CC-MAIN-2026-43 crawl. A corpus scraped from the web for a small language can be thin, repetitive or machine translated. ## Who sells low-resource language data? Low-resource language data is sold by research distributors and training-data companies, and some is commissioned from native speakers. | Seller | What it publishes | Price | | --- | --- | --- | | Linguistic Data Consortium | LORELEI packs, such as 7.2 million words of Yoruba with 810,000 words translated into English | Shown only after login | | Appen | Off-the-shelf datasets, such as a 66,000-word Swahili (Kenya) pronunciation dictionary | Quote on request | | Defined.ai | A crowd it says spans more than 500 languages, dialects and locales | Not published | | LXT | Data across more than 1,000 language locales, by its own count | Not published | Coverage claims are self-reported, so ask each seller for word counts per language and for who wrote the text. See the [Linguistic Data Consortium](https://sellmydatatoai.net/data-buyers/linguistic-data-consortium), [Appen](https://sellmydatatoai.net/data-buyers/appen) and [LXT](https://sellmydatatoai.net/data-buyers/lxt) profiles. ## How much does low-resource language data cost? Low-resource language data has no public per-word rate from the sellers we checked. LDC shows fees for its LORELEI packs, including the Wolof pack of more than 225,000 words translated into English, only to logged-in users, and Appen's catalog adds datasets to a quote list rather than showing prices. For commissioned text, ask for a quote that separates translation, native review and annotation, so you can compare sellers line by line. Our guide to [AI training data pricing models](https://sellmydatatoai.net/guides/ai-training-data-pricing-models) explains per-unit and flat-fee deals. ## What rights checks apply to translated text? Translated text carries two sets of rights: those in the source text and those in the translation. 17 USC 101 lists a translation as a derivative work, so the source owner's rights survive, and it names translations among the nine kinds of commissioned work that can be a work made for hire when both sides sign a written agreement. Without that agreement or a written assignment, a freelance translator keeps the copyright. Ask for the contract behind each translator's work and for the license covering the source texts. Personal information in correspondence, chat or social posts still falls under the CCPA and other state privacy laws, whatever the language. ## How should you handle Indigenous and community language data? Indigenous and community language data needs the consent of the community, not just of the person who wrote it down. The CARE Principles for Indigenous Data Governance, published by the Global Indigenous Data Alliance, set collective benefit, authority to control, responsibility and ethics as the tests for using Indigenous data. For Native American and Alaska Native languages, ask whether the tribal government or language program concerned approved the license. Tribes are sovereign nations, and some set their own rules for research and data. ## What should you check in a low-resource sample? A low-resource sample should be checked by native speakers as well as by automatic tools, in every language the seller lists. - Native speaker review of a random sample in every language - Language identification run across the full set to catch mislabeled text - Share of machine-translated text disclosed and labeled - Script and orthography documented where a language uses more than one - Domain spread across news, everyday writing and specialist text shown in the manifest Our guide on [how to evaluate a dataset sample](https://sellmydatatoai.net/guides/how-to-evaluate-dataset-samples) covers the tests. Meta's FLORES-200, a human-translated benchmark covering 200 languages, is one way to measure gains after training. ## If you hold UK or EU data If text comes from the UK or EU, UK GDPR or EU GDPR applies to any personal data in it. Providers of general-purpose AI models in the EU must also publish a summary of training content under the EU AI Act, which our [EU AI Act guide](https://sellmydatatoai.net/guides/eu-ai-act-training-data-transparency) explains. ## How do you source vetted low-resource data? You source vetted low-resource data by naming the languages, dialects, scripts and word counts you need. Every seller we shortlist passes our [eight-point vetting standard](https://sellmydatatoai.net/how-we-vet). [Book a free consultation](https://sellmydatatoai.net/book-a-consultation?side=buyer) or [submit a data requirement](https://sellmydatatoai.net/request-data). ## Frequently asked questions ### What counts as a low-resource language? A low-resource language is one with too little digital text, translation data or annotation to train a strong model, whatever the number of speakers. Meta's No Language Left Behind project counts 150 low-resource languages among the 200 its model translates, and Google says only a few of more than 7,000 spoken languages are well represented online. ### Can I license LDC data for commercial training? LDC licenses corpora to members and non-members under user agreements, and says for-profit members keep commercial rights to data acquired during a for-profit membership. Each corpus page names its license and shows fees after login. Read the user agreement for the specific corpus you want, and take independent legal advice before training a commercial model. ### Do freelance translators own the translations they produce? Freelance translators own the copyright in their translations unless they sign a written assignment or a work made for hire agreement. US copyright law names translations among nine kinds of commissioned work that can be made for hire, but only with a written agreement signed by both sides. Ask sellers for the translator contracts behind any corpus. ### Is machine-translated text acceptable in a low-resource corpus? Machine-translated text is a weak substitute in a low-resource corpus, because a model trained on it learns the translation system's errors rather than native usage. Meta notes that reliable translation data is missing for many languages, which limits those systems too. Ask sellers to label any machine-translated share and have native speakers review a sample before you agree a price. ## Sources - [No Language Left Behind](https://ai.meta.com/research/no-language-left-behind/), Meta AI, accessed November 2026 - [3 ways AI is scaling helpful technologies worldwide](https://blog.google/technology/ai/ways-ai-is-scaling-helpful/), Google, accessed November 2026 - [Statistics of Common Crawl monthly archives: distribution of languages](https://commoncrawl.github.io/cc-crawl-statistics/plots/languages), Common Crawl, accessed November 2026 - [LORELEI Yoruba Representative Language Pack (LDC2024T10)](https://catalog.ldc.upenn.edu/LDC2024T10), Linguistic Data Consortium, accessed November 2026 - [LORELEI Wolof Representative Language Pack (LDC2022T03)](https://catalog.ldc.upenn.edu/LDC2022T03), Linguistic Data Consortium, accessed November 2026 - [Membership benefits](https://www.ldc.upenn.edu/node/34), Linguistic Data Consortium, accessed November 2026 - [Off-the-shelf datasets: Chinese or Swahili](https://datasets.appen.com/language-chinese-or-swahili), Appen, accessed November 2026 - [Defined.ai homepage](https://defined.ai/), Defined.ai, accessed November 2026 - [LXT homepage](https://www.lxt.ai/), LXT, accessed November 2026 - [17 U.S. Code section 101: definitions](https://www.law.cornell.edu/uscode/text/17/101), Legal Information Institute, Cornell Law School, accessed November 2026 - [Works Made for Hire (Circular 30)](https://www.copyright.gov/circs/circ30.pdf), US Copyright Office, accessed November 2026 - [CARE Principles for Indigenous Data Governance](https://www.gida-global.org/care), Global Indigenous Data Alliance, accessed November 2026 - [California Civil Code section 1798.140 (CCPA definitions)](https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?sectionNum=1798.140.&lawCode=CIV), California Legislative Information, accessed November 2026 --- Book a free consultation: https://sellmydatatoai.net/book-a-consultation