Japanese-English Parallel Datasets

Link: http://phontron.com/japanese-translation-data.php

YALIGN

Link: http://yalign.machinalis.com/

Intro: Statistical Machine Translation relies on parallel corpora for training translation models. However these corpora are limited and take time to create. Yalign is designed to automate this process by finding sentences that are close translation matches from comparable corpora. This opens up avenues for harvesting parallel corpora from sources like translated documents and the web.

Champollion Tool Kit - Text Sentence Aligner

Link: http://champollion.sourceforge.net/

Intro: Built around LDC's champollion sentence aligner kernel, Champollion Tool Kit (CTK) aims to providing ready-to-use parallel text sentence alignment tools for as many language pairs as possible.

Champollion depends heavily on lexical information, but uses sentence length information as well. A translation lexicon is required. Past experiments indicate that champollion's performance improves as the translation lexicon become larger.

WIT3 - Web Inventory of Transcribed and Translated Talks

Link: https://wit3.fbk.eu/

Intro: WIT3 - acronym for Web Inventory of Transcribed and Translated Talks - is a ready-to-use version for research purposes of the multilingual transcriptions of TED talks. 

Since 2007, the TED Conference has been posting on its website all video recordings of its talks, English subtitles and their translations in more than 80 languages. In order to make this collection of talks more effectively usable by the research community, the original textual contents are redistributed here, together with MT benchmarks and processing tools.

Parallel Text Mining for SMT

Problem: given a relatively large collection of parallel texts and a state-of-the-art SMT system, how to incrementally & automatically mine the parallel texts available on the Web. The newly added texts should ensure to improve the current SMT system.

Papers related:

1) Large Scale Parallel Document Mining for Machine Translation. COLING 2010. Link.
2) TBA

SMT without Parallel Corpora

Toward Statistical Machine Translation without Parallel Corpora (link). EMNLP 2011.

OPUS - The Open Parallel Corpus

http://opus.lingfil.uu.se/

Intro

OPUS is a growing collection of translated texts from the web. In the OPUS project we try to convert and align free online data, to add linguistic annotation, and to provide the community with a publicly available parallel corpus. OPUS is based on open source products and the corpus is also delivered as an open content package. We used several tools to compile the current collection. All pre-processing is done automatically. No manual corrections have been carried out.

It's amazing. It's free!

--
Cheers,
Vu

YouAlign - Online document alignment solution

http://www.youalign.com/

"Welcome to YouAlign, your online document alignment solution. No software to purchase, no software to install. With YouAlign you can quickly and easily create bitexts from your archived documents. A YouAlign bitext contains a document and its translation aligned at the sentence level. YouAlign generates TMX files that can be loaded into your translation memory. YouAlign can also generate HTML files that you can publish on the Internet, or use with a full-text search engine to search for terminology and phraseology in context.

YouAlign is powered by the AlignFactory engine, which supports all kinds of formats, including Microsoft Word, Excel and PowerPoint, PDF, HTML, XML, Corel WordPerfect, RTF, Lotus WordPro and plain text."

Searchable Parallel Corpora

1. CABAL: Un concordancier en ligne pour la linguistique contrastive

http://cabal.rezo.net/ (University of Poitiers)

English, French

Environ 200 articles sont actuellement en ligne (soit environ 400 000 mots). La majorité sont issus du Monde diplomatique et datés de 1998 à décembre 2003.

2. The CLUVI corpus:

http://sli.uvigo.es/CLUVI/index_en.html

English, French, Spanish, Galician,

Corpus: UNESCO Corpus of English-Galician-French-Spanish scientific-technical divulgation

3. German(-English) parallel corpora (Europarl and German News)

http://corpus.leeds.ac.uk/paraquery.html

English, German

4. WebTCE (Translation Corpus Explorer)

http://khnt.hit.uib.no/webtce.htm

English, German, French, Spanish, Norwegian, Danish

5. EVROKORPUS Parallel corpora

http://evrokorpus.gov.si/index.php?jezik=angl

223 million words. English, French, German, Italian, Slovene and Spanish. Searches must involve Slovene and one other language.

6. TERMACOR terminology and corpus

http://evrokorpus.gov.si/k2/index.php?jezik=angl

98 million words in 22 European Languages. EU Commission data.

7. COMPARA Portuguese-English parallel corpus

http://www.linguateca.pt/COMPARA/

Three million words.

Portuguese, English

8. Termsearch

http://www.termsearch.info/ or a faster interface at:

http://www.bible-study-in-geneva.info/termsearch/

English, French, Russian

Major international treaties, conventions, agreements, etc. 792 documents.

9. English-Inuktitut Parallel Corpus

http://www.inuktitutcomputing.ca/NunavutHansard/en/

3.5 million words (of English), 1.5 million words of Inuktitut

English, Inuktitut (an Inuit Language of North-Eastern Canada)

10. English-Russian Parallel Corpus

http://ruscorpora.ru/search-para.html

English, Russian, (some German?)

Interface only in Russian.

About 9 million words

11. http://WeBiText.com

12. http://www.linguee.de/

13. http://corpus.consumer.es/corpus/

14. OPUS:
http://www.let.rug.nl/tiedeman/OPUS/
http://www.let.rug.nl/tiedeman/OPUS/bin/opuscqp.pl

15. LinearB

http://linearb.co.uk/

16. MyMemories

http://mymemory.translated.net/

17. Natura corpora

http://linguateca.di.uminho.pt/nat/nat.pl

18. Compara

http://www.linguateca.pt/COMPARA/Welcome.html

19. CLUVI

http://sli.uvigo.es/CLUVI/

20. English-Chinese

http://score.crpp.nie.edu.sg/cgi-bin/babel/paraconc.pl

21. TransSearch

http://www.tsrali.com/

22. English-Vietnamese
http://hellochao.com/
http://linkdict.com/Bilingual/

23. WebLitera - free international online book library

http://weblitera.com/

(to be updated)