Wiki Parallel Data Extractor

Link: https://github.com/clab/wikipedia-parallel-titles
Intro: Tools for extracting parallel corpora from article titles across languages in Wikipedia

Brat

Link: http://brat.nlplab.org/introduction.html

Intro: brat is a web-based tool for text annotation; that is, for adding notes to existing text documents. brat is designed in particular for structured annotation, where the notes are not free-form text but have a fixed form that can be automatically processed and interpreted by a computer.

Wikipedia and LM

Link: http://trulymadlywordly.blogspot.sg/2011/03/creating-text-corpus-from-wikipedia.html
Intro: How to leverage Wikipedia repository to create huge LM data.

Available Data from the CommonCrawl

Link: http://statmt.org/ngrams/
Intro: Multi-language data used for training large-scale LMs crawled from CommonCrawl.

The ClueWeb09 Dataset

Link: http://www.lemurproject.org/clueweb09/

Intro: The ClueWeb09 dataset was created to support research on information retrieval and related human language technologies. It consists of about 1 billion web pages in ten languages that were collected in January and February 2009. The dataset is used by several tracks of the TREC conference.

Note: Huge corpus for LM

TalkBank

Link: http://talkbank.org/

Intro: The goal of TalkBank is to foster fundamental research in the study of human and animal communication. It will construct sample databases within each of the subfields studying communication. It will use these databases to advance the development of standards and tools for creating, sharing, searching, and commenting upon primary materials via networked computers.

EVBCorpus - English-Vietnamese Bilingual Corpus

http://code.google.com/p/evbcorpus/