Tay Nung dictionary

Link: https://sites.google.com/site/tndict/home

Intro: Thanks to some guys on facebook of VNese NLP group, I just know about this. One of interesting problems is to preserve and develop the local regional languages (e.g. Tay Nung in the above link) in parallel with official Vietnamese language.

P.S.: I will be back one day about this issue.

Hansard corpus

Link: http://www.hansard-corpus.org/

Intro: This Hansard corpus (or collection of texts) contains nearly every speech given in the British Parliament from 1803-2005, and it allows you to search these speeches (including semantically-based searches) in ways that are not possible with any other resource.

OLAC - Open Language Archives Community

Link: http://www.language-archives.org/

Intro: OLAC, the Open Language Archives Community, is an international partnership of institutions and individuals who are creating a worldwide virtual library of language resources by: (i) developing consensus on best current practice for the digital archiving of language resources, and (ii) developing a network of interoperating repositories and services for housing and accessing such resources.

NAACL 2015 papers

Link: http://naacl.org/naacl-hlt-2015/papers.html

Here is my subjective list of remarkable papers relating to MT research:

 

*** Neural Machine Translation

 

Paul Baltescu and Phil Blunsom. "Pragmatic Neural Language Modelling in Machine Translation"

 

Adrià de Gispert, Gonzalo Iglesias, Bill Byrne. "Fast and Accurate Preordering for SMT using Neural Networks"

 

*** Continuous Models for Statistical Machine Translation

 

Frédéric Blain, Fethi Bougares, Amir Hazem, Loïc Barrault, Holger Schwenk. "Continuous Adaptation to User Feedback for Statistical Machine Translation"

 

Kai Zhao, Hany Hassan, Michael Auli. "Learning Translation Models from Monolingual Continuous Representations"

 

*** Multi-language Translation 

 

Raj Dabre, Fabien Cromieres, Sadao Kurohashi, Pushpak Bhattacharyya. "Leveraging Small Multilingual Corpora for SMT Using Many Pivot Languages"

 

*** Video to Text Translation 

 

Subhashini Venugopalan, Huijuan Xu, Jeff Donahue, Marcus Rohrbach, Raymond Mooney, Kate Saenko. "Translating Videos to Natural Language Using Deep Recurrent Neural Networks" 

 

*** Others

 

Jonathan H. Clark, Chris Dyer, Alon Lavie. "Locally Non-Linear Learning for Statistical Machine Translation via Discretization and Structured Regularization"

 

Graham Neubig, Philip Arthur, Kevin Duh. "Multi-Target Machine Translation with Multi-Synchronous Context-free Grammars"

 

Aurelien Waite and Bill Byrne. "The Geometry of Statistical Machine Translation"

Other papers are also worth reading:

*** News Processing

Areej Alhothali and Jesse Hoey. "Good News or Bad News: Using Affect Control Theory to Analyze Readers' Reaction Towards News Articles"

 

 

ACL 2015 papers

Link: http://acl2015.org/accepted_papers.html

Here is my subjective list of remarkable papers relating to MT research:

*** Conventional Statistical Machine Translation

 A CONTEXT-AWARE TOPIC MODEL FOR STATISTICAL MACHINE TRANSLATION
Jinsong Su, Deyi Xiong, Yang Liu, Xianpei Han, Hongyu Lin and Junfeng Yao

NON-LINEAR LEARNING FOR STATISTICAL MACHINE TRANSLATION
Shujian Huang, Huadong Chen, Xinyu Dai and Jiajun Chen

MULTI-TASK LEARNING FOR MULTIPLE LANGUAGE TRANSLATION
Daxiang Dong, Hua Wu, Wei He, Dianhai Yu and Haifeng Wang

WHAT’S IN A DOMAIN? ANALYZING GENRE AND TOPIC DIFFERENCES IN STATISTICAL MACHINE TRANSLATION
Marlies van der Wees, Arianna Bisazza, Wouter Weerkamp and Christof Monz

*** Neural Machine Translation

ADDRESSING THE RARE WORD PROBLEM IN NEURAL MACHINE TRANSLATION
Thang Luong, Ilya Sutskever, Quoc Le, Oriol Vinyals and Wojciech Zaremba

ENCODING SOURCE LANGUAGE WITH CONVOLUTIONAL NEURAL NETWORK FOR MACHINE TRANSLATION
Fandong Meng, Zhengdong Lu, Mingxuan Wang, Hang Li, Wenbin Jiang and Qun Liu

IMPROVED NEURAL NETWORK FEATURES, ARCHITECTURE AND LEARNING FOR STATISTICAL MACHINE TRANSLATION
Hendra Setiawan, Zhongqiang Huang, Jacob Devlin, Thomas Lamar and Rabih Zbib

NON-PROJECTIVE DEPENDENCY-BASED PRE-REORDERING WITH RECURRENT NEURAL NETWORK FOR MACHINE TRANSLATION
Antonio Valerio Miceli Barone

ON USING VERY LARGE TARGET VOCABULARY FOR NEURAL MACHINE TRANSLATION
Sebastien Jean, Kyunghyun Cho, Roland Memisevic and Yoshua Bengio

CONTEXT-DEPENDENT TRANSLATION SELECTION USING CONVOLUTIONAL NEURAL NETWORK
Baotian Hu, Zhaopeng Tu, Zhengdong Lu and Hang Li

*** Machine Translation Evaluation and Quality Estimation

ONLINE MULTITASK LEARNING FOR MACHINE TRANSLATION QUALITY ESTIMATION
José G. C. de Souza, Matteo Negri, Marco Turchi and Elisa Ricci

PAIRWISE NEURAL MACHINE TRANSLATION EVALUATION
Francisco Guzmán, Shafiq Joty, Lluís Màrquez and Preslav Nakov

EVALUATING MACHINE TRANSLATION SYSTEMS WITH SECOND LANGUAGE PROFICIENCY TESTS
Takuya Matsuzaki, Akira Fujita, Naoya Todo and Noriko H. Arai

Some notes:

*** According to my observation, there are some research trends depending on data characteristics:
- very big data
- heterogeneous data
- multi-lingual data

*** And of course, deep learning research is still very hot.

Deep Learning for NLP

1) CSLM: Continuous Space Language Model toolkit
Link: http://www-lium.univ-lemans.fr/cslm/

Intro: CSLM toolkit is open-source software which implements the so-called continuous space language model.

The basic idea of this approach is to project the word indices onto a continuous space and to use a probability estimator operating on this space. Since the resulting probability functions are smooth functions of the word representation, better generalization to unknown events can be expected. A neural network can be used to simultaneously learn the projection of the words onto the continuous space and to estimate the n-gram probabilities. This is still a n-gram approach, but the LM probabilities are interpolated for any possible context of length n-1 instead of backing-off to shorter contexts. This approach was successfully used in large vocabulary continuous speech recognition and in phrase-based SMT systems.

2) Recurrent Neural Network LM (RNNLM)

Link: http://rnnlm.org/

Intro: Neural network based language models are nowdays among the most successful techniques for statistical language modeling. They can be easily applied in wide range of tasks, including automatic speech recognition and machine translation, and provide significant improvements over classic backoff n-gram models. The 'rnnlm' toolkit can be used to train, evaluate and use such models.

3) word2vec

Link: https://github.com/dav/word2vec or http://deeplearning4j.org/word2vec.html

Intro: This tool provides an efficient implementation of the continuous bag-of-words and skip-gram architectures for computing vector representations of words. These representations can be subsequently used in many natural language processing applications and for further research.

4) Long Short Term Memory (LSTM)
Link: http://www.bioinf.jku.at/software/lstm/
Intro: Software for the state of the art recurrent neural network. Long Short-Term Memory Software

5) DL4J Deep Learning for Java
Link: http://deeplearning4j.org/
Intro: Deeplearning4j is the first commercial-grade, open-source, distributed deep-learning library written for Java and Scala. Integrated with Hadoop and Spark, DL4J is designed to be used in business environments, rather than as a research tool. It aims to be cutting-edge plug and play, more convention than configuration, which allows for fast prototyping for non-researchers.

6) CURRENNT
Link: http://sourceforge.net/projects/currennt/
Intro: CUDA-enabled machine learning library for recurrent neural networks which can run both on Windows or Linux machines with CUDA-supported capability. CURRENNT is a machine learning library for Recurrent Neural Networks (RNNs) which uses NVIDIA graphics cards to accelerate the computations. The library implements uni- and bidirectional Long Short-Term Memory (LSTM) architectures and supports deep networks as well as very large data sets that do not fit into main memory.

7) ...

*** Deep learning materials

- For NLP

a) http://www.socher.org/index.php/DeepLearningTutorial/DeepLearningTutorial

b) http://cl.naist.jp/~kevinduh/a/deep2014/

c) http://ronan.collobert.com/pub/matos/2009_tutorial_nips.pdf

- For general background

a) http://cl.naist.jp/~kevinduh/notes/duh13deepadvances.pdf

b) http://www.trivedigaurav.com/blog/quoc-les-lectures-on-deep-learning/

c) http://deeplearning.net/reading-list/
d) (new book, 2014, draft  version) http://www.iro.umontreal.ca/~bengioy/dlbook/
e) http://people.idsia.ch/~juergen/deep-learning-overview.html 

- ...

(to be updated)

Statistical Methods in Language and Linguistic Research

Link: https://www.equinoxpub.com/equinox/books/showbook.asp?bkid=348&keyword=

Very large-scale corpus (COCA)

COCA (Corpus of Contemporary American English)

Link: http://corpus.byu.edu/coca/

Intro: The Corpus of Contemporary American English (COCA) is the largest freely-available corpus of English, and the only large and balanced corpus of American English. The corpus was created by Mark Davies of Brigham Young University, and it is used by tens of thousands of users every month (linguists, teachers, translators, and other researchers). COCA is also related to other large corpora that we have created.

The corpus contains more than 450 million words of text and is equally divided among spoken, fiction, popular magazines, newspapers, and academic texts. It includes 20 million words each year from 1990-2012 and the corpus is also updated regularly (the most recent texts are from Summer 2012). Because of its design, it is perhaps the only corpus of English that is suitable for looking at current, ongoing changes in the language (see the 2011 article in Literary and Linguistic Computing).

***
Ngram corpus from COCA:
1) COCA Ngrams:
Link: see this post.

2) COHA Ngrams:
Link: http://www.ngrams.info/download_coha.asp

Intro: The Corpus of Historical American English (COHA) contain 400 million words of text from 1810-2009, and all of the n-grams from the corpus can be freely downloaded. They contain all n-grams that occur at least three times total in the corpus, and you can see the frequency of each of these n-grams in each decade from the 1810s-2000s. This data can be used offline to carry out powerful searches on a wide range of phenomena in the history of American English.

---------------------------------
My thoughts:

- I have been developing a language-generic n-gram-based spell checking tool. So, this ngram corpus will be very beneficial.

- Other tasks in English NLP may need this corpus.

Interested Papers at EMNLP 2012

1) Tenses in SMT

D12-1026: Zhengxian Gong; Min Zhang; Chew Lim Tan; Guodong Zhou
N-gram-based Tense Models for Statistical Machine Translation

2)

D12-1041: Nan Duan; Mu Li; Ming Zhou
Forced Derivation Tree based Model Training to Statistical Machine Translation


3) ...

Interested Papers at ACL 2012

1) New NLP topic: automatic document dating
P12-1011: Nathanael Chambers
Labeling Documents with Timestamps: Learning from their Time Expressions

2)
P12-1050: Arianna Bisazza; Marcello Federico
Modified Distortion Matrices for Phrase-Based Statistical Machine Translation

3) ...

Linguistic Resources and Tools at the JRC

Link: http://langtech.jrc.ec.europa.eu/JRC_Resources.html

Bookmarks for Corpus-based Linguists

http://www.uow.edu.au/~dlee/CBLLinks.htm

Interested Articles in Computational Linguistics Journal (Vol 35 issue 1)

Towards Automatic Error Analysis of Machine Translation Output

Maja Popović, Hermann Ney

Computational Linguistics Accepted for publication: 1–50.

Posted online on 14 Jul 2011.
Abstract | PDF (366 KB) | PDF Plus (367 KB)

Information Status Distinctions and Referring Expressions: An Empirical Study of References to People in News Summaries

Advaith Siddharthan, Ani Nenkova, Kathleen McKeown

Computational Linguistics Accepted for publication: 1–54.

Posted online on 14 Jul 2011.
Abstract | PDF (296 KB) | PDF Plus (297 KB)

Half-context language models

Hinrich Schütze, Michael Walsh

Computational Linguistics Accepted for publication: 1–40.

Posted online on 14 Jul 2011.
Abstract | PDF (280 KB) | PDF Plus (281 KB)

Scientext corpus

http://scientext.msh-alpes.fr

Scientext is a new, on-line French and English corpus of scientific texts. The corpus includes 4.8 million running tokens in French, 13 million words of research articles in English (medicine and biology), and an English-language sub-corpus of French undergraduate students’ texts (1,1 million words). The corpus is organized to facilitate the linguistic study of authorial position and reasoning in scientific articles through phraseology and lexico-grammatical markers linked to causality.

MultiTree - A Digital Library of Language Relationships

http://multitree.org/

(look quite beautiful! :D)

SIGNLL Links

http://ifarm.nl/signll/links/

--
Cheers,
Vu

MyMemory

http://mymemory.translated.net/ --> translations between various languages.

It's so great!

--
Cheers,
Vu

NLP/Computational Linguistics Anthology

There are very useful resources that support for research in the field of Computational Linguistics and Natural Language Processing. Some of them are currently available on the web.

* ACL Anthology: http://www.aclweb.org/anthology-new/

- archive papers of major conferences or journals such as: ACL, NAACL, EMNLP, COLING, Journal of computational Linguistics, ...

* The ACL Anthology Network: http://belobog.si.umich.edu/clair/anthology/index.cgi

- very helpful network built based on data archived from ACL Anthology. It plays a role as a social network that unveils relationships between papers and authors.

* ACL Anthology Reference Corpus (ACL ARC): http://acl-arc.comp.nus.edu.sg/

- a corpus recently built by some leading researchers around the world aims at boosting the research in scientific domain.

* ACL Anthology SearchBench: http://aclasb.dfki.de/

--
Cheers,
Vu