NAACL 2015 papers

Link: http://naacl.org/naacl-hlt-2015/papers.html

Here is my subjective list of remarkable papers relating to MT research:

 

*** Neural Machine Translation

 

Paul Baltescu and Phil Blunsom. "Pragmatic Neural Language Modelling in Machine Translation"

 

Adrià de Gispert, Gonzalo Iglesias, Bill Byrne. "Fast and Accurate Preordering for SMT using Neural Networks"

 

*** Continuous Models for Statistical Machine Translation

 

Frédéric Blain, Fethi Bougares, Amir Hazem, Loïc Barrault, Holger Schwenk. "Continuous Adaptation to User Feedback for Statistical Machine Translation"

 

Kai Zhao, Hany Hassan, Michael Auli. "Learning Translation Models from Monolingual Continuous Representations"

 

*** Multi-language Translation 

 

Raj Dabre, Fabien Cromieres, Sadao Kurohashi, Pushpak Bhattacharyya. "Leveraging Small Multilingual Corpora for SMT Using Many Pivot Languages"

 

*** Video to Text Translation 

 

Subhashini Venugopalan, Huijuan Xu, Jeff Donahue, Marcus Rohrbach, Raymond Mooney, Kate Saenko. "Translating Videos to Natural Language Using Deep Recurrent Neural Networks" 

 

*** Others

 

Jonathan H. Clark, Chris Dyer, Alon Lavie. "Locally Non-Linear Learning for Statistical Machine Translation via Discretization and Structured Regularization"

 

Graham Neubig, Philip Arthur, Kevin Duh. "Multi-Target Machine Translation with Multi-Synchronous Context-free Grammars"

 

Aurelien Waite and Bill Byrne. "The Geometry of Statistical Machine Translation"

Other papers are also worth reading:

*** News Processing

Areej Alhothali and Jesse Hoey. "Good News or Bad News: Using Affect Control Theory to Analyze Readers' Reaction Towards News Articles"

 

 

ACL 2015 papers

Link: http://acl2015.org/accepted_papers.html

Here is my subjective list of remarkable papers relating to MT research:

*** Conventional Statistical Machine Translation

 A CONTEXT-AWARE TOPIC MODEL FOR STATISTICAL MACHINE TRANSLATION
Jinsong Su, Deyi Xiong, Yang Liu, Xianpei Han, Hongyu Lin and Junfeng Yao

NON-LINEAR LEARNING FOR STATISTICAL MACHINE TRANSLATION
Shujian Huang, Huadong Chen, Xinyu Dai and Jiajun Chen

MULTI-TASK LEARNING FOR MULTIPLE LANGUAGE TRANSLATION
Daxiang Dong, Hua Wu, Wei He, Dianhai Yu and Haifeng Wang

WHAT’S IN A DOMAIN? ANALYZING GENRE AND TOPIC DIFFERENCES IN STATISTICAL MACHINE TRANSLATION
Marlies van der Wees, Arianna Bisazza, Wouter Weerkamp and Christof Monz

*** Neural Machine Translation

ADDRESSING THE RARE WORD PROBLEM IN NEURAL MACHINE TRANSLATION
Thang Luong, Ilya Sutskever, Quoc Le, Oriol Vinyals and Wojciech Zaremba

ENCODING SOURCE LANGUAGE WITH CONVOLUTIONAL NEURAL NETWORK FOR MACHINE TRANSLATION
Fandong Meng, Zhengdong Lu, Mingxuan Wang, Hang Li, Wenbin Jiang and Qun Liu

IMPROVED NEURAL NETWORK FEATURES, ARCHITECTURE AND LEARNING FOR STATISTICAL MACHINE TRANSLATION
Hendra Setiawan, Zhongqiang Huang, Jacob Devlin, Thomas Lamar and Rabih Zbib

NON-PROJECTIVE DEPENDENCY-BASED PRE-REORDERING WITH RECURRENT NEURAL NETWORK FOR MACHINE TRANSLATION
Antonio Valerio Miceli Barone

ON USING VERY LARGE TARGET VOCABULARY FOR NEURAL MACHINE TRANSLATION
Sebastien Jean, Kyunghyun Cho, Roland Memisevic and Yoshua Bengio

CONTEXT-DEPENDENT TRANSLATION SELECTION USING CONVOLUTIONAL NEURAL NETWORK
Baotian Hu, Zhaopeng Tu, Zhengdong Lu and Hang Li

*** Machine Translation Evaluation and Quality Estimation

ONLINE MULTITASK LEARNING FOR MACHINE TRANSLATION QUALITY ESTIMATION
José G. C. de Souza, Matteo Negri, Marco Turchi and Elisa Ricci

PAIRWISE NEURAL MACHINE TRANSLATION EVALUATION
Francisco Guzmán, Shafiq Joty, Lluís Màrquez and Preslav Nakov

EVALUATING MACHINE TRANSLATION SYSTEMS WITH SECOND LANGUAGE PROFICIENCY TESTS
Takuya Matsuzaki, Akira Fujita, Naoya Todo and Noriko H. Arai

Some notes:

*** According to my observation, there are some research trends depending on data characteristics:
- very big data
- heterogeneous data
- multi-lingual data

*** And of course, deep learning research is still very hot.

Representation Learning

This research topic is very important in machine learning and a first step for all kinds of machine learning algorithms. Having a robust representation definitely plays a vital role for good performance/accuracy.

Recent progress of representation learning is using deep architecture (normally referring to deep learning). Here I collect some of very good papers worth reading:

1) Representation Learning: A Review and New Perspectives (Yoshua Bengio et al., 2014)

2) Deep Learning of Representations for Unsupervised and Transfer Learning (Yoshua Bengio et al., 2012)

3) ...

Neural Machine Translation

Scientists around the world (especially Google guys) are moving the approaches of Statistical Machine Translation (SMT) (e.g. word-based, statistical with phrase-based or hierarchical, syntax-based) to the next level, namely Neural Machine Translation.

In general, Neural Machine Translation aims to simplify the SMT approaches by taking the source as an input sequence and producing the target as an output sequence via a single, large neural networks.

Here I am trying to catch up the recent progress of Neural Machine Translation.

*** People & Group

1) LISA Lab, University of Montreal 2014 led by Prof. Yoshua Bengio

Latest Demo: http://104.131.78.120/

2) Quoc Viet Le and co. at Google (e.g. Ilya Sutskever, Nal Kalchbrenner)

3) Phil Blunsom's group at Oxford Uni.

4) Dzmitry Bahdanau at Jacobs University Bremen

5) Richard Socher at Stanford Uni.

6) Kyunghyun Cho at NYU?

7) ...

*** Notable Papers

1a) Sequence to Sequence Learning with Neural Networks (Ilya Sutskever et al., NIPS 2014)

Note:

- The core idea behind neural machine translation.

1b) Generating Sequences With Recurrent Neural Networks (Alex Graves et al., ? 2014)
- TBA

2) Neural Machine Translation by Jointly Learning to Align and Translate (Dzmitry Bahdanau et al., EMNLP 2014)

3) On the Properties of Neural Machine Translation: Encoder–Decoder Approaches (Kyunghyun Cho et al., SSST-8 2014)

4) Addressing the Rare Word Problem in Neural Machine Translation (Thang Luong et al., drafted version 2014)

5) On Using Monolingual Corpora in Neural Machine Translation (Caglar Gulcehre et al., arXiv 2015)

6) Ask Me Anything: Dynamic Memory Networks for Natural Language Processing (Richard Socher and co. at MetaMind, arXiv June 2015)
- MT result still not yet released!

7) Effective Approaches to Attention-based Neural Machine Translation (Thang Luong et al., EMNLP'15)

8) (to be updated)

In addition, some other approaches utilized neural processing to enhance the current state-of-the-art SMT framework, for example:

*** For Language Model:

1) Decoding with large-scale neural language models improves translation (Ashish et al., EMNLP 2013)

Note:

- Resulting toolkit: NPLM ver 0.3 (http://nlg.isi.edu/software/nplm/)

Comments

- It is quite hard to choose the optimized parameters (e.g. hidden layer nodes, input and output embedding dimensions) across data-sets and domains.

- In Moses, NPLM feature will slow down the decoder speed.

- It actually improves the translation performance when being used with n-gram LM features. But I am not sure whether it can completely replace n-gram LM features.

2) OxLM: A Neural Language Modelling Framework for Machine Translation (Paul Baltescu et al., The Prague Bulletin of Mathematical Linguistics 2014)

Note:

- Resulting toolkit: OxLM (https://github.com/pauldb89/oxlm)

- Moses already has this feature.

3) rwthlm - A toolkit for training neural network language models (feed-forward, recurrent, and long short-term memory neural networks). The software was written by Martin Sundermeyer.

4) (to be updated)

*** For Translation Model:

1) Fast and Robust Neural Network Joint Models for Statistical Machine Translation (Devlin et al, ACL 2014)

Note: 

- ACL 2014 best paper award. 

- Accoding to the paper, they obtained a very impressive performance for Arabic-English Translation; good performance for Chinese-English Translation (datasets: OpenMT 2012, BOLT; domains: news, web forums).

- Moses already has this feature. Basic implementation of this model is already included in Moses under the name "BilingualLM".

- NPLM can be used to train the models for this.

Comments

- Personally, I tried this model with Moses and evaluated with conversational domains (e.g. SMS, Chat, conversational telephone speech) using OpenMT'15 datasets. I obtained good (but not very impressive, 0.7-1.0 BLEU score) performance compared to basic baseline. Using this model together with other strong features did not give significantly better performance as said in the paper :(.

- Optimizing parameters for this model is an exhausted task.

2) Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation (Kyunghyun Cho et al., EMNLP 2014)

3) (to be updated)

*** For Reordering Model:

1) Advancements in Reordering Models for Statistical Machine Translation (Minwei Feng et al., ACL 2013)

2) A Neural Reordering Model for Phrase-based Translation (Peng Li et al., COLING 2014)

3) (to be updated)

NIPS paper repository

Link: http://papers.nips.cc/

Scientific Writing Assistant

SWAN: http://cs.joensuu.fi/swan/

Intro: This Swan - Scientific Writing AssistaNt - aims at helping writers with the content, not the grammar or spelling. It guides you towards known good scientific writing practices and helps your readers find your contribution. The tool was designed to help you with your writing, not to merely point out errors. Using the tool should be simple; just enter your text sections into the tool, optionally make some manual elaboration and click the "Evaluate" button. Once you have used the tool with one of your own scientific papers, do let us know how it has helped to you.

Interested Papers at EMNLP 2012

1) Tenses in SMT

D12-1026: Zhengxian Gong; Min Zhang; Chew Lim Tan; Guodong Zhou
N-gram-based Tense Models for Statistical Machine Translation

2)

D12-1041: Nan Duan; Mu Li; Ming Zhou
Forced Derivation Tree based Model Training to Statistical Machine Translation


3) ...

Interested Papers at ACL 2012

1) New NLP topic: automatic document dating
P12-1011: Nathanael Chambers
Labeling Documents with Timestamps: Learning from their Time Expressions

2)
P12-1050: Arianna Bisazza; Marcello Federico
Modified Distortion Matrices for Phrase-Based Statistical Machine Translation

3) ...

Crowdsourcing for NLP annotations

My new journal article: http://www.springerlink.com/content/n5q5n08853t06131/

Abstract

Crowd-sourcing has emerged as a new method for obtaining annotations for training models for machine learning. While many variants of this process exist, they largely differ in their methods of motivating subjects to contribute and the scale of their applications. To date, there has yet to be a study that helps the practitioner to decide what form an annotation application should take to best reach its objectives within the constraints of a project. To fill this gap, we provide a faceted analysis of crowdsourcing from a practitioner’s perspective, and show how our facets apply to existing published crowdsourced annotation applications. We then summarize how the major crowdsourcing genres fill different parts of this multi-dimensional space, which leads to our recommendations on the potential opportunities crowdsourcing offers to future annotation efforts. 

Is HCI Research Relevant to the Practice of Natural Language Processing?

http://isluis.home.xs4all.nl/publications/tabudag09_vdsluis_etal.pdf

How to Get Published (Elsevier)

http://www.elsevier.com/wps/find/authorsview.authors/trainingwebcasts

SMT without Parallel Corpora

Toward Statistical Machine Translation without Parallel Corpora (link). EMNLP 2011.

Scientific Summarization

This post aims to collect newest research papers in the literature about scientific summarization:

1) Abstract Summarization:
http://www.springerlink.com/content/q505w20k054k10w4/

2) TBA

3) Review
http://www.springerlink.com/content/4455125331140684/

A New Unsupervised Approach to Word Segmentation

See article link.

System Combination for Machine Translation

This post is to collect papers regarding to system combination problem for Machine Translation systems. (collect everything first, filter later then)

1) Felipe Sánchez-Martínez. Choosing the best machine translation system to translate a sentence by using only source-language information. In Proceedings of the 15th Annual Conference of the European Associtation for Machine Translation, p. 97-104, May 30-31, 2011, Leuven, Belgium.

2) Víctor M. Sánchez-Cartagena, Felipe Sánchez-Martínez, Juan Antonio Pérez-Ortiz. Enriching a statistical machine translation system trained on small parallel corpora with rule-based bilingual phrases. In Proceedings of the International Conference on Recent Advances in Natural Language Processing, RANLP 2011, p ?-?, September 12-14, 2011, Hissar, Bulgaria (forthcoming)


--
Cheers,
Vu

Bootstrapping for Named Entity Extraction & Recognition

This post aims to collect papers related to bootstrapping methods for Named Entity Entraction & Recognition.

1)

2)

(to be updated)

Error Analysis for Machine Translation Output

This post is to collect some papers in the literature referring to error analysis of (Statistical) Machine Translation Output. I aim to apply for English-Vietnamese translation outputs.

1) BLAST: http://www.ida.liu.se/~sarst/blast/

(Demo paper at ACL'2011: http://www.aclweb.org/anthology/P/P11/P11-4010.pdf)

2)

Maja Popovic et al. Towards Automatic Error Analysis of Machine Translation Output. (Computational Linguistics 2011)

3)

Mireia F. et al. Overcoming statistical machine translation limitations: error analysis and proposed solutions for the Catalan–Spanish language pair. LREC 2011.

4)

S. Condon. Machine Translation Errors: English and Iraqi Arabic. TALIP 2011.

5)

Maja Popović, Adrià de Gispert, Deepa Gupta, Patrik Lambert, Hermann Ney, José B. Mariño and Rafael Banchs. Morpho-syntactic Information for Automatic Error Analysis of Statistical Machine Translation Output. HLT/NAACL Workshop on Statistical Machine Translation, pages 1-6, New York, NY, June 2006.

Maja Popović and Hermann Ney. Error Analysis of Verb Inflections in Spanish Translation Output. TC-Star Workshop on Speech-to-Speech Translation, pages 99-103, Barcelona, Spain, June 2006.

David Vilar et al. Error Analysis of Statistical Machine Translation Output. LREC 2006.

Interested Articles in Computational Linguistics Journal (Vol 35 issue 1)

Towards Automatic Error Analysis of Machine Translation Output

Maja Popović, Hermann Ney

Computational Linguistics Accepted for publication: 1–50.

Posted online on 14 Jul 2011.
Abstract | PDF (366 KB) | PDF Plus (367 KB)

Information Status Distinctions and Referring Expressions: An Empirical Study of References to People in News Summaries

Advaith Siddharthan, Ani Nenkova, Kathleen McKeown

Computational Linguistics Accepted for publication: 1–54.

Posted online on 14 Jul 2011.
Abstract | PDF (296 KB) | PDF Plus (297 KB)

Half-context language models

Hinrich Schütze, Michael Walsh

Computational Linguistics Accepted for publication: 1–40.

Posted online on 14 Jul 2011.
Abstract | PDF (280 KB) | PDF Plus (281 KB)

Interested Papers at EMNLP 2011

Accepted Papers: http://conferences.inf.ed.ac.uk/emnlp2011/papers.html

1) A Weakly-supervised Approach to Argumentative Zoning of Scientific Documents
Yufan Guo, Anna Korhonen and Thierry Poibeau

2) Linear Text Segmentation Using Affinity Propagation
Anna Kazantseva and Stan Szpakowicz

3) Identifying Relations for Open Information Extraction
Anthony Fader, Stephen Soderland and Oren Etzioni

4) Active Learning with Amazon Mechanical Turk
Florian Laws, Christian Scheible and Hinrich Schütze

5) Extreme Extraction — Machine Reading in a Week
Marjorie Freedman, Lance Ramshaw, Elizabeth Boschee, Ryan Gabbard, Nicolas Ward and Ralph Weischedel

6) Discovering Relations between Noun Categories
Thahir Mohamed, Estevam Hruschka and Tom Mitchell

7) Bootstrapped Named Entity Recognition for Product Attribute Extraction
Duangmanee Putthividhya and Junling Hu

8) Predicting a Scientific Community’s Response to an Article
Dani Yogatama, Michael Heilman, Brendan O'Connor, Chris Dyer, Bryan R. Routledge and Noah A. Smith

9) Language Models for Machine Translation: Original vs. Translated Texts
Gennadi Lembersky, Noam Ordan and Shuly Wintner

10) Twitter Catches The Flu: Detecting Influenza Epidemics using Twitter
Eiji Aramaki, Sachiko Maskawa and Mizuki Morita

11) Rumor has it: Identifying Misinformation in Microblogs
Vahed Qazvinian, Emily Rosengren, Dragomir R. Radev and Qiaozhu Mei

Interested Papers at SIGIR 2011

SIGIR 2011 accepted papers link: http://www.sigir2011.org/papers.htm

My Interested Papers:
1) Summarizing the Differences in Multilingual News
Xiaojun Wan, Houping Jia

2) Multifaceted Toponym Recognition for Streaming News
Michael Lieberman, Hanan Samet

3) Toward Social Context Summarization For Web Documents
Zi Yang, cai keke, Jie Tang, Li Zhang, Zhong Su, Juanzi Li

4) Evolutionary Timeline Summarization: a Balanced Optimization Framework via Iterative Substitution
Rui Yan, Xiaojun Wan, Jahna Otterbacher, Liang Kong, Yan Zhang, Xiaoming Li

5) The Economics in Interactive Information Retrieval
Leif Azzopardi

6) Composite Hashing with Multiple Information Sources
Dan Zhang, Fei Wang, Luo Si

7) Inverted Indexes for Phrases and Strings
Manish Patil, Sharma V. Thankachan, Rahul Shah, Wing-Kai Hon, Jeffrey Vitter, Sabrina Chandrasekaran

8) Multimedia Answering: Enriching Text QA with Media Information
Liqiang Nie, Meng Wang, Zha Zhengjun, Li Guangda, Tat Seng Chua

9) SCENE : A Scalable Two-Stage Personalized News Recommendation System
Lei Li, Dingding Wang, Tao Li

10) Ranking Related News Predictions
Nattiya Kanhabua, Roi Blanco, Michael Matthews

--
Cheers,
Vu