Deep Learning Frameworks

There are a lot of deep learning frameworks out there, depending on your usage purpose or the familiarity of your programming languages or working tasks, here I only summarize ones that I am familiar with:

1) TensorFlow by Google (released on 10 Nov 2015): http://tensorflow.org/

Comment: 

2) VELES by Samsung (released on 11 Nov 2015): https://velesnet.ml/

Comment:

3) cnn (lightweight and very fast neural network library in C++, also in Python, works both on Windows and Linux machines): https://github.com/kaishengyao/cnn

Comment: cnn has been proven to be much faster than Theano both with and without GPU. Also, it offers the advantage for software production of neural network models since it has been developing in C++ and more importantly, it supports both Windows and Linux platforms. 

4) to be updated

Torch vs. Theano vs. Caffe

Link: http://fastml.com/torch-vs-theano/

Here is my summary:

- Torch and Theano are better to be used for research purpose on deep learning (DL) whereas Caffe is more scaled for DL application development.

- Torch and Theano are competitive in terms of speech and performance via different benchmarks. Hence, choosing one of them depends the ease of use from users.

(to be updated)

WaCky - The Web-As-Corpus Kool Yinitiative

Link: http://wacky.sslmit.unibo.it/doku.php?id=tools

Some useful C++ programming libaries

1) http://partow.net/programming/
2)

Topic modeling packages

1) MALLET: http://mallet.cs.umass.edu/
2) TMT: http://nlp.stanford.edu/software/tmt/tmt-0.4/
3) GENSIM: http://radimrehurek.com/gensim/index.html

SENNA toolkit

Intro: SENNA is a software distributed under a non-commercial license, which outputs a host of Natural Language Processing (NLP) predictions: part-of-speech (POS) tags, chunking (CHK), name entity recognition (NER), semantic role labeling (SRL) and syntactic parsing (PSG).

SENNA is fast because it uses a simple architecture, self-contained because it does not rely on the output of existing NLP system, and accurate because it offers state-of-the-art or near state-of-the-art performance.

Link: http://ronan.collobert.com/senna/

String Toolkit for Text Processing

Link: http://www.codeproject.com/Articles/23198/C-String-Toolkit-StrTk-Tokenizer

It can be used as lite version of Boost.

ICU - International Components for Unicode

Link: http://site.icu-project.org/

Intro: ICU is a mature, widely used set of C/C++ and Java libraries providing Unicode and Globalization support for software applications. ICU is widely portable and gives applications the same results on all platforms and between C/C++ and Java software.

Here are a few highlights of the services provided by ICU:

Code Page Conversion: Convert text data to or from Unicode and nearly any other character set or encoding. ICU's conversion tables are based on charset data collected by IBM over the course of many decades, and is the most complete available anywhere.

Collation: Compare strings according to the conventions and standards of a particular language, region or country. ICU's collation is based on the Unicode Collation Algorithm plus locale-specific comparison rules from the Common Locale Data Repository, a comprehensive source for this type of data.

Formatting: Format numbers, dates, times and currency amounts according the conventions of a chosen locale. This includes translating month and day names into the selected language, choosing appropriate abbreviations, ordering fields correctly, etc. This data also comes from the Common Locale Data Repository.

Time Calculations: Multiple types of calendars are provided beyond the traditional Gregorian calendar. A thorough set of timezone calculation APIs are provided.

Unicode Support: ICU closely tracks the Unicode standard, providing easy access to all of the many Unicode character properties, Unicode Normalization, Case Folding and other fundamental operations as specified by the Unicode Standard.

Regular Expression: ICU's regular expressions fully support Unicode while providing very competitive performance.

Bidi: support for handling text containing a mixture of left to right (English) and right to left (Arabic or Hebrew) data.

Text Boundaries: Locate the positions of words, sentences, paragraphs within a range of text, or identify locations that would be suitable for line wrapping when displaying the text.

NiuTrans: A Statistical Machine Translation System

Link: http://www.nlplab.com/NiuPlan/NiuTrans.html#introduction

Intro: NiuTrans is an open-source statistical machine translation system developed by the Natural Language Processing Group at Northeastern University, China. The NiuTrans system is fully developed in C++ language. So it runs fast and uses less memory. Currently it has already supported phrase-based, hierarchical phrase-based and syntax-based (string-to-tree, tree-to-string and tree-to-tree) models for research-oriented studies.

Paper (ACL 2012 Demo): http://www.nlplab.com/NiuPlan/paper/ACL2012-NiuTrans.pdf

Docent - Document-level SMT

Link: https://github.com/chardmeier/docent/wiki

Intro: Docent is a decoder for phrase-based Statistical Machine Translation (SMT). Unlike most existing SMT decoders, it treats complete documents, rather than single sentences, as translation units and permits the inclusion of features with cross-sentence dependencies to facilitate the development of discourse-level models for SMT. Docent implements the local search decoding approach described by Hardmeier et al. (EMNLP 2012).
Paper: https://aclweb.org/anthology-new/D/D12/D12-1108.pdf

Intel® Threading Building Blocks (Intel® TBB)

Link: http://threadingbuildingblocks.org/

Intro: Intel® Threading Building Blocks (Intel® TBB) offers a rich and complete approach to expressing parallelism in a C++ program. It is a library that helps you take advantage of multi-core processor performance without having to be a threading expert. Intel TBB is not just a threads-replacement library. It represents a higher-level, task-based parallelism that abstracts platform details and threading mechanisms for scalability and performance. 

ACCURAT Toolkit

Link: http://www.accurat-project.eu/

Intro: The ACCURAT project (http://www.accurat-project.eu/) is pleased to announce the release of ACCURAT Toolkit - a collection of tools for comparable corpora collection and multi-level alignment and information extraction from comparable corpora. By using the ACCURAT Toolkit, users may obtain:

- Comparable corpora from the Web (current news corpora, filtered Wikipedia corpora, and narrow domain focussed corpora);

- Comparable document alignments;

- Semi-parallel sentence/phrase mapping from comparable corpora (for SMT training purposes or other tasks);

- Translated terminology extracted and mapped from bilingual comparable corpora;

- Translated named entities extracted and mapped from bilingual comparable corpora.

OpenFst Library

Link: http://www.openfst.org/twiki/bin/view/FST/WebHome

Intro: OpenFst is a library for constructing, combining, optimizing, and searching weighted finite-state transducers (FSTs). Weighted finite-state transducers are automata where each transition has an input label, an output label, and a weight. The more familiar finite-state acceptor is represented as a transducer with each transition's input and output label equal. Finite-state acceptors are used to represent sets of strings (specifically, regular or rational sets); finite-state transducers are used to represent binary relations between pairs of strings (specifically, rational transductions). The weights can be used to represent the cost of taking a particular transition.

Champollion Tool Kit - Text Sentence Aligner

Link: http://champollion.sourceforge.net/

Intro: Built around LDC's champollion sentence aligner kernel, Champollion Tool Kit (CTK) aims to providing ready-to-use parallel text sentence alignment tools for as many language pairs as possible.

Champollion depends heavily on lexical information, but uses sentence length information as well. A translation lexicon is required. Past experiments indicate that champollion's performance improves as the translation lexicon become larger.

Calais

Link: http://www.opencalais.com/

Intro: The OpenCalais Web Service automatically creates rich semantic metadata for the content you submit – in well under a second. Using natural language processing (NLP), machine learning and other methods, Calais analyzes your document and finds the entities within it. But, Calais goes well beyond classic entity identification and returns the facts and events hidden within your text as well.

Albatross Toolkit

http://www.object-craft.com.au/projects/albatross/

http://www.nactem.ac.uk/brat-annotation/

Albatross is a small and flexible Python toolkit for developing highly stateful web applications. The toolkit has been designed to take a lot of the pain out of constructing intranet applications although you can also use Albatross for deploying publicly accessed web applications.

Senna - NLP toolbox from NEC-Labs

http://ml.nec-labs.com/senna/

SENNA is a software distributed under a non-commercial license, which outputs a host of Natural Language Processing (NLP) predictions: part-of-speech (POS) tags, chunking (CHK), name entity recognition (NER) and semantic role labeling (SRL).

Exhibit - Publishing Framework for Data-Rich Interactive Web Pages

http://www.simile-widgets.org/exhibit/

Exhibit lets you easily create web pages with advanced text search and filtering functionalities, with interactive maps, timelines, and other visualizations.

TER-Plus (TERp)

TERp: http://www.umiacs.umd.edu/~snover/terp/

Intro
TERp is an automatic evaluation metric for Machine Translation, which takes as input a set of reference translations, and a set of machine translation output for that same data. It aligns the MT output to the reference translations, and measures the number of 'edits' needed to transform the MT output into the reference translation. TERp is an extension of TER (Translation Edit Rate) that utilizes phrasal substitutions (using automatically generated paraphrases), stemming, synonyms, relaxed shifting constraints and other improvements.

Open Source Machine Translation System Combination

MANY: http://www-lium.univ-lemans.fr/~barrault/MANY/

Intro

MANY is an MT system combination software which architecture is described is the following picture :

The combination can be decomposed into three steps

• 1-best hypotheses from all M systems are aligned in order to build M confusion networks (one for each system considered as backbone).
• All CNs are connected into a single lattice. The first nodes of each CN are connected to a unique first node with probabilities equal to the priors probabilities assigned to the corresponding backbone. The final nodes are connected to a single final node with arc probability of one.
• A token pass decoder is used along with a language model to decode the resulting lattice and the best hypothesis is generated.
  

--
Cheers,
Vu