NIPS paper repository

Link: http://papers.nips.cc/

CASMACAT

Link: http://www.casmacat.eu/index.php?n=Installation.HomePage
Intro: a CAT tool for MT

Competitive Programming Book

Link: https://sites.google.com/site/stevenhalim/
Intro: for programming contests

VisuAlgo

Link: http://visualgo.net/

Intro:  a tool to help his students better understand data structures and algorithms, by allowing them to learn the basics on their own and at their own pace.

IBM model 1

*** Way to get IBM model 1 score

CORPUS=METEO

##########################
#estimating IBM Model 1 with GIZA++

# First step of Moses training is symmetric
perl train-factored-phrase-model.perl -bin-dir . -scripts-root-dir . -root-dir . -corpus $CORPUS -f f -e e -first-step 1 -last-step 1 -alignment grow-diag-final-and -lm 0:3:lmfile >& log.train

mkdir -p ./giza.f-e

./snt2cooc.out ./corpus/e.vcb ./corpus/f.vcb ./corpus/f-e-int-train.snt > ./giza.f-e/f-e.cooc

# GIZA++ alignment is not symmetric
./GIZA++ -CoocurrenceFile ./giza.f-e/f-e.cooc -c ./corpus/f-e-int-train.snt -m1 19 -m2 0 -m3 0 -m4 0 -mh 0 -m5 0 -model1dumpfrequency 1 -nodumps 0 -o ./giza.f-e/f-e -onlyaldumps 0 -s ./corpus/e.vcb -t ./corpus/f.vcb -emprobforempty 0.0 -probsmooth 0.0 >& LOG.f-e
# Output file: giza.f-e/f-e.t1.X
# Format:
# e_code f_code P(f_word | e_word)

# With this script you transform codes into words (looking up into the vocabulary built in the first step
cat giza.f-e/f-e.t1.19 | perl code2word.pl ./corpus/e.vcb ./corpus/f.vcb > f-e.ibm1.giza

##########################
#estimating IBM Model 1 with a standalone software
perl ibm1.pl 20 $CORPUS.f $CORPUS.e > f-e.ibm1.standalone

Machine Learning materials

*** Lecture notes or courses
1) http://dk-techlogic.blogspot.in/2012/05/best-machine-learning-resources.html?m=1
2) https://gtnlp.wordpress.com/readinglist/
3) http://cs229.stanford.edu/materials.html
4) http://ciml.info/
5) Machine Learning for NLP: http://www.cs.columbia.edu/~mcollins/courses/6998-2012/lectures.html

*** ML Community
1) http://www.metacademy.org/roadmaps/
2) http://fastml.com/
3) ...

*** Toolkits
1) Liblinear và Liblinear with SBM (C++, Java,...)
Link: http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/…

2) StreamSVM (C++)
Link: http://www.ibis.t.u-tokyo.ac.jp/masin/streamsvm.html

3) Vowpal Wabbit (C++, Python wrapper)
Link: http://hunch.net/~vw/

4) SGD
Link: http://leon.bottou.org/projects/sgd

5) Super-big list of ML softwares
Link: http://mloss.org/software/

6) ...

(to be updated)

The ClueWeb09 Dataset

Link: http://www.lemurproject.org/clueweb09/

Intro: The ClueWeb09 dataset was created to support research on information retrieval and related human language technologies. It consists of about 1 billion web pages in ten languages that were collected in January and February 2009. The dataset is used by several tracks of the TREC conference.

Note: Huge corpus for LM