Thursday, 16 April 2015
Thursday, 12 March 2015
Thursday, 5 March 2015
Vowpal Wabbit
Intro: The Vowpal Wabbit (VW) project is a fast out-of-core learning system sponsored by Microsoft Research and (previously) Yahoo! Research. Support is available through the mailing list.
There are two ways to have a fast learning algorithm: (a) start with a slow algorithm and speed it up, or (b) build an intrinsically fast learning algorithm. This project is about approach (b), and it's reached a state where it may be useful to others as a platform for research and experimentation.
Sunday, 1 March 2015
Training Google 1T web corpus with IRSTLM
Thanks to http://www44.atwiki.jp/keisks/pages/50.html , here is the way to train enormous LM with Google 1T web corpus using IRSTLM:
I will validate it soon.
build-sublm.pl --size 3 --ngrams "gunzip -c 3gms/*.gz" --sublm LM.000 --witten-bell merge-sublm.pl --size 3 --sublm LM -lm g_3grams_LM.gz compile-lm g_3grams_LM.gz g_3grams_LM.blm (if you get the error: "lt-compile-lm: lmtable.h:247: virtual double lmtable::setlogOOVpenalty(int): Assertion `dub > dict->size()' failed.") compile-lm -dub=100000000 g_3grams_LM g_3grams_LM.blm (make the -dub option bigger)
I will validate it soon.
Friday, 27 February 2015
Simple batch programming on Windows
Some simple tricks for batch programming on Windows environment:
1) For loop (refer: http://ss64.com/nt/for_l.html)
Syntax
FOR /L %%parameter IN (start,step,end) DO command
Key
start : The first number
step : The amount by which to increment the sequence
end : The last number
command : The command to carry out, including any
command-line parameters.
%%parameter : A replaceable parameter:
in a batch file use %%G (on the command line %G)
Example
FOR /L %%G IN (0,1,195) DO (
echo %%G & ping 1.1.1.1 -n 1 -w 1000 >nul
)
2) ...
1) For loop (refer: http://ss64.com/nt/for_l.html)
Syntax
FOR /L %%parameter IN (start,step,end) DO command
Key
start : The first number
step : The amount by which to increment the sequence
end : The last number
command : The command to carry out, including any
command-line parameters.
%%parameter : A replaceable parameter:
in a batch file use %%G (on the command line %G)
Example
FOR /L %%G IN (0,1,195) DO (
echo %%G & ping 1.1.1.1 -n 1 -w 1000 >nul
)
Thursday, 26 February 2015
SEAlang - Southeast Asian Languages Library
Intro: The SEAlang Library was established in 2005. It provides language reference materials for Southeast Asia; initially focused on the non-roman script languages used throughout the mainland, and now concentrating on the languages of insular SEA.
* Take note of the Vietnamese corpus (including dictionary, bitexts, ...)
Sunday, 22 February 2015
How to be a good graduate student
1) From Kevin Murphy: http://www.cs.ubc.ca/~murphyk/Teaching/guideForStudents.html
2) ...
(to be updated)
2) ...
(to be updated)
Labels:
graduate advice,
graduate student,
links,
phd,
research
Subscribe to:
Posts (Atom)