HTML Parsers

1) Jericho HTML Parser: http://jericho.htmlparser.net/docs/index.html

2) boilerpipe: http://researchlog-duyvuleo.blogspot.com/search?q=boilerpipe

3) HTML Parser: http://htmlparser.sourceforge.net/

4) TBA

Common Crawl

http://www.commoncrawl.org/

Common Crawl Foundation is a California 501(c)3 non-profit founded by Gil Elbaz with the goal of democratizing access to web information by producing and maintaining an open repository of web crawl data that is universally accessible.

WebSPHINX

Link: http://www.cs.cmu.edu/~rcm/websphinx/

Intro: WebSPHINX ( Website-Specific Processors for HTML INformation eXtraction) is a Java class library and interactive development environment for web crawlers. A web crawler (also called a robot or spider) is a program that browses and processes Web pages automatically.