Create Great Parser - Extract Relevant Text From HTML/Blogs

Question

I\'m trying to create a generalized HTML parser that works well on Blog Posts. I want to point my parser at the specific entrie\'s URL and get back clean text of the post it

Answer 1

Boy, do I have the perfect solution for you.

Arc90's readability algorithm does exactly this. Given HTML content, it picks out the content of the main blog post text, ignoring headers, footers, navigation, etc.

Here are implementations in:

• JavaScript
• Perl
• PHP
• Python
• Ruby
• C#

I'll be releasing a Perl port to CPAN in a couple of days. Done.

Hope this helps!