Body Text extraction from websites e.g. extract only article heading and text not all text in site

Asked 2011-04-21T15:02:05.313

I am looking for algorithms that allow text extraction from websites. I do not mean "strip html", or any of the hundreds of libraries that allow this.

So for example for a news article I would like to identify the heading and all the text, but not the comments section and so on.

Are there any algorithms for that out there? Thank you!

Edit Report

1 Answer

In computer science literature this problem is usually referred to as the page segmentation or boiler plate detection problem. See the report Boilerplate Detection using Shallow Text Features. Also, I have a few reports and software sites bookmarked that address the problem. Also, see this stackoverflow question.

answered 2011-04-23T02:35:39.907

Body Text extraction from websites e.g. extract only article heading and text not all text in site

1 Answer

Your Answer