Blog Post Extraction Using Titles ------------- by songlinhai

Blog Post Extraction Using Title Finding
Linhai Song, Xueqi Cheng, Yan Guo, Bo Wu, Yu Wang

accepted by ccir'2009: Proceedings of Chinese conference on Information Retrieval
Abstract: With the development of Web2.0, web mining applications pay more attention to blog pages. In order to prevent noises in blog pages from affecting the precision of web mining algorithms, it is very necessary to acquire posts from blog pages correctly. In this paper, we propose a blog post extraction algorithm which uses title finding. There are two stages in this algorithm. In the first stage, text nodes which indicate the title of the post are found and used as the beginning of the post. We take a machine learning approach to realize this stage, and employ SVM as classification model. In the second stage, we find the end of the post. Two methods are introduced in this stage, one uses VIPS segmentation results, and the other is based on hand-coded rules. Experiments are conducted to see how we find titles and how we extract posts. Experimental results show that our algorithm has ideal effects.