php网页解析器的应用 simple_html_dom

<?php
include_once('simple_html_dom.php');

function parse_html($url)

{
$opts = array(
'http'=>array(
'method'=>"GET",
'timeout'=>20,
)
);


//设置超时
$context = stream_context_create($opts);


//从一个URL或者文件创建一个DOM对象
$html = file_get_html($url, false, $context);


if($html == false)
{
echo "404";
}


// 寻找所有的img标签
foreach($html->find('.xxx') as $element)
{
foreach($lf5_g_pic->find('a') as $g_pic)
echo $g_pic->href."\n";


foreach($lf5_g_pic->find('img') as $img_s)   
echo $img_s->src."\n\n\n";
}

}

parse_html('http://www.google.com');

?>

在simple_html_dom.php中需要注意超时一定次数便认为此链接不可访问，代码如下

function file_get_html($url, $use_include_path = false, $context=null, $offset = -1, $maxLen=-1, $lowercase = true, $forceTag
sClosed=true, $target_charset = DEFAULT_TARGET_CHARSET, $stripRN=true, $defaultBRText=DEFAULT_BR_TEXT)
{
    // We DO force the tags to be terminated.
    $dom = new simple_html_dom(null, $lowercase, $forceTagsClosed, $target_charset, $defaultBRText);
    // For sourceforge users: uncomment the next line and comment the retreive_url_contents line 2 lines down if it is not already done.


    //add by cleanfield
    $cnt=0;
    while($cnt < 3 && ($contents = file_get_contents($url, $use_include_path, $context, $offset))===FALSE)
    {
        $cnt++;
    }


    //$contents = file_get_contents($url, $use_include_path, $context, $offset);

   ......

｝

在抓取页面时，一般都是批量任务，所以一定要注意设置每次拉取页面的超时时间，同时也要有重试。

附：

http://simplehtmldom.sourceforge.net/manual.htm      simplehtmldom说明文档