php网页解析器,php网页解析器的应用 simple_html_dom

include_once('simple_html_dom.php');

function parse_html($url) {

$opts = array(

'http'=>array(

'method'=>"GET",

'timeout'=>20,

)

);

//设置超时

$context = stream_context_create($opts);

//从一个URL或者文件创建一个DOM对象

$html = file_get_html($url, false, $context);

if($html == false)

{

echo "404";

}

// 寻找所有的img标签

foreach($html->find('.xxx') as $element)

{

foreach($lf5_g_pic->find('a') as $g_pic)

echo $g_pic->href."\n";

foreach($lf5_g_pic->find('img') as $img_s)

echo $img_s->src."\n\n\n";

}

}

parse_html('http://www.google.com');

?>

在simple_html_dom.php中需要注意超时一定次数便认为此链接不可访问，代码如下

function file_get_html($url, $use_include_path = false, $context=null, $offset = -1, $maxLen=-1, $lowercase = true, $forceTag

sClosed=true, $target_charset = DEFAULT_TARGET_CHARSET, $stripRN=true, $defaultBRText=DEFAULT_BR_TEXT)

{

// We DO force the tags to be terminated.

$dom = new simple_html_dom(null, $lowercase, $forceTagsClosed, $target_charset, $defaultBRText);

// For sourceforge users: uncomment the next line and comment the retreive_url_contents line 2 lines down if it is not already done.

//add by cleanfield

$cnt=0;

while($cnt < 3 && ($contents = file_get_contents($url, $use_include_path, $context, $offset))===FALSE)

{

$cnt++;

}

//$contents = file_get_contents($url, $use_include_path, $context, $offset);

......

｝

在抓取页面时，一般都是批量任务，所以一定要注意设置每次拉取页面的超时时间，同时也要有重试。

附：