php实现的简单的csdn博客文章抓取(续:添加用户名搜索提示)

最新推荐文章于 2024-09-28 16:31:15 发布

yysdsyl

最新推荐文章于 2024-09-28 16:31:15 发布

阅读量1.4k

点赞数

文章标签： php url query blog extension header

本文链接：https://blog.csdn.net/yysdsyl/article/details/4632141

版权

本文介绍了如何使用PHP实现类似Google Suggest的CSDN博客用户名搜索提示功能。首先通过爬虫获取CSND用户列表，然后利用jQuery和AJAX实现前端交互，展示搜索建议。

摘要由CSDN通过智能技术生成

上篇blog文章实现了根据指定的csdn用户名列出其所有的blog文章的功能，不过这个其实没多大实际用处，我们必须得精确知道感兴趣用户的用户名才能列出他的blog文章，所以闲来想弄一个类似于google suggest的输入提示功能。

要实现搜索提示，当然手头得有csdn用户列表可供使用，毫无疑问这个用户列表要靠自己去crawl，于是写了个极其简单的爬取php脚本，如下：

<?php // require_once("config.php"); require_once("database.php"); require_once("crawl_func.php"); /fetch url content function get_url_content($url) { if(extension_loaded('curl')) { $ch = curl_init($url); curl_setopt($ch, CURLOPT_HEADER, 0); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $content = curl_exec($ch); curl_close($ch); } else { $content = file_get_contents($url); } //!$content && die("get $url content error."); !$content && $content = "zfq"; return $content; } / set_time_limit(0); header("Content-Type:text/html; charset=utf-8"); zfqsql_open(); addUrlToDb($START_CRAWL_URL); $url_counter = 0; crawl.... print "start to crawl ... <br />/n"; while($crawlurl = getUrlToCrawl()) { hack usleep ($CRAWL_THREAD_SLEEP_TIME); print "handle URL : ".$crawlurl."<br />/n"; $content = get_url_content($crawlurl); $reg_pattern = '/(http:(blog|hi).csdn.net//([a-zA-Z0-9_-]+)//?)/"/'; if(preg_match_all($reg_pattern, $content, $matches)) { $tempcrawurls = $matches[1]; $tempblogname = $matches[3]; $num = count($matches[0]); for($i = 0; $i < $num; $i++) { addUrlToDb($tempcrawurls[$i]); addBlognameToDb($tempblogname[$i]); } } $reg_pattern = '/(http:(forum|topic).csdn.net//[^/"]+)/"/'; if(preg_match_all($reg_pattern, $content, $matches)) { $exclude_pattern = '/User/'; $tempcrawurls = $matches[1]; $num = count($matches[0]); for($i = 0; $i < $num; $i++) { preg_match($exclude_pattern, $tempcrawurls[$i], $exclude_match); if($exclude_match) { continue; } addUrlToDb($tempcrawurls[$i]); } } $url_counter++; updateUrlToDb($crawlurl); } ?>

crawl的过程比较慢，截止到发帖时间已经爬取了3、4小时的时间，爬取到的用户总数已经达到2w多，已经分析的url数目仅1000多条，待分析和爬取的url目前为止是6w条左右，这个数目还会继续增加，当然爬取得到的用户数目和待爬取的url数目越往后关系越不大。

crawl得到所需的csdn用户列表后，剩下的工作就是怎样实现搜索提示的功能，这部分的实现参考了别人的代码，在此就不作介绍，只给出相关代码和效果图

index.htm

<html> <head> <meta http-equiv=Content-Type content="text/html;charset=utf-8"> <title>for list csdn blog entry</title> <mce:script type="text/javascript" src="jquery-1.2.1.pack.js" mce_src="jquery-1.2.1.pack.js"></mce:script> <mce:script type="text/javascript"></mce:script> <mce:style type="text/css"></mce:style><style type="text/css" mce_bogus="1"> body { font-family: Helvetica; font-size: 11px; color: #000000; } h3 { margin: 0px; padding: 0px; } .suggestionsBox { position: relative; left: 300px; margin: 10px 0px 0px 0px; width: 200px; background-color: #212427; -moz-border-radius: 7px; -webkit-border-radius: 7px; border: 2px solid #000; color: #00ff00; } .suggestionList { margin: 0px; padding: 0px; } .suggestionList li { margin: 0px 0px 3px 0px; padding: 3px; cursor: pointer; } .suggestionList li:hover { background-color: #1f20ff; }</style> </head> <h1 align="center"> CSDN BLOG ENTRY LIST </h1> <form method="GET" action="get_csdnblog.php"> <br /> <br /> <br /> <table cellSpacing=0 cellPadding=0 width=400 align="center"> <tr> <td align="center"><input maxLength=256 size=50 name="blogname" id="blogname" type="text" οnkeyup="lookup(this.value);" οnblur="fill();"> <input type=submit value=" List "></td> </tr> </table> <div class="suggestionsBox" id="suggestions" style="display: none;" mce_style="display: none;"> <img src="upArrow.png" mce_src="upArrow.png" style="position: relative; top: -12px; left: 30px;" alt="upArrow" /> <div class="suggestionList" id="autoSuggestionsList"> </div> </div> </form> </body> </html>

/bloginputhint.php

<?php // PHP5 Implementation - uses MySQLi. // mysqli('localhost', 'yourUsername', 'yourPassword', 'yourDatabase'); $db = new mysqli('localhost', 'root' ,'841025', 'csdnblog'); if(!$db) { // Show error if we cannot connect. echo 'ERROR: Could not connect to the database.'; } else { // Is there a posted query string? if(isset($_POST['queryString'])) { $queryString = $db->real_escape_string($_POST['queryString']); // Is the string length greater than 0? if(strlen($queryString) >0) { // Run the query: We use LIKE '$queryString%' // The percentage sign is a wild-card, in my example of countries it works like this... // $queryString = 'Uni'; // Returned data = 'United States, United Kindom'; // YOU NEED TO ALTER THE QUERY TO MATCH YOUR DATABASE. // eg: SELECT yourColumnName FROM yourTable WHERE yourColumnName LIKE '$queryString%' LIMIT 10 $query = $db->query("SELECT user FROM blogname WHERE user LIKE '$queryString%' LIMIT 10"); if($query) { // While there are results loop through them - fetching an Object (i like PHP5 btw!). while ($result = $query ->fetch_object()) { // Format the results, im using <li> for the list, you can change it. // The onClick function fills the textbox with the result. // YOU MUST CHANGE: $result->value to $result->your_colum echo '<li onClick="fill(/''.$result->user.'/');">'.$result->user.'</li>'; } } else { echo 'ERROR: There was a problem with the query.'; } } else { // Dont do anything. } // There is a queryString. } else { echo 'There should be no direct access to this script!'; } } ?>

效果图：