##1、线上demo
http://demo.blueyian.top/marathon/index.php
完整的代码包请上gayhub取用。觉得有用的可以给个star :)
地址:https://github.com/KongYian/marathon-query
##2、截图
###2.1、首页
###2.2、查询结果
##3、实现
###3.1、分析目标网站
我们的目标是http://www.runchina.org.cn/portal.php?mod=score&ac=personal 因此先来分析一下此网站是如何实现成绩查询。
多尝试输入几次自己的查询信息,打开F12观察NetWork和Application里面的数据,我们可以简单的判断出查询的大致流程如下图:
在反复试验的过程和检查中,我们会发现这个网站木有什么CRSF等保护,除了一个±*/的验证码,其他就是一个赤裸裸的接口了。
###3.2、流程
在实际做的过程中,我将第一步和第二步放在一起作为了一个接口(命名为-- 接口1)。获取验证码图片和PHPSESSIONID,代码如下:
其中关键操作在代码注释中–
<?php
$verify_code_url = "http://www.runchina.org.cn/template/default/public/js/securimage/securimage_show.php";
$query_url = "http://www.runchina.org.cn/portal.php?mod=score&ac=personal";
$cookie_file = "../tmp.cookie";
showAuthcode($verify_code_url,$cookie_file);
$handle = fopen($cookie_file,'r');
$line= '';
while (!feof($handle))
{
$line .= fgets($handle);
}
preg_match("/PHPSESSID(?<right>.*)/",$line,$sessionArr);
fclose($handle);
$session = trimall($sessionArr['right'],' ');
$sessionString = "PHPSESSID=".$session.';';
$res = curlLogin($query_url,$cookie_file,$sessionString);
preg_match_all('/Set-Cookie:(.*);/iU',$res,$out);
$tmp = implode(';',$out[1]);
$cookieString = $sessionString.$tmp; //此变量围第二次请求使用的cookie值
echo json_encode(['data'=>$cookieString]);
exit;
function trimall($str)//删除空格
{
$oldchar=array(" "," ","\t","\n","\r");
$newchar=array("","","","","");
return
str_replace($oldchar,$newchar,$str);
}
function showAuthcode( $authcode_url,$cookieFile)
{
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $authcode_url);
curl_setopt($curl, CURLOPT_COOKIEJAR, $cookieFile);
//将获取的cookie以文件的形式保存
curl_setopt($curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36');
curl_setopt($curl, CURLOPT_HEADER, 0);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
$img = curl_exec($curl);
curl_close($curl);
$fp = fopen("../image/verifyCode.jpg","w");
//获取验证码的图片
fwrite($fp,$img);
fclose($fp);
}
function curlLogin($url,$cookiefile,$session)
{
$headers = [
"Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
"Accept-Encoding:gzip, deflate",
"Accept-Language:zh-CN,zh;q=0.9",
"Connection:keep-alive",
"Cookie:".$session,
"Host:www.runchina.org.cn",
"Upgrade-Insecure-Requests:1",
"User-Agent:Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36",
];
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_ACCEPT_ENCODING, "gzip, deflate, sdch");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.221 Safari/537.36 SE 2.X MetaSr 1.0");
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookiefile);
//用请求验证码接口获取的cookie作为本次请求的set-cookie 获取新的cookie文件,但是这里我们没有以文件形式存储而是直接输出,这样避免了多次文件IO的消耗。
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_HEADER, true);
$contents = curl_exec($ch);
curl_close($ch);
return $contents;
}
我们从接口一获取到了要用的cookieString和验证码的图片,下面就来模拟表单提交了,模拟提交的代码如下:
$query_url = "http://www.runchina.org.cn/portal.php?mod=score&ac=personal";
$idnum = $_POST['idnum'];
$name = $_POST['name'];
$code = $_POST['code'];
$cookie = $_POST['cookie'];
$params = [
'idnum'=>$idnum,
'name'=>$name,
'captcha_code'=>$code
];
$https = query($query_url,$params,$cookie);
function query($query_url,$params,$cookie){
$headers = [
"Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
"Accept-Encoding:gzip, deflate",
"Accept-Language:zh-CN,zh;q=0.9",
"Connection:keep-alive",
"Cookie:".$cookie,
"Host:www.runchina.org.cn",
"Upgrade-Insecure-Requests:1",
"User-Agent:Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36",
];
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $query_url);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($params));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$resp = curl_exec($ch);
curl_close($ch);
return $resp;
}
function trimall($str)//删除空格
{
$oldchar=array(" "," ","\t","\n","\r");
$newchar=array("","","","","");
return
str_replace($oldchar,$newchar,$str);
}
变量$https
便是我们请求的结果了,如果正确的话是一个结果页面,如果没有数据或者出错的话则没有数据了。
拿到页面之后,我们就可以来操作dom了,这里我用的是‘simple_html_dom’,具体如何食用可以google一下。以下是我操作的具体代码:
require_once 'simple_html_dom.php';
$htmlDom = str_get_html($https);
$out = [];
foreach($htmlDom->find('.myScore tbody tr') as $kk => $e) {
if($kk != 0){
foreach ($e->children as $k => $child) {
switch ($k){
case 0: $out[$kk]['date'] = $child->plaintext ;break;
case 1: $out[$kk]['name'] = trimall($child->plaintext) ;break;
case 2: $out[$kk]['type'] = trimall($child->plaintext) ;break;
case 3:
$out[$kk]['raceNetTime'] = $child->plaintext ;
if(strpos($out[$kk]['raceNetTime'],'PB') !== false){
$out[$kk]['pbColor'] = 'pink';
}else{
$out[$kk]['pbColor'] = '';
}
break;
case 4:$out[$kk]['raceTrueTime'] = $child->plaintext ;break;
// case 5: $out[$kk]['raceDetailTime'] = trimall($child->innertext) ;break;
}
}
}
}
//释放资源对象,会很占用内存
$htmlDom->clear();
unset($htmlDom);
最后我们得到的数据结构如下:
前端的话,用的很乱,vue,jq,layer,妹子UI都用了,正真的demo…
放一些JS代码吧:
<script>
var vm = new Vue({
el:'#app',
data:{
name:localStorage.getItem('name')=='undefined'?'':localStorage.getItem('name'),
idnum:localStorage.getItem('idnum')=='undefined'?'':localStorage.getItem('idnum'),
code:'',
showSearch : 1,
result:'',
imageSrc : 'image/verifyCode.jpg',
isPBColor :'pink'
},
beforeCreate:function(){
cookie = init();
},
filters:{
},
methods:{
query:function () {
if(!(this.name && this.idnum && this.code && cookie)){
layer.msg('每一项都要填写:)');
return false;
}
var load = layer.load();
$.ajax({
url:'action/search.php',
data:{
name:this.name,
idnum:this.idnum,
code:this.code,
cookie:cookie,
},
dataType:'json',
type:'post',
success:function (response) {
if(response.status == 1){
vm.result = response.data;
vm.showSearch = 0;
localStorage.setItem('name',vm.name);
localStorage.setItem('idnum',vm.idnum);
}else{
layer.msg('未查询到成绩,再试试吧QAQ');
vm.reload();
return false;
}
},
error:function () {
layer.msg('服务器开小差啦,稍后再试');
},
complete:function () {
layer.close(load)
}
})
},
reload:function () {
window.location.reload();
}
}
})
function init() {
var cookieString;
$.ajax({
url:'action/init.php',
dataType:'json',
type:'post',
async:false,
success:function (response) {
cookieString = response.data
},
error:function () {
}
})
return cookieString;
}
</script>