PHP CURL模拟登录抓取数据

最新推荐文章于 2021-03-09 19:45:39 发布

轩辕朗逸

最新推荐文章于 2021-03-09 19:45:39 发布

阅读量676

点赞数 1

分类专栏： PHP 文章标签： php curl 抓取

本文链接：https://blog.csdn.net/kissxia/article/details/50250979

版权

PHP 专栏收录该内容

29 篇文章 0 订阅

订阅专栏

1、CURL模拟登录的方法

<?php
/**
 * Created by PhpStorm.
 * User: kungyu
 * Date: 2015/11/26
 * Time: 15:56
 */
function login_post($url, $cookie, $header){ //方法采用的的是GET请求，若是POST请求，增加一个传入参数 $post
    $curl = curl_init();
    curl_setopt($curl,CURLOPT_URL,$url);
    curl_setopt($curl, CURLOPT_HTTPHEADER, $header);
    curl_setopt($curl, CURLOPT_HEADER, 0);
    curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($curl, CURLOPT_COOKIEJAR, $cookie);
    /*curl_setopt($curl, CURLOPT_POST, 1);*/                                 //POST 请求需要开启该处注释
    curl_setopt($curl, CURLOPT_NOBODY, true);
    /* curl_setopt($curl, CURLOPT_POSTFIELDS, http_build_query($post));*/    //POST 请求需要开启该处注释
    $res = curl_exec($curl);
    curl_close($curl);
    return $res;
}

function get_content($url, $cookie,$post) {
$header = array (
    'Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
    "Cache-Control:max-age=0",
    /*"Accept-Encoding:gzip, deflate",*/
    "Accept-Language:zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3",
    "Connection:keep-alive",
    "Content-Type:application/x-www-form-urlencoded; charset=UTF-8",
    "Host:******",
    "Origin:******",
    "User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.134 Safari/537.36",
    "x-insight:activate"
);
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie); //读取cookie
    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_NOBODY, false);
    curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($post));
    $rs = curl_exec($ch); 
    curl_close($ch);
    return $rs;
}

2、登录系统创建cookie文件

include("D:/reg/function.php");
$loginUrl = "**********";
$cookieFile = tempnam('D:/reg','cookie');
login_post($loginUrl,$cookieFile,$header);
file_put_contents('D:/reg/cookie_path.txt',$cookieFile);

3、抓取页面数据

$dir = dirname(__FILE__);
$cookieFile = file_get_contents($dir."/cookie_path.txt");
$detail_url = "*******";
$search_arr = array(
            'param'=> 'data'
        );
$search_res = get_content($search_url, $cookieFile, $search_arr);

//剩下的就是数据的正则匹配根据自己想要的获取数据。