PHP 爬取需要运行 JS 的页面 (Run JS While Grabing Web Page With PHP)

最新推荐文章于 2022-04-10 19:08:47 发布

weixin_33728708

最新推荐文章于 2022-04-10 19:08:47 发布

阅读量292

点赞数

文章标签： php 爬虫 javascript ViewUI

原文链接：https://my.oschina.net/u/234439/blog/1493057

版权

2019独角兽企业重金招聘Python工程师标准>>>

参照安装

以CentOS为例，下载Linux 64-bit版本(32/64区分好)

wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2

解压文件

tar xvf phantomjs-2.1.1-linux-x86_64.tar.bz2

移动文件到bin目录下

cp phantomjs-2.1.1-linux-x86_64/bin/phantomjs /usr/local/bin

写一个js来进行判断

/**
 * Created by liwei on 2017/3/6.
 */
console.log(' hello world');

运行命令

phantomjs helloworld.js

也可以参考

https://laravel-china.org/topics/3590/php-crawls-the-page-that-needs-to-run-js-run-js-grabing-web-page-with-php-while

和官网http://jonnnnyw.github.io/php-phantomjs/4.0/

<?php
/**
 * Created by PhpStorm.
 * User: liwei
 * Date: 2017/3/6
 * Time: 下午4:18
 */

require __DIR__ . '/vendor/autoload.php';


use JonnyW\PhantomJs\Client;

$client = Client::getInstance();

/**
 * @see JonnyW\PhantomJs\Http\Request
 **/
$client->getEngine()->setPath('/usr/local/bin/phantomjs');

$request = $client->getMessageFactory()->createRequest('http://www.****.com/ckplayer/js/play.php?v=45554f535e1b140c5b0f4a520342544e0259144042575e5102150c5b5e56544f4b480f4a4c4c575b03494f4e411e4d4a491c53131d130f5b1c4b591546597efd2a&t=qq', 'GET');
$request->setHeaders(["Referer"=>"http://www.****.com/play/45453-0-1.html"]);


/**
 * @see JonnyW\PhantomJs\Http\Response
 **/
$response = $client->getMessageFactory()->createResponse();

// Send the request
$client->send($request, $response);

$html =  $response->getContent();
$reg = '|\#url=(http[^"]+)|';
if(preg_match($reg, $html ,$matches)){

    $url = urldecode($matches[1]);
    echo $url;
    //var_dump($matches);
}
exit;

if($response->getStatus() === 200) {

    // Dump the requested page content
    echo $response->getContent();
}

转载于:https://my.oschina.net/u/234439/blog/1493057