Laravel中curl_multi并发爬取百度关键词排名
框架使用的Laravel
Linux中crontab定时任务添加
*/1 * * * * /data/wwwroot/default/newgdh/artisanCommand
项目newgdh中添加artisanCommand
!/bin/sh
/usr/local/php/bin/php /data/wwwroot/default/newgdh/artisan schedule:run
App\Console\kernal.php中添加
protected $commands = [
\App\Console\Commands\GetKeyWordSortCommand::class,
];
protected function schedule(Schedule $schedule)
{
$schedule->command('GetKeyWordSortCommand')->everyMinute();// 本地测试用时间,可以自己修改
}
生成 App\Console\Commands\GetKeyWordSortCommand.php
php artisan make:command GetKeyWordSortCommand
并且
protected $signature = 'GetKeyWordSortCommand';
protected $description = '百度关键词定时爬取';
class GetKeyWordSortCommand extends Command
{
use DispatchesJobs;
注意: use DispatchesJobs;因为在handle中无法直接使用dispatch()推送任务到队列
public function handle()
{
// 获取关键词词库
$keywordsList = Keyword::get();
foreach($keywordsList as $k=>$v ){
$url_arr = [];
$page = 1;
for($i=0;$i<2;$i++){
$url_arr[$i] = [