Laravel中curl_multi并发爬取百度关键词排名

Laravel中curl_multi并发爬取百度关键词排名

---

框架使用的Laravel

Linux中crontab定时任务添加

*/1 * * * *    /data/wwwroot/default/newgdh/artisanCommand

项目newgdh中添加artisanCommand

!/bin/sh
/usr/local/php/bin/php /data/wwwroot/default/newgdh/artisan schedule:run

---

App\Console\kernal.php中添加

 protected $commands = [
        \App\Console\Commands\GetKeyWordSortCommand::class,
    ];

   protected function schedule(Schedule $schedule)
    {
        $schedule->command('GetKeyWordSortCommand')->everyMinute();// 本地测试用时间，可以自己修改
    }

---

生成 App\Console\Commands\GetKeyWordSortCommand.php

php artisan make:command GetKeyWordSortCommand

并且
 protected $signature = 'GetKeyWordSortCommand';
 protected $description = '百度关键词定时爬取';	

---

laravel队列

class GetKeyWordSortCommand extends Command
{
use DispatchesJobs;
注意： use DispatchesJobs；因为在handle中无法直接使用dispatch（）推送任务到队列

public function handle()
    {
        // 获取关键词词库
        $keywordsList = Keyword::get();
        foreach($keywordsList as $k=>$v ){
            $url_arr = [];
            $page = 1;
            for($i=0;$i<2;$i++){
                $url_arr[$i] = [