方法1:分块结果 chunk
function test4()
{
$start = microtime(true);
$num = 0;
MapCity::chunk(10, function () use (&$num) {
$num += 10;
});
$end = microtime(true);
$this->info($num);
$this->info($end-$start);
}
3410
1.0684580802917
查看SQL日志,发现chunk方法居然是分页读取的,效率其实很低的,从时间上就能看出来。
2020-01-10 03:16:42: select * from `map_city` order by `map_city`.`id` asc limit 10 offset 0
2020-01-10 03:16:42: select * from `map_city` order by `map_city`.`id` asc limit 10 offset 10
2020-01-10 03:16:42: select * from `map_city` order by `map_city`.`id` asc limit 10 offset 20
…
方法2:游标
function test5()
{
$start = microtime(true);
$num = 0;
foreach (MapCity::cursor() as $v) {
$num++;
}
$end = microtime(true);
$this->info($num);
$this->info($end-$start);
}
3408
0.032788038253784
查看SQL日志,发现cursor方法只读取一次
2020-01-10 03:16:43: select * from `map_city`
官方如是说:该游标只执行一个查询。处理大量数据时,使用 cursor 方法可以大幅度减少内存的使用量。
在便利的情况下能大量节省内存开销,很有可能使用了生成器,于是就去看一下源码。
Illuminate\Database\Connection
第344行:
public function cursor($query, $bindings = [], $useReadPdo = true)
{
$statement = $this->run($query, $bindings, function ($query, $bindings) use ($useReadPdo) {
if ($this->pretending()) {
return [];
}
// First we will create a statement for the query. Then, we will set the fetch
// mode and prepare the bindings for the query. Once that's done we will be
// ready to execute the query against the database and return the cursor.
$statement = $this->prepared($this->getPdoForSelect($useReadPdo)
->prepare($query));
$this->bindValues(
$statement, $this->prepareBindings($bindings)
);
// Next, we'll execute the query against the database and return the statement
// so we can return the cursor. The cursor will use a PHP generator to give
// back one row at a time without using a bunch of memory to render them.
$statement->execute();
return $statement;
});
while ($record = $statement->fetch()) {
yield $record;
}
}
在解析PDO结果集的时候不是一次性转换,而是使用生成器的方式逐条获取,的确在处理大量集合的时候使用生成器的方式
来遍历能大幅度减小内存开销。