Generators in PHP

If you’ve followed my previous posts about iterators then you’ll know that iteration is an important programming concept, but implementing the required interfaces to create an iterable object can be a hassle at best because of the amount of boilerplate code that is required. With the release of PHP 5.5, we finally have generators!

In this article we’ll take a look at generators which provide an easy way to implement simple iterators without the overhead or complexity of the Iterator interface.

How do Generators Work?

According to Wikipedia, a generator “is very similar to a function that returns an array, in that a generator has parameters, can be called, and generates a sequence of values”. A generator is basically a normal function, but instead of returning a value it yields as many values as it needs to. It looks like a function but acts like an iterator.

Generators use the yield keyword instead of return. It acts similar to return in that it returns a value to the caller of the function, but instead of removing the function from the stack, yield saves its state. This allows the function to continue from where it was when it’s called again. In fact, you cannot return a value from a generator although you can use return without a value to terminate its execution.

The PHP manual states: “When a generator function is called, it returns an object that can be iterated over.” This is an object of the internal Generator class and implements the Iterator interface in the same way a forward-only iterator object does. As you iterate over that object, PHP calls the generator each time it needs a value. The state is saved when the generator yields so that it can be resumed when the next value is required.

<?php
function nums() {
    echo "The generator has startedn"; 
    for ($i = 0; $i < 5; ++$i) {
        yield $i;
        echo "Yielded $in";
    }
    echo "The generator has endedn"; 
}

foreach (nums() as $v);

The output of the above code will be:

The generator has started
Yielded 0
Yielded 1
Yielded 2
Yielded 3
Yielded 4
The generator has ended

Our First Generator

Generators are not a new concept and already exist in languages such as C#, Python, JavaScript, and Ruby (enumerators), and are usually identified by their use of the yield keyword. The following is an example in Python:

def file_lines(filename):
    file = open(filename)
    for line in file:
        yield line
    file.close()

for line in file_lines('somefile'):
    #do some work here

Let’s rewrite the example Python generator in PHP. (Note that both snippets do not perform any sort of error checking.)

<?php
function file_lines($filename) {
    $file = fopen($filename, 'r'); 
    while (($line = fgets($file)) !== false) {
        yield $line; 
    } 
    fclose($file); 
}

foreach (file_lines('somefile') as $line) {
    // do some work here
}

The generator function opens a file and then yields each line of the file as and when it is required. Each time the generator is called, it continues from where it left off. It doesn’t start from the beginning again as its state had been saved when the yield statement was executed. Once all lines have been read, the generator simply terminates and the loop ends.

Returning Keys

PHP iterators consist of key/value pairs. In our example, only a value was returned and therefore the keys were numeric (keys are numeric by default). If you wish to return an associative pair, simply change the yield statement to include the key using array syntax.

<?php
function file_lines($filename) {
    ...
        yield $key => $line; 
    ...
}

foreach (file_lines('somefile') as $key => $line) {
    // do some work here
}

Injecting Values

yield does not only return values; it can receive values from the outside as well. This is done by calling the send() method of the generator object with the value you wish to pass. This value can then be used in computing or doing other stuff. The method sends the value to the generator as a result of the yield expression and resumes execution.

<?php
function nums() {
    for ($i = 0; $i < 5; ++$i) {
        // get a value from the caller
        $cmd = (yield $i);
        if ($cmd == 'stop') {
            return; // exit the generator
        }
    }
}

$gen = nums();

foreach ($gen as $v) {
    // we are satisfied
    if ($v == 3) {
        $gen->send('stop');
    }
    echo "{$v}n";
}

The output will be:

0
1
2
3

Saving Memory with Generators

Generators are great for when you are calculating large sets and you don’t want to allocate memory for all of the results at the same time or when you don’t know if you will need all of the results, Due to the way results are processed, the memory footprint can be reduced to a very bare minimum by allocating memory for only the current result.

Imagine the file() function which returns all of the lines in a file as an array. Running a simple benchmark for file() and our demo file_lines() functions, each using the same random 100 paragraph text file generated using Lipsum, showed the file function used up to 110 times more memory than the generator.

<?php
// Test 1
$m = memory_get_peak_usage();
foreach (file_lines('lipsum.txt') as $l);
echo memory_get_peak_usage() - $m, "n"; //Outputs 7336

// Test 2
$m = memory_get_peak_usage();
foreach (file('lipsum.txt') as $l);
echo memory_get_peak_usage() - $m, "n"; // Outputs 148112