Creating custom @requires annotations for PHPUnit

I was working on a project this weekend that required skipping certain tests in a particular environment (Travis CI). I wrote something like this in my base test class:

private function skipIfTravis()
{
    if (getenv('TRAVIS') === true) {
        $this->markTestSkipped('This test should not run if on Travis.');
    }
}

Then in a test, I would use it like this:

public function test_it_can_do_something_that_wont_work_on_travis()
{
    $this->skipIfTravis();

    // Do stuff...
}

This worked, but it didn't feel right. I remembered that there was a @requires annotation in PHPUnit that works natively to allow you to skip a test under a certain version of PHP or with certain extensions disabled, so I set out to write my own custom @requires block.

Note: Of course, I could've just made up a custom annotation named @skipIfTravis or something. The syntax may have been cleaner. But I was primarily interested in learning—how do PHPUnit annotations work? What does it look like to extend a pre-existing annotation? How do you not just check for the annotation, but also check its value? I'll show you what I found and then you can run willy-nilly with your own naming schemes.

The reference

The only article I could find that referenced this concept was Creating Your Own PHPUnit @requires Annotations, which got me 90% of the way there, but with a syntax I didn't particularly love.

As you can see in the example below that I copied from their site, we're extending PHPUnit's checkRequirements() method, inspecting the current annotation's requires block, and then testing our conditions;

protected function checkRequirements() {

    parent::checkRequirements();

    $annotations = $this->getAnnotations();

    foreach ( array( 'class', 'method' ) as $depth ) {

        if ( empty( $annotations[ $depth ]['requires'] ) ) {
            continue;
        }

        $requires = array_flip( $annotations[ $depth ]['requires'] );

        if ( isset( $requires['WordPress multisite'] ) && ! is_multisite() ) {
            $this->markTestSkipped( 'Multisite must be enabled.' );
        } else if ( isset( $requires['WordPress !multisite'] ) && is_multisite() ) {
            $this->markTestSkipped( 'Multisite must not be enabled.' );
        }
    }
}

So, I adapted this for my needs. I figured, if I wanted to require that the code was not running on Travis, then I'd have to add a possible value of !Travis to the @requires annotation. It already feels a bit smelly, that we're requiring a negative, but let's just roll with it for now.

As you can see in the code below, we're running the parent method, grabbing all the annotations (which are grouped by class annotations and method annotations), checking for an annotation named @requires, and looking for a value of !Travis. If so, we're checking if the TRAVIS environment variable is true, and if so, we're skipping the test.

protected function checkRequirements()
{
    parent::checkRequirements();

    $annotations = $this->getAnnotations();

    foreach (['class', 'method'] as $depth) {
        if (empty($annotations[$depth]['requires'])) {
            continue;
        }

        $requires = array_flip($annotations[$depth]['requires']);

        if (isset($requires['!Travis']) && getenv('TRAVIS') === true) {
            $this->markTestSkipped('This test does not run on Travis.');
        }
    }
}

It didn't feel quite right. I'm in love with Laravel's Collection class, and there's a simple helper that allows you to convert an array to a Collection: collect(). So I converted this array into a Collection and then used each() to replace the foreach(['class', 'method']). I also dropped the array_flip and simplified the checking and accessing of our requires blocks:

collect($this->getAnnotations())->each(function ($location) {
    if (! isset($location['requires'])) {
        return;
    }

    if (in_array('!Travis', $location['requires']) && getenv('TRAVIS') === true) {
        $this->markTestSkipped('This test does not run on Travis.');
    }
});

So, I grabbed my base TestClass and placed this code into it, and now I can annotate any test to be skipped on Travis.

Full code, annotated

// TestCase.php
// Extending parent checkRequirements method
protected function checkRequirements()
{
    parent::checkRequirements();

    // Convert the list of annotations, which can be grouped by class and/or 
    // method, to an Illuminate collection, and then act on each
    collect($this->getAnnotations())->each(function ($location) {
        // Exit early if this annotation isn't @requires
        if (! isset($location['requires'])) {
            return;
        }

        // Look for a key with our value !Travis under the @requires annotation
        if (in_array('!Travis', $location['requires']) && getenv('TRAVIS') == true) {
            $this->markTestSkipped('This test does not run on Travis.');
        }
    });
}

// SomeTest
/**
 * @requires !Travis
 */
public function test_it_does_something_i_would_like_to_skip_on_travis()
{
    // Test stuff
}

Postscript

Looking at it afterwards, I think it'd probably be better in this instance to use a custom-named annotation like @skipIfTravis instead of overloading the @requires annotation.

Let's imagine, just for a second, we had a @skipIfTravis AND a @skipIfLocal—not because that's a great idea, but just because it's an interesting opportunity to look at a broader architecture.

    // TestCase.php
    protected function checkRequirements()
    {
        parent::checkRequirements();

        collect($this->getAnnotations())->each(function ($location) {
            $this->handleTravisSkips($location);
            $this->handleLocalSkips($location);
        });
    }

    private function handleTravisSkips($location)
    {
        if (! array_key_exists(['skipIfTravis', $location)) {
            return;
        }

        if (getenv('TRAVIS') === true) {
            $this->markTestSkipped('This test does not run on Travis.');
        }
    }

    private function handleLocalSkips($location)
    {
        if (! array_key_exists('skipIfLocal', $location)) {
            return;
        }

        if (getenv('LOCAL') === true) {
            $this->markTestSkipped('This test does not run on local environments.');
        }
    }

You can probably sense a bit of a smell here, where we're duplicating the structure with the handlers.

If you're like me, you're dreaming of allowing this requirements-checker to have "annotation handlers" registered. Something like $this->registerHandler($locationKey, $classToHandleThisLocation).

And I'm sure there are all sorts of more complicated and interesting frameworks or tools already out there (if so, let me know on Twitter!) I just had a little fun with this and wanted to document it as I went along. I hope it helped someone!

Post-Postscript

I forgot to mention this, but: if you did indeed only have a single annotation, you could clean up the collection operations a bit. We're really just filtering out the options that don't meet three criteria (has a @requires annotation, !Travis in the requires annotation array, and travis environment variable true), and then marking test as skipped for those which remain. I did a bit of optimization, and then worked with Illuminate filter-guru Adam Wathan to clean it up even a bit more. Note that we're relying on Illuminate's array_get to combine the requires checking and !Travis checking, and then using and to only skip the test if the getenv returns true.

collect($this->getAnnotations())->filter(function ($location) {
    return in_array('!Travis', array_get($location, 'requires', []));
})->each(function ($location) {
    getenv('TRAVIS') and $this->markTestSkipped('This test hates Travis.');
});