What does PHPUnit Strict mode do?

Question

I am wondering about what \"strict mode is in PHPUnit\" ?

eg:

phpunit --strict

or in phpunit.xml

Answer 1

Short answer: for long running tests use an annotation to increase the allowed run time:

@large // 10 seconds
@medium // 5 seconds
@small // 1 second max <-- Default, so no point using

Long Answer:

Here is an updated set of info that was derived with the help of @Crozin.

In my case the error was that a test was taking too long (>1 second.) (Doctrine ORM schema drop + create can slow things down, see this ZendCast for what I was doing). This was causing an issue (and some output) from PHP_Invoker. Strict mode doesnt allow any output.

By Reading / Reverse engineering /usr/share/php/pear/share/pear/PHPUnit/Util/Test.php::getSize() (and getGroups() on the same class) .. I figured out there are 3 undocumented annotations we can use:

@large  // 10 seconds
@medium // 5 seconds
@small // 1 second max run time

They can be specified on the class level or on the method level. Issue #490 on the PHPUnit github hints at issues with supplying both class level and method level so YMMV if you mix them. As crozin said, the allotted time outs are 10,5,1 seconds respectively.

A alternate solution was to increase how long an invoked function is allowed to run (on my slow computer).

sudo vi /usr/share/php/pear/share/pear/PHP/Invoker.php

Increase line 1 "declare(ticks = 1);" to
    "declare(ticks = 10);" // or any higher int that meets your needs 

Here is a bunch of information about strict mode that helped me find the solution:

PHP_Invoker
A utility class for invoking callables with a timeout. This package is required to enforce test timeouts in strict mode. [PHPUnit Install Instructions]

Strict Mode Tests that do not assert anything are marked as incomplete Test that are incomplete (or skipped) yield no code coverage Slideshare by Sebastian Bergmann (slide 10)

Note
Please note that PHPUnit swallows all output that is emitted during the execution of a test. In strict mode, a test that emits output will fail. Testing output section of PHPUnit Manual