The PHP Unit Testing framework.
BSD 3-Clause "New" or "Revised" License
18997
327
2148

In the docs we find this example for withConsecutive():

<?php
use PHPUnit\Framework\TestCase;

class FooTest extends TestCase
{
    public function testFunctionCalledTwoTimesWithSpecificArguments()
    {
        $mock = $this->getMockBuilder(stdClass::class)
                     ->setMethods(['set'])
                     ->getMock();

        $mock->expects($this->exactly(2))
             ->method('set')
             ->withConsecutive(
                 [$this->equalTo('foo'), $this->greaterThan(0)],
                 [$this->equalTo('bar'), $this->greaterThan(0)]
             );

        $mock->set('foo', 21);
        $mock->set('bar', 48);
    }
}

If your class calls method set with parameter foo first and with parameter bar second, the test will pass. However, if you call it with bar first and foo second the test will fail. This leads to brittle tests (as is mentioned in a note under the at() docs).

It would be lovely to have a way to check method 'foo' is called exactly twice. Once with parameter 'x' and once with parameter 'y', without checking the order.

A concrete example is testing that certain events are dispatched when it doesn't matter in what order they are dispatched (of course sometimes you do care about the order and in that case you can use withConsecutive()).

  • expectDeprecation(), expectDeprecationMessage(), and expectDeprecationMessageMatches()
  • expectError(), expectErrorMessage(), and expectErrorMessageMatches()
  • expectNotice(), expectNoticeMessage(), and expectNoticeMessageMatches()
  • expectWarning(), expectWarningMessage(), and expectWarningMessageMatches()

In PHPUnit 9, it was possible to get a non-zero exit code from phpunit if any tests encountered a warning, notice, or deprecation (with the proper configuration). This was very helpful for strictly prohibiting all error events.

I appreciate the distinction between test outcomes and test issues in PHPUnit 10, but I find myself missing the option for the process to return a non-zero exit code when there are any issues. This is still possible for warnings with failOnWarning, but it appears that no such option exists for notices or deprecations. Perhaps it would make sense to have boolean options failOnNotice and failOnDeprecation to fill this role? (This would also provide parity with the new displayDetailsOnTestsThatTrigger* options.)

If this functionality already exists, but I just missed it, my apologies! Thanks for all the work on PHPUnit 10. I'm very excited to start using it (especially for the code coverage color/css options, as a colorblind person).

Q A
PHPUnit version 10.0.16
PHP version 8.2.3
Installation Method Composer

Summary

When a test is using DataProvider attribute, it throws an error during execution.

Current behavior

Following error is thrown during execution:

/var/www/recon # vendor/bin/phpunit tests/Unit/BadNumberTest.php

An error occurred inside PHPUnit.

Message:  Attribute class "PHPUnit\Framework\Attributes\DataProvider" not found
Location: /var/www/recon/vendor/phpunit/phpunit/src/Metadata/Parser/AttributeParser.php:310

#0 /var/www/recon/vendor/phpunit/phpunit/src/Metadata/Parser/AttributeParser.php(310): ReflectionAttribute->newInstance()
#1 /var/www/recon/vendor/phpunit/phpunit/src/Metadata/Parser/ParserChain.php(48): PHPUnit\Metadata\Parser\AttributeParser->forMethod('Tests\\Unit\\BadN...', 'testUsage')     
#2 /var/www/recon/vendor/phpunit/phpunit/src/Metadata/Parser/CachingParser.php(57): PHPUnit\Metadata\Parser\ParserChain->forMethod('Tests\\Unit\\BadN...', 'testUsage')       
#3 /var/www/recon/vendor/phpunit/phpunit/src/Metadata/Api/DataProvider.php(57): PHPUnit\Metadata\Parser\CachingParser->forMethod('Tests\\Unit\\BadN...', 'testUsage')
#4 /var/www/recon/vendor/phpunit/phpunit/src/Framework/TestBuilder.php(38): PHPUnit\Metadata\Api\DataProvider->providedData('Tests\\Unit\\BadN...', 'testUsage')
#5 /var/www/recon/vendor/phpunit/phpunit/src/Framework/TestSuite.php(490): PHPUnit\Framework\TestBuilder->build(Object(ReflectionClass), 'testUsage')
#6 /var/www/recon/vendor/phpunit/phpunit/src/Framework/TestSuite.php(131): PHPUnit\Framework\TestSuite->addTestMethod(Object(ReflectionClass), Object(ReflectionMethod))      
#7 /var/www/recon/vendor/phpunit/phpunit/src/TextUI/Configuration/TestSuiteBuilder.php(96): PHPUnit\Framework\TestSuite::fromClassReflector(Object(ReflectionClass))
#8 /var/www/recon/vendor/phpunit/phpunit/src/TextUI/Configuration/TestSuiteBuilder.php(46): PHPUnit\TextUI\Configuration\TestSuiteBuilder->testSuiteFromPath('/var/www/recon/...', Array)
#9 /var/www/recon/vendor/phpunit/phpunit/src/TextUI/Application.php(303): PHPUnit\TextUI\Configuration\TestSuiteBuilder->build(Object(PHPUnit\TextUI\Configuration\Configuration))
#10 /var/www/recon/vendor/phpunit/phpunit/src/TextUI/Application.php(99): PHPUnit\TextUI\Application->buildTestSuite(Object(PHPUnit\TextUI\Configuration\Configuration))      
#11 /var/www/recon/vendor/phpunit/phpunit/phpunit(90): PHPUnit\TextUI\Application->run(Array)
#12 /var/www/recon/vendor/bin/phpunit(123): include('/var/www/recon/...')
#13 {main}

How to reproduce

Minimal reproduction test:

<?php

declare(strict_types=1);

namespace Tests\Unit;

use PHPUnit\Framework\Attributes\DataProvider;
use PHPUnit\Framework\TestCase;

final class BadNumberTest extends TestCase
{
    #[DataProvider('myProvider')]
    public function testUsage(string $expected, string $input): void
    {
        $this->assertEmpty('');
    }

    public static function myProvider(): array
    {
        return [['', '']];
    }
}

If we remove the attribute and function arguments, test will work just fine.

Expected behavior

Test is executed

Composer

PHPUnit is not wrapped in a framework, I just use packages from Symfony and CakePHP.

composer info | sort
cakephp/cache                      4.4.11           Easy to use Caching library with support for multiple caching backends
cakephp/chronos                    2.3.2            A simple API extension for DateTime.
cakephp/collection                 4.4.11           Work easily with arrays and iterators by having a battery of utility traversal methods
cakephp/core                       4.4.11           CakePHP Framework Core classes
cakephp/database                   4.4.11           Flexible and powerful Database abstraction library with a familiar PDO-like API
cakephp/datasource                 4.4.11           Provides connection managing and traits for Entities and Queries that can be reused for different datastores
cakephp/event                      4.4.11           CakePHP event dispatcher library that helps implementing the observer pattern
cakephp/i18n                       4.4.11           CakePHP Internationalization library with support for messages translation and dates and numbers localization
cakephp/log                        4.4.11           CakePHP logging library with support for multiple different streams
cakephp/orm                        4.4.11           CakePHP ORM - Provides a flexible and powerful ORM implementing a data-mapper pattern.
cakephp/utility                    4.4.11           CakePHP Utility classes such as Inflector, String, Hash, and Security
cakephp/validation                 4.4.11           CakePHP Validation library
composer/pcre                      3.1.0            PCRE wrapping library that offers type-safe preg_* replacements.
composer/semver                    3.3.2            Semver library that offers utilities, version constraint parsing and validation.
composer/xdebug-handler            3.0.3            Restarts a process without Xdebug.
doctrine/annotations               2.0.1            Docblock Annotations Parser
doctrine/inflector                 2.0.6            PHP Doctrine Inflector is a small library that can perform string manipulations with regard to upper/lowercase and ...    
doctrine/lexer                     3.0.0            PHP Doctrine Lexer parser library that can be used in Top-Down, Recursive Descent Parsers.
dragonmantank/cron-expression      v3.3.2           CRON for PHP: Calculate the next or previous run date and determine if a CRON expression is due
egulias/email-validator            4.0.1            A library for validating emails against several RFCs
ezyang/htmlpurifier                v4.16.0          Standards compliant HTML filter written in PHP
francerz/enum                      v0.1.1           PHP Enumeration tool
francerz/php-power-data            v0.1.28          PHP data structures for complex data handling
francerz/sql-builder               v0.5.10          SQL Builder
friendsofphp/php-cs-fixer          v3.15.1          A tool to automatically fix PHP code style
gregwar/gnuplot                    v1.0.2           GnuPlot library
guzzlehttp/guzzle                  7.5.0            Guzzle is a PHP HTTP client library
guzzlehttp/promises                1.5.2            Guzzle promises library
guzzlehttp/psr7                    2.4.4            PSR-7 message implementation that also provides common utility methods
illuminate/collections             v10.3.3          The Illuminate Collections package.
illuminate/conditionable           v10.3.3          The Illuminate Conditionable package.
illuminate/container               v10.3.3          The Illuminate Container package.
illuminate/contracts               v10.3.3          The Illuminate Contracts package.
illuminate/macroable               v10.3.3          The Illuminate Macroable package.
illuminate/support                 v10.3.3          The Illuminate Support package.
league/csv                         9.9.0            CSV data manipulation made easy in PHP
lorisleiva/cron-translator         v0.4.2           Makes CRON expressions human-readable
maennchen/zipstream-php            v2.4.0           ZipStream is a library for dynamically streaming dynamic zip files from PHP without writing to the disk at all on t...    
markbaker/complex                  3.0.2            PHP Class for working with complex numbers
markbaker/matrix                   3.0.1            PHP Class for working with matrices
mjphaynes/php-resque               v4.x-dev 48c81ea Redis backed library for creating background jobs and processing them later.
monolog/monolog                    2.9.1            Sends your logs to files, sockets, inboxes, databases and various web services
myclabs/deep-copy                  1.11.1           Create deep copies (clones) of your objects
myclabs/php-enum                   1.8.4            PHP Enum implementation
napp/salesforce-api                1.0.7            A library for interacting with the Salesforce REST API and managing the OAuth flow
nesbot/carbon                      2.66.0           An API extension for DateTime that supports 281 different languages.
nikic/php-parser                   v4.15.4          A PHP parser written in PHP
phar-io/manifest                   2.0.3            Component for reading phar.io manifest information from a PHP Archive (PHAR)
phar-io/version                    3.2.1            Library for handling version information and constraints
phpoffice/phpspreadsheet           1.28.0           PHPSpreadsheet - Read, Create and Write Spreadsheet documents in PHP - Spreadsheet engine
phpspec/php-diff                   v1.1.3           A comprehensive library for generating differences between two hashable objects (strings or arrays).
phpstan/phpstan                    1.10.6           PHPStan - PHP Static Analysis Tool
phpunit/php-code-coverage          10.0.2           Library that provides collection, processing, and rendering functionality for PHP code coverage information.
phpunit/php-file-iterator          4.0.1            FilterIterator implementation that filters files based on a list of suffixes.
phpunit/php-invoker                4.0.0            Invoke callables with a timeout
phpunit/php-text-template          3.0.0            Simple template engine.
phpunit/php-timer                  6.0.0            Utility class for timing
phpunit/phpunit                    10.0.16          The PHP Unit Testing framework.
predis/predis                      v1.1.10          Flexible and feature-complete Redis client for PHP and HHVM
psr/cache                          3.0.0            Common interface for caching libraries
psr/container                      2.0.2            Common Container Interface (PHP FIG PSR-11)
psr/event-dispatcher               1.0.0            Standard interfaces for event handling.
psr/http-client                    1.0.1            Common interface for HTTP clients
psr/http-factory                   1.0.1            Common interfaces for PSR-7 HTTP message factories
psr/http-message                   1.0.1            Common interface for HTTP messages
psr/log                            2.0.0            Common interface for logging libraries
psr/simple-cache                   2.0.0            Common interfaces for simple caching
ralouphie/getallheaders            3.0.3            A polyfill for getallheaders.
robmorgan/phinx                    0.13.4           Phinx makes it ridiculously easy to manage the database migrations for your PHP app.
sebastian/cli-parser               2.0.0            Library for parsing CLI options
sebastian/code-unit                2.0.0            Collection of value objects that represent the PHP code units
sebastian/code-unit-reverse-lookup 3.0.0            Looks up which function or method a line of code belongs to
sebastian/comparator               5.0.0            Provides the functionality to compare PHP values for equality
sebastian/complexity               3.0.0            Library for calculating the complexity of PHP code units
sebastian/diff                     5.0.0            Diff implementation
sebastian/environment              6.0.0            Provides functionality to handle HHVM/PHP environments
sebastian/exporter                 5.0.0            Provides the functionality to export PHP variables for visualization
sebastian/global-state             6.0.0            Snapshotting of global state
sebastian/lines-of-code            2.0.0            Library for counting the lines of code in PHP source code
sebastian/object-enumerator        5.0.0            Traverses array structures and object graphs to enumerate all referenced objects
sebastian/object-reflector         3.0.0            Allows reflection of object attributes, including inherited and non-public ones
sebastian/recursion-context        5.0.0            Provides functionality to recursively process PHP variables
sebastian/type                     4.0.0            Collection of value objects that represent the types of the PHP type system
sebastian/version                  4.0.1            Library that helps with managing the version number of Git-hosted PHP projects
symfony/config                     v6.2.7           Helps you find, load, combine, autofill and validate configuration values of any kind
symfony/console                    v6.2.7           Eases the creation of beautiful and testable command line interfaces
symfony/deprecation-contracts      v3.2.1           A generic function and convention to trigger deprecation notices
symfony/dotenv                     v6.2.7           Registers environment variables from a .env file
symfony/error-handler              v6.2.7           Provides tools to manage errors and ease debugging PHP code
symfony/event-dispatcher           v6.2.7           Provides tools that allow your application components to communicate with each other by dispatching events and list...    
symfony/event-dispatcher-contracts v3.2.1           Generic abstractions related to dispatching event
symfony/filesystem                 v6.2.7           Provides basic utilities for the filesystem
symfony/finder                     v6.2.7           Finds files and directories via an intuitive fluent interface
symfony/google-mailer              v6.2.7           Symfony Google Mailer Bridge
symfony/http-foundation            v6.2.7           Defines an object-oriented layer for the HTTP specification
symfony/http-kernel                v6.2.7           Provides a structured process for converting a Request into a Response
symfony/mailer                     v6.2.7           Helps sending emails
symfony/mime                       v6.2.7           Allows manipulating MIME messages
symfony/options-resolver           v6.2.7           Provides an improved replacement for the array_replace PHP function
symfony/polyfill-ctype             v1.27.0          Symfony polyfill for ctype functions
symfony/polyfill-intl-grapheme     v1.27.0          Symfony polyfill for intl's grapheme_* functions
symfony/polyfill-intl-idn          v1.27.0          Symfony polyfill for intl's idn_to_ascii and idn_to_utf8 functions
symfony/polyfill-intl-normalizer   v1.27.0          Symfony polyfill for intl's Normalizer class and related functions
symfony/polyfill-mbstring          v1.27.0          Symfony polyfill for the Mbstring extension
symfony/polyfill-php72             v1.27.0          Symfony polyfill backporting some PHP 7.2+ features to lower PHP versions
symfony/polyfill-php80             v1.27.0          Symfony polyfill backporting some PHP 8.0+ features to lower PHP versions
symfony/polyfill-php81             v1.27.0          Symfony polyfill backporting some PHP 8.1+ features to lower PHP versions
symfony/process                    v6.2.7           Executes commands in sub-processes
symfony/routing                    v6.2.7           Maps an HTTP request to a set of configuration variables
symfony/service-contracts          v3.2.1           Generic abstractions related to writing services
symfony/stopwatch                  v6.2.7           Provides a way to profile code
symfony/string                     v6.2.7           Provides an object-oriented API to strings and deals with bytes, UTF-8 code points and grapheme clusters in a unifi...    
symfony/translation                v6.2.7           Provides tools to internationalize your application
symfony/translation-contracts      v3.2.1           Generic abstractions related to translation
symfony/twig-bridge                v6.2.7           Provides integration for Twig with various Symfony components
symfony/var-dumper                 v6.2.7           Provides mechanisms for walking through any arbitrary PHP variable
symfony/yaml                       v6.2.7           Loads and dumps YAML files
theseer/tokenizer                  1.2.1            A small library for converting tokenized PHP source code into XML and potentially other formats
troydavisson/phrets                2.6.3            RETS library in PHP
twig/twig                          v3.5.1           Twig, the flexible, fast, and secure template language for PHP
voku/portable-ascii                2.0.1            Portable ASCII library - performance optimized (ascii) string functions for php.
webmozart/assert                   1.11.0           Assertions to validate method input/output with nice error messages.
wyndow/fuzzywuzzy                  v0.6.0           Fuzzy string matching based on FuzzyWuzzy from Seatgeek

After updating to PHPUnit 10, I have noticed it prints some warnings when tests are using symbols marked as deprecated.

While I think this might be useful when it comes to third party code (although I'm not sure if PHPUnit is the kind of tool that should highlight this, and it might fall more into the static analysis world, but that's a different story), it is a bit inconvenient when it complains about deprecated symbols from the project itself.

I have a couple of deprecated things that I intend to remove on the next major version, but while they are in the project, I also want to keep their tests around. When I remove these, I will remove their tests as well.

And I know this is not making the tests fail in any way, but it can distract you from actual warnings that should be addressed.

Because of this, it would be great to be able to either disable this check entirely (making it behave as PHPUnit 9), or disable it for certain namespaces/paths.

We have this code (PHPUnit 9.6, PHPUnit 10.0)

$result = '';

foreach (ini_get_all(null, false) as $key => $value) {
    $result .= sprintf(
        '@ini_set(%s, %s);' . "\n",
        self::exportVariable($key),
        self::exportVariable((string) $value)
    );
}

to generate code that when run in a separate process reproduces the PHP configuration in the parent process.

This can, for instance, lead to

@ini_set('mbstring.internal_encoding', '');

being run, which triggers a warning because mbstring.internal_encoding is deprecated.

We should avoid generating ini_set() calls for values that are not configured in the parent process.

It is unclear whether $value === null can be safely used to detect whether a value is configured or not.

As the error suppression operator @ is used, this should only ever be a problem when a custom error handler does not honor the fact that such a warning should be ignored.

Q A
PHPUnit version 8.5.0
PHP version 7.2.24
Installation Method Composer

This issue is related to the MockBuilder::setMethods() method being deprecated, and replaced with addMethods() and onlyMethods() (detailed in issue 3911).

I currently have this code for creating a mock "PaymentGateway" object with a stubbed "charge" method, and it works:

$gateway = $this->getMockBuilder('PaymentGateway')
                ->setMethods(['charge'])
                ->getMock();

However, with setMethods being deprecated, I've tried to replace it with addMethods:

$gateway = $this->getMockBuilder('PaymentGateway')
                ->addMethods(['charge'])
                ->getMock();

However this gives me a Class PaymentGateway does not exist warning.

Does the class actually need to exist before we can mock it? Or am I misunderstanding how addMethods works?

Summary

Method getAll is not covered with beStrictAboutCoverageMetadata="true". It works without this option enabled.

Current behavior

class A
{
    /**
     * @return array<array{id: string, label: string}>
     */
    public function getAll(): array
    {
        return [
            [
                'id' => 1,
                'label' => 'test',
            ],
            // other values
        ];
    }
}


#[CoversClass(A::class)]
class ATest extends TestCase
{
    private A $a;

    public function setUp(): void
    {
        $this->a = new A();
    }

    public function testGetAll(): void
    {
        $data = $this->a->getAll();

        $this->assertIsArray($data);

        foreach ($data as $item) {
            self::assertArrayHasKey('id', $item);
            self::assertIsString($item['id']);

            self::assertArrayHasKey('label', $item);
            self::assertIsString($item['label']);
        }
    }
}

phpunit.xml

<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/10.0/phpunit.xsd"
         bootstrap="tests/bootstrap.php"
         executionOrder="depends,defects"
         beStrictAboutOutputDuringTests="true"
         beStrictAboutTestsThatDoNotTestAnything="true"
         backupGlobals="false"
         colors="true"
         cacheDirectory="./tmp/phpunit/.phpunit.cache"
         backupStaticProperties="false"
         requireCoverageMetadata="true"
         beStrictAboutCoverageMetadata="true"
         displayDetailsOnIncompleteTests="true"
         displayDetailsOnTestsThatTriggerErrors="true"
         displayDetailsOnTestsThatTriggerWarnings="true"
         displayDetailsOnTestsThatTriggerNotices="true"
         displayDetailsOnTestsThatTriggerDeprecations="true"
         displayDetailsOnSkippedTests="true"
>
    <coverage>
        <include>
            <directory suffix=".php">./src</directory>
        </include>
    </coverage>
    <php>
        <ini name="memory_limit" value="512M"/>
    </php>
</phpunit>

CMD

./vendor/bin/phpunit -c phpunit.xml --log-junit report.xml --coverage-text --coverage-cobertura=coverage.cobertura.xml tests

Thank you!

Expected behavior

Have the method covered.

Q A
PHPUnit version 10.0.16
PHP version 8.2.3
Installation Method Composer

Summary

Not sure to understand what happen, but since I updated one of my project to PHPUnit 10, I got some conflicts with test classes having same names but in different namespaces (I'm using some obscur static mecanisms).

Let say class \MyTests\A\MyClassTest and \MyTests\B\MyClassTest having common \MyTests\C\MyTestCasse parent for example.

I tried to investigate a little bit, and found this line is overriding $suiteClassName using first loaded class having same name, whatever namespace is. Is it expected ?

Thanks!

Q A
PHPUnit version 10.0.17
PHP version 8.1.14
Installation Method Composer

Summary

In PHPUnit 10, the @dataProvider annotation got unexpectedly stricter. Specifically, in previous versions I commonly wrote this:

/**
 * @dataProvider provider()
 */
public function testThing(int $a, int $b) { ... }

public function provider(): iterable { ... }

The () on the end of the provider method was ignored. As of v10, it is no longer ignored. This is going to create... a lot of drudge work changing all of my provider references to remove the ().

Ideally they'd convert to attributes at some point, but annotations are still supported for now and this was an unexpected snag.

Current behavior

The code example above gives an error along the lines of:

1) Crell\Tutorials\RectangleTest::testThing
The data provider specified for Crell\Tutorials\RectangleTest::testThing is invalid
Method Crell\Tutorials\RectangleTest::provider()() does not exist

(Note the double () in what it thinks is the method name, indicating that v10 thinks the () is part of the method name.)

How to reproduce

See code example above.

Expected behavior

As in v9, the () should be stripped off and ignored. Or if that's not feasible, this would be something to mention in the upgrading docs as it's an unexpected break.

I would like to introduce a --log-xml <file> CLI option as well as a <logging><xml outputFile="logfile.xml"/> XML configuration file setting to generate logfiles in a new XML format.information.

The goal of this is to have a XML-based test result format that we control. JUnit XML is outside of our control and not really a standard to begin with.

In our project's testRoutes_shouldDenyCustomersAccessToAdminPages() test, we have a for-loop that iterates each and every "admin" prefixed route of our PHP App.

Now sometimes an error is thrown, or the assertion fails, and we would like to know for which route exactly, for that, said for-loop stores the context in lastRoute instance-variable of test-case, and TestListener prefixes the logged message with said context, like:

public function addError(Test $test, \Throwable $t, float $time): void
{
    $this->onThrow($test, $t);
}
public function addFailure(Test $test, AssertionFailedError $error, float $time): void
{
    $this->onThrow($test, $error);
}

private function onThrow(Test $test, \Throwable $error)
{
    if ($test instanceof MyTest) {
        if ($test->lastRoute !== null) {
            self::prefixMessage($error, 'While testing route = ' . $test->lastRoute . ":\n");
        }
    }
}

private static function prefixMessage($obj, $prefix)
{
    try {
        $reflection = new \ReflectionClass($obj);
        $property = $reflection->getProperty('message');
        $property->setAccessible(true);
        $property->setValue($obj, $prefix . $property->getValue($obj));
    } catch (\ReflectionException $e) {
    }
}

@sebastianbergmann How can we achive above with the new TestHook API?
Which looks like:

interface AfterTestErrorHook extends TestHook
{
    public function executeAfterTestError(string $test, string $message, float $time): void;
}

interface AfterTestFailureHook extends TestHook
{
    public function executeAfterTestFailure(string $test, string $message, float $time): void;
}

I mean, we could Make the lastRoute a static-variable (instead of instance-variable).
But how can we alter the $message?

Solutions

As you deprecated our side of handling this,
please consider implementing one of below solutions into PHPUnit API.

# 1 You could change your existing API's $message to &$message, I mean, to a reference, then use reflection like above (to update message).

# 2 Or, like some other frameworks, provide withContext(...) method, and the next failure (no matter if error or assert) should log the context before actual message, then clear context.

Also, test-runner should clear context after each test-method (of course).

Q A
PHPUnit version 9.5.20
PHP version 8.1.5
Installation Method Composer

Summary

I have a some phpt-tests. In Github Actions they produce the expected output and work fine, but locally the produce different output. The output contains the added line "Standard input code:5:\n" with the number changing between tests. The bug occured, when we switched from phive to using composer. Here is a composer info output from my reproducer:

composer info | sort

doctrine/instantiator              1.4.1   A small, lightweight utility to instantiate objects in PHP without invoking their constructors
myclabs/deep-copy                  1.11.0  Create deep copies (clones) of your objects
nikic/php-parser                   v4.13.2 A PHP parser written in PHP
phar-io/manifest                   2.0.3   Component for reading phar.io manifest information from a PHP Archive (PHAR)
phar-io/version                    3.2.1   Library for handling version information and constraints
phpdocumentor/reflection-common    2.2.0   Common reflection classes used by phpdocumentor to reflect the code structure
phpdocumentor/reflection-docblock  5.3.0   With this component, a library can provide support for annotations via DocBlocks or otherwise retrieve information that is embedded ...
phpdocumentor/type-resolver        1.6.1   A PSR-5 based resolver of Class names, Types and Structural Element Names
phpspec/prophecy                   v1.15.0 Highly opinionated mocking framework for PHP 5.3+
phpunit/php-code-coverage          9.2.15  Library that provides collection, processing, and rendering functionality for PHP code coverage information.
phpunit/php-file-iterator          3.0.6   FilterIterator implementation that filters files based on a list of suffixes.
phpunit/php-invoker                3.1.1   Invoke callables with a timeout
phpunit/php-text-template          2.0.4   Simple template engine.
phpunit/php-timer                  5.0.3   Utility class for timing
phpunit/phpunit                    9.5.20  The PHP Unit Testing framework.
sebastian/cli-parser               1.0.1   Library for parsing CLI options
sebastian/code-unit                1.0.8   Collection of value objects that represent the PHP code units
sebastian/code-unit-reverse-lookup 2.0.3   Looks up which function or method a line of code belongs to
sebastian/comparator               4.0.6   Provides the functionality to compare PHP values for equality
sebastian/complexity               2.0.2   Library for calculating the complexity of PHP code units
sebastian/diff                     4.0.4   Diff implementation
sebastian/environment              5.1.4   Provides functionality to handle HHVM/PHP environments
sebastian/exporter                 4.0.4   Provides the functionality to export PHP variables for visualization
sebastian/global-state             5.0.5   Snapshotting of global state
sebastian/lines-of-code            1.0.3   Library for counting the lines of code in PHP source code
sebastian/object-enumerator        4.0.4   Traverses array structures and object graphs to enumerate all referenced objects
sebastian/object-reflector         2.0.4   Allows reflection of object attributes, including inherited and non-public ones
sebastian/recursion-context        4.0.4   Provides functionality to recursively process PHP variables
sebastian/resource-operations      3.0.3   Provides a list of PHP built-in functions that operate on resources
sebastian/type                     3.0.0   Collection of value objects that represent the types of the PHP type system
sebastian/version                  3.0.2   Library that helps with managing the version number of Git-hosted PHP projects
symfony/polyfill-ctype             v1.25.0 Symfony polyfill for ctype functions
theseer/tokenizer                  1.2.1   A small library for converting tokenized PHP source code into XML and potentially other formats
webmozart/assert                   1.10.0  Assertions to validate method input/output with nice error messages.

Current behavior

Locally phpt-tests contain unexpected output "Standard input code:..."

How to reproduce

I have created a minimal reproducer: https://github.com/dbrumann/phpt-reproducer

Test file (tests/Example.phpt):

--TEST--
Reproduce phpt adding output locally
--FILE--
<?php

var_dump('test');

--EXPECT--
string(4) "test"

phpunit.xml:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
    bootstrap="vendor/autoload.php"
    colors="true"
>
    <testsuites>
        <testsuite name="phpt">
            <directory suffix=".phpt">./tests</directory>
        </testsuite>
    </testsuites>
    <coverage>
        <include>
            <directory>./src</directory>
        </include>
    </coverage>
</phpunit>

Expected behavior

Test passes, but instead I get an error because of the added output:

1) /Users/dbr/Projects/phpt-reproducer/tests/Example.phpt
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'string(4) "test"'
+'Standard input code:3:\n
+string(4) "test"'
Q A
PHPUnit version 10.0.16
PHP version 8.2
Installation Method Composer

Summary

Output with displayDetailsOnTestsThatTriggerDeprecations="true" only includes the Deprecation in the tests summary, but does not list the file location.

Current behavior

Deprecations mentioned in summary when run in PHPStorm. The same issue is also happening with testdox enabled.

OK, but some tests have issues!
Tests: 627, Assertions: 1355, Deprecations: 1.

How to reproduce

This is my configuration:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit
        bootstrap="PHPUnitBootstrap.php"
        backupGlobals="false"
        colors="true"
        cacheDirectory=".phpunit.cache"
        displayDetailsOnTestsThatTriggerDeprecations="true"
        displayDetailsOnTestsThatTriggerErrors="true"
        displayDetailsOnTestsThatTriggerNotices="true"
        displayDetailsOnTestsThatTriggerWarnings="true"
>

Expected behavior

The expected behaviour is a list of the deprecations and which files they reside in, so they can be fixed!

Q A
PHPUnit version 10.0.16
PHP version 8.2.3
Installation Method Composer
amphp/amp                            v2.6.2           A non-blocking concurrency framework for PHP applications.
amphp/byte-stream                    v1.8.1           A stream abstraction to make working with non-blocking I/O simple.
amphp/parallel                       v1.4.2           Parallel processing component for Amp.
amphp/parallel-functions             v1.1.0           Parallel processing made simple.
amphp/parser                         v1.1.0           A generator parser to make streaming parsers simple.
amphp/process                        v1.1.4           Asynchronous process manager.
amphp/serialization                  v1.0.0           Serialization tools for IPC and data storage in PHP.
amphp/sync                           v1.4.2           Mutex, Semaphore, and other synchronization tools for Amp.
barryvdh/laravel-ide-helper          v2.13.0          Laravel IDE Helper, generates correct PHPDocs for all Facade classes, to improve auto-completion.
barryvdh/reflection-docblock         v2.1.0          
brick/math                           0.11.0           Arbitrary-precision arithmetic library
composer/class-map-generator         1.0.0            Utilities to scan PHP code and generate class maps.
composer/pcre                        3.1.0            PCRE wrapping library that offers type-safe preg_* replacements.
composer/semver                      3.3.2            Semver library that offers utilities, version constraint parsing and validation.
composer/xdebug-handler              3.0.3            Restarts a process without Xdebug.
dflydev/dot-access-data              v3.0.2           Given a deep data structure, access data by dot notation.
doctrine/annotations                 2.0.1            Docblock Annotations Parser
doctrine/cache                       2.2.0            PHP Doctrine Cache library is a popular cache implementation that supports many different drivers such as redis, memcache, apc, mongodb and others.
doctrine/collections                 2.1.2            PHP Doctrine Collections library that adds additional functionality on top of PHP arrays.
doctrine/dbal                        3.6.1            Powerful PHP database abstraction layer (DBAL) with many features for database schema introspection and management.
doctrine/deprecations                v1.0.0           A small layer on top of trigger_error(E_USER_DEPRECATED) or PSR-3 logging with options to disable all deprecations or selectively for packages.
doctrine/event-manager               2.0.0            The Doctrine Event Manager is a simple PHP event system that was built to be used with the various Doctrine projects.
doctrine/inflector                   2.0.6            PHP Doctrine Inflector is a small library that can perform string manipulations with regard to upper/lowercase and singular/plural forms of words.
doctrine/lexer                       3.0.0            PHP Doctrine Lexer parser library that can be used in Top-Down, Recursive Descent Parsers.
dragonmantank/cron-expression        v3.3.2           CRON for PHP: Calculate the next or previous run date and determine if a CRON expression is due
egulias/email-validator              4.0.1            A library for validating emails against several RFCs
fakerphp/faker                       v1.21.0          Faker is a PHP library that generates fake data for you.
friendsofphp/php-cs-fixer            v3.15.1          A tool to automatically fix PHP code style
fruitcake/php-cors                   v1.2.0           Cross-origin resource sharing library for the Symfony HttpFoundation
gitonomy/gitlib                      v1.3.7           Library for accessing git
graham-campbell/result-type          v1.1.1           An Implementation Of The Result Type
guzzlehttp/uri-template              v1.0.1           A polyfill class for uri_template of PHP
laravel/framework                    v10.4.1          The Laravel Framework.
laravel/serializable-closure         v1.3.0           Laravel Serializable Closure provides an easy and secure way to serialize closures in PHP.
league/commonmark                    2.3.9            Highly-extensible PHP Markdown parser which fully supports the CommonMark spec and GitHub-Flavored Markdown (GFM)
league/config                        v1.2.0           Define configuration arrays with strict schemas and access values with dot notation
league/flysystem                     3.12.3           File storage abstraction for PHP
league/fractal                       0.20.1           Handle the output of complex data structures ready for API output.
league/mime-type-detection           1.11.0           Mime-type detection for Flysystem
matanyadaev/laravel-eloquent-spatial 3.1.2            Spatial library for Laravel
monolog/monolog                      3.3.1            Sends your logs to files, sockets, inboxes, databases and various web services
myclabs/deep-copy                    1.11.1           Create deep copies (clones) of your objects
nesbot/carbon                        2.66.0           An API extension for DateTime that supports 281 different languages.
nette/schema                         v1.2.3           ๐Ÿ“ Nette Schema: validating data structures against a given Schema.
nette/utils                          v4.0.0           ๐Ÿ›   Nette Utils: lightweight utilities for string & array manipulation, image handling, safe JSON encoding/decoding, validation, slug or strong password generatin...
nikic/php-parser                     v4.15.4          A PHP parser written in PHP
nunomaduro/termwind                  v1.15.1          Its like Tailwind CSS, but for the console.
ondram/ci-detector                   4.1.0            Detect continuous integration environment and provide unified access to properties of current build
phar-io/manifest                     2.0.3            Component for reading phar.io manifest information from a PHP Archive (PHAR)
phar-io/version                      3.2.1            Library for handling version information and constraints
phayes/geophp                        1.2              GeoPHP is a open-source native PHP library for doing geometry operations. It is written entirely in PHP and can therefore run on shared hosts. It can read and write...
phpdocumentor/reflection-common      2.2.0            Common reflection classes used by phpdocumentor to reflect the code structure
phpdocumentor/type-resolver          1.7.0            A PSR-5 based resolver of Class names, Types and Structural Element Names
phpoption/phpoption                  1.9.1            Option Type for PHP
phpro/grumphp                        v1.15.0          A composer plugin that enables source code quality checks.
phpstan/phpdoc-parser                1.16.1           PHPDoc parser with support for nullable, intersection and generic types
phpunit/php-code-coverage            10.0.2           Library that provides collection, processing, and rendering functionality for PHP code coverage information.
phpunit/php-file-iterator            4.0.1            FilterIterator implementation that filters files based on a list of suffixes.
phpunit/php-invoker                  4.0.0            Invoke callables with a timeout
phpunit/php-text-template            3.0.0            Simple template engine.
phpunit/php-timer                    6.0.0            Utility class for timing
phpunit/phpunit                      10.0.16          The PHP Unit Testing framework.
psr/cache                            3.0.0            Common interface for caching libraries
psr/container                        2.0.2            Common Container Interface (PHP FIG PSR-11)
psr/event-dispatcher                 1.0.0            Standard interfaces for event handling.
psr/log                              3.0.0            Common interface for logging libraries
psr/simple-cache                     3.0.0            Common interfaces for simple caching
ramsey/collection                    2.0.0            A PHP library for representing and manipulating collections.
ramsey/uuid                          4.x-dev bf2bee2  A PHP library for generating and working with universally unique identifiers (UUIDs).
sebastian/cli-parser                 2.0.0            Library for parsing CLI options
sebastian/code-unit                  2.0.0            Collection of value objects that represent the PHP code units
sebastian/code-unit-reverse-lookup   3.0.0            Looks up which function or method a line of code belongs to
sebastian/comparator                 5.0.0            Provides the functionality to compare PHP values for equality
sebastian/complexity                 3.0.0            Library for calculating the complexity of PHP code units
sebastian/diff                       5.0.0            Diff implementation
sebastian/environment                6.0.0            Provides functionality to handle HHVM/PHP environments
sebastian/exporter                   5.0.0            Provides the functionality to export PHP variables for visualization
sebastian/global-state               6.0.0            Snapshotting of global state
sebastian/lines-of-code              2.0.0            Library for counting the lines of code in PHP source code
sebastian/object-enumerator          5.0.0            Traverses array structures and object graphs to enumerate all referenced objects
sebastian/object-reflector           3.0.0            Allows reflection of object attributes, including inherited and non-public ones
sebastian/recursion-context          5.0.0            Provides functionality to recursively process PHP variables
sebastian/type                       4.0.0            Collection of value objects that represent the types of the PHP type system
sebastian/version                    4.0.1            Library that helps with managing the version number of Git-hosted PHP projects
seld/jsonlint                        1.9.0            JSON Linter
spatie/fractalistic                  2.9.5            A developer friendly wrapper around Fractal
spatie/laravel-fractal               6.0.3            An easy to use Fractal integration for Laravel applications
spatie/laravel-package-tools         1.14.2           Tools for creating Laravel packages
symfony/config                       v6.2.7           Helps you find, load, combine, autofill and validate configuration values of any kind
symfony/console                      v6.2.7           Eases the creation of beautiful and testable command line interfaces
symfony/css-selector                 v6.2.7           Converts CSS selectors to XPath expressions
symfony/dependency-injection         v6.2.7           Allows you to standardize and centralize the way objects are constructed in your application
symfony/deprecation-contracts        v3.2.1           A generic function and convention to trigger deprecation notices
symfony/dotenv                       v6.2.7           Registers environment variables from a .env file
symfony/error-handler                v6.2.7           Provides tools to manage errors and ease debugging PHP code
symfony/event-dispatcher             v6.2.7           Provides tools that allow your application components to communicate with each other by dispatching events and listening to them
symfony/event-dispatcher-contracts   v3.2.1           Generic abstractions related to dispatching event
symfony/filesystem                   v6.2.7           Provides basic utilities for the filesystem
symfony/finder                       v6.2.7           Finds files and directories via an intuitive fluent interface
symfony/http-foundation              v6.2.7           Defines an object-oriented layer for the HTTP specification
symfony/http-kernel                  v6.2.7           Provides a structured process for converting a Request into a Response
symfony/mailer                       v6.2.7           Helps sending emails
symfony/mime                         v6.2.7           Allows manipulating MIME messages
symfony/options-resolver             v6.2.7           Provides an improved replacement for the array_replace PHP function
symfony/polyfill-ctype               v1.27.0          Symfony polyfill for ctype functions
symfony/polyfill-intl-grapheme       v1.27.0          Symfony polyfill for intl's grapheme_* functions
symfony/polyfill-intl-idn            v1.27.0          Symfony polyfill for intl's idn_to_ascii and idn_to_utf8 functions
symfony/polyfill-intl-normalizer     v1.27.0          Symfony polyfill for intl's Normalizer class and related functions
symfony/polyfill-mbstring            v1.27.0          Symfony polyfill for the Mbstring extension
symfony/polyfill-php72               v1.27.0          Symfony polyfill backporting some PHP 7.2+ features to lower PHP versions
symfony/polyfill-php80               v1.27.0          Symfony polyfill backporting some PHP 8.0+ features to lower PHP versions
symfony/polyfill-php81               v1.27.0          Symfony polyfill backporting some PHP 8.1+ features to lower PHP versions
symfony/polyfill-uuid                v1.27.0          Symfony polyfill for uuid functions
symfony/process                      v6.2.7           Executes commands in sub-processes
symfony/routing                      v6.2.7           Maps an HTTP request to a set of configuration variables
symfony/service-contracts            v3.2.1           Generic abstractions related to writing services
symfony/stopwatch                    v6.2.7           Provides a way to profile code
symfony/string                       v6.2.7           Provides an object-oriented API to strings and deals with bytes, UTF-8 code points and grapheme clusters in a unified way
symfony/translation                  v6.2.7           Provides tools to internationalize your application
symfony/translation-contracts        v3.2.1           Generic abstractions related to translation
symfony/uid                          v6.2.7           Provides an object-oriented API to generate and represent UIDs
symfony/var-dumper                   v6.2.7           Provides mechanisms for walking through any arbitrary PHP variable
symfony/var-exporter                 v6.2.7           Allows exporting any serializable PHP data structure to plain PHP code
symfony/yaml                         v6.2.7           Loads and dumps YAML files
theseer/tokenizer                    1.2.1            A small library for converting tokenized PHP source code into XML and potentially other formats
tijsverkoyen/css-to-inline-styles    2.2.6            CssToInlineStyles is a class that enables you to convert HTML-pages/files into HTML-pages/files with inline styles. This is very useful when you're sending emails.
vlucas/phpdotenv                     v5.5.0           Loads environment variables from `.env` to `getenv()`, `$_ENV` and `$_SERVER` automagically.
voku/portable-ascii                  2.0.1            Portable ASCII library - performance optimized (ascii) string functions for php.
webmozart/assert                     1.11.0           Assertions to validate method input/output with nice error messages.

Summary

I'm using the Laravel framework and trying to create a simple stub for the final class Symfony\Component\Console\Helper\ProgressBar (I'm writing a unit test for the console command):

// test file
use Illuminate\Foundation\Testing\TestCase;

class MyCommandTest extends TestCase
{
    use CreatesApplication;

    public function test(): void
    {
        $progressBarMock = new class {
            public function start(): void
            {}
            public function advance(): void
            {}
            public function finish(): void
            {}
        };
        
        $outputStyleMock = $this->createMock(OutputStyle::class);
        $outputStyleMock
            ->expects($this->once())
            ->method('createProgressBar')
            ->willReturn($progressBarMock);
        $command = $this->createMock(MyCommand::class); // in real code I use `getMockBuilder()` but it doesn't matter
        $command->setOutput($outputStyleMock);
    }
}

// command file
use Illuminate\Console\Command;

class MyCommand extends Command
{
    protected $signature = 'mycommand:run';
    protected $description = 'My awesome command';

    public function handle(): int
    {
        $total = 100;
        $progressBar = $this->output->createProgressBar($total);
        $progressBar->start();

        for ($i = 0; $i < $total; $i++) {
            // some logic here
            $progressBar->advance();
        }

        $progressBar->finish();

        return 0;
    }
}

Current behavior

Method createProgressBar may not return value of type class@anonymous, its declared return type is "Symfony\Component\Console\Helper\ProgressBar"

How to reproduce

Run the code snippet provided above

Expected behavior

No errors expected (as in PHPUnit 9)

The assertObjectHasAttribute() is deprecated because the word "attribute" means something different in the PHP world now. But it was deprecated without any replacement, stating that it is bad code causing this assertion, one should use classes etc instead.

The problem is that I am using PhpUnit to unit-test my JSON API, and I cannot see a replacement for just checking the JSON structure. Of course, assertTrue(property_exists()) works, but the error message yielded is not nearly as good as assertObjectHasAttribute was.

Can we consider making assertObjectHasProperty()? That name will never be wrong, because it is the exact same word as in property_exists().

Exception code used to be mixed, then int|string, now they are only int. TestCase::expectException() still allows int|string.

Starting with PHPUnit 10 and its event system, the test runner can be extended.

To extend PHPUnit's test runner, you implement the PHPUnit\Runner\Extension\Extension interface. This interface requires a bootstrap() method to be implemented. This method is called by PHPUnit's test runner to bootstrap a configured extension. An extension is configured in PHPUnit's XML configuration file.

For test runner extensions that intend to change how the test runner presents progress information or test result information, the default progress printer and the default result printer currently have to be disabled using the --no-output, --no-progress, or --no-results CLI options.

This ticket proposes the addition of another interface, PHPUnit\Runner\Extension\OutputReplacing:

namespace PHPUnit\Runner\Extension;

interface OutputReplacing
{
    public function replacesDefaultProgressPrinter(): bool;

    public function replacesDefaultResultPrinter(): bool;
}

When implemented by a test runner extension's bootstrap class in addition to PHPUnit\Runner\Extension\Extension, the PHPUnit test runner will query the extension using these methods on whether or not the extension intends to replace the test runner's default progress printer, the test runner's default result printer, or both.

When multiple test runner extensions are configured (and therefore bootstrapped), only one may replace the test runner's default progress printer or the test runner's default result printer, respectively. The test runner will error out when more than one test runner extension intends to replace a default printer.