1
0
mirror of https://github.com/php/php-src.git synced 2026-04-03 22:22:18 +02:00
Commit Graph

720 Commits

Author SHA1 Message Date
Max Semenik
bdd7c48764 run-tests: drop support for ancient Valgrind versions
Valgrind 3.3.0 was released in 2007, not even RHEL has crap older than
this. It could be argued that 3.8.0, released in 2012, could be a safe
cutoff too.

Closes GH-6728.
2021-03-01 11:21:30 +01:00
Christoph M. Becker
51914610ab Merge branch 'PHP-8.0'
* PHP-8.0:
  Capture Microsoft-defined HRESULT exit codes exit codes
2021-02-25 12:05:27 +01:00
Christoph M. Becker
fb683030bf Merge branch 'PHP-7.4' into PHP-8.0
* PHP-7.4:
  Capture Microsoft-defined HRESULT exit codes exit codes
2021-02-25 12:03:43 +01:00
Dylan K. Taylor
a480bf8093 Capture Microsoft-defined HRESULT exit codes exit codes
The lack of such a check leads to false-passes of tests on Windows
which expect no output, but produce a segfault or similar issue. I
discovered this a while ago due to bad tests in an extension I maintain.

Closes GH-6722.
2021-02-25 12:01:04 +01:00
Max Semenik
dda0cea5cd run-tests: add skip cache
Currently every --SKIPIF-- section in every test file results in 1
extra execution of PHP, every --EXTENSIONS-- section - in 2 executions.
This is quite wasteful, as skip checking code is extremely repetitive
and extensions are fixed for every binary/ini/command parameters
combination.

This patch adds caching to all such checks.

On my machine, the gains are quite noticeable: 36s instead of 43s
with -j16, 292s instead of 337s without concurrency. Cache stats are
3780 hits, 1247 misses in the latter case. In the future, tests could
be adjusted to have more uniform skip checks to improve performance even
more.

Closes GH-6681.
2021-02-23 15:05:49 +01:00
Christoph M. Becker
b84e3b1ced Merge branch 'PHP-8.0'
* PHP-8.0:
  run-tests: fixed exit code not being set on BORKED tests
2021-02-23 13:23:22 +01:00
Christoph M. Becker
8162c3d1fd Merge branch 'PHP-7.4' into PHP-8.0
* PHP-7.4:
  run-tests: fixed exit code not being set on BORKED tests
2021-02-23 13:22:50 +01:00
Christoph M. Becker
67bb06cc9b Merge branch 'PHP-7.3' into PHP-7.4
* PHP-7.3:
  run-tests: fixed exit code not being set on BORKED tests
2021-02-23 13:15:30 +01:00
Dylan K. Taylor
729cd8bacd run-tests: fixed exit code not being set on BORKED tests
When no test paths are specified this shows up when 'make test' is used on a PECL extension without specifying tests to run (or in php-src too, I guess...)

Closes GH-6717.
2021-02-23 12:50:03 +01:00
Nikita Popov
4ea90c15de Merge branch 'PHP-8.0'
* PHP-8.0:
  Color LEAK&FAIL like a failure
2021-02-17 16:40:33 +01:00
Nikita Popov
cbdd21a22d Color LEAK&FAIL like a failure 2021-02-17 16:40:19 +01:00
Max Semenik
b11771271e Remove stray mentions of mbstring.func_overload
This feature has been completely removed.

Closes GH-6688.
2021-02-15 09:47:28 +01:00
Nikita Popov
b10416a652 Deprecate passing null to non-nullable arg of internal function
This deprecates passing null to non-nullable scale arguments of
internal functions, with the eventual goal of making the behavior
consistent with userland functions, where null is never accepted
for non-nullable arguments.

This change is expected to cause quite a lot of fallout. In most
cases, calling code should be adjusted to avoid passing null. In
some cases, PHP should be adjusted to make some function arguments
nullable. I have already fixed a number of functions before landing
this, but feel free to file a bug if you encounter a function that
doesn't accept null, but probably should. (The rule of thumb for
this to be applicable is that the function must have special behavior
for 0 or "", which is distinct from the natural behavior of the
parameter.)

RFC: https://wiki.php.net/rfc/deprecate_null_to_scalar_internal_arg

Closes GH-6475.
2021-02-11 21:46:13 +01:00
Max Semenik
0d3ab673aa run-tests: fix JUnit counts
In 19680f886f, I forgot about my own TODO. At least, now I understand what
this line was about :P
2021-02-11 09:47:56 +01:00
Max Semenik
19680f886f run-tests.php: move JUnit stuff into a class
This is part one of my work that was announced at
https://externals.io/message/110391

Closes GH-6671.
2021-02-08 18:28:11 +01:00
twosee
23a7b0f323 Fix environment variable not work in run-tests.php
Closes GH-6657.
2021-02-01 17:27:19 +08:00
Nikita Popov
866e3e03de Create .php and .sh on valgrind failure 2021-02-01 10:10:09 +01:00
Nikita Popov
26e0d389bc Show slow SKIPIF sections as well 2020-12-21 22:55:49 +01:00
Nikita Popov
3b4fb2ebea Determine run-tests executables consistently 2020-12-01 12:36:25 +01:00
Nikita Popov
6fdc988592 Don't pass null to strlen() 2020-12-01 12:36:18 +01:00
Nikita Popov
48725d89aa Merge branch 'PHP-8.0'
* PHP-8.0:
  Fixed bug #80437
2020-11-30 11:42:07 +01:00
Nikita Popov
15073d8e1d Fixed bug #80437
Only chmod if the file has been created.
2020-11-30 11:40:46 +01:00
Christoph M. Becker
c3885f2316 Add --repeat flag to usage info of run-tests.php
Closes GH-6429.
2020-11-15 16:41:46 +01:00
Nikita Popov
1b3b430f47 Add --repeat testing mode
This testing mode executes the test multiple times in the same
process (but in different requests). It is primarily intended to
catch tracing JIT bugs, but also catches state leaks across
requests.

Closes GH-6365.
2020-10-30 17:29:33 +01:00
bashkarev
f17b2a75a5 Don't set xdebug.default_enable in run-tests
This throws a deprecation warning in XDebug 3.

Closes GH-6324.
2020-10-26 13:10:26 +01:00
Nikita Popov
b2700811f3 Skip arginfo test under msan
Msan is missing interceptors for some functions that result in
false positives.
2020-10-15 10:40:23 +02:00
Nikita Popov
9623756ab7 Remove unused --temp-urlbase option from run-tests.php
This was part of the removed HTML functionality.
2020-09-23 12:34:59 +02:00
Berbe
bfeb2f6abc Fix: Gracefully handle empty user input in run-tests.php
Closes GH-6016.
2020-08-24 11:46:01 +02:00
Tyson Andre
0461c1574a Add run-tests.php --context [n] option.
Mentioned in https://github.com/php/php-src/pull/5965#discussion_r467621123

This PR proposes 3 lines of context so the impact can be seen in tests.
Other `diff` programs show around 3 lines of context.
(This helps indicate exactly which position a test should be updated
to add a new expected line at)

Use the mapping for choosing order to display diffs

Properly include context in cases where the expected output had more lines than
the actual output, e.g.

```
--FILE--
A
A1
A
C
NEARBY
--EXPECTF--
A
B
A1
B
A
B
A
B
NEARBY
```

Closes GH-5968
2020-08-16 09:31:38 -04:00
Tyson Andre
7be61bec80 Colorize --show-diff, only on stdout
Don't add colors to the saved `.diff` file.

Related to a41cf3e1d2
2020-08-13 11:27:32 -04:00
Tyson Andre
45d69fb809 Support NO_COLOR environment variable in run-tests.php
And add a --color option for run-tests.php

See https://no-color.org/

> an informal standard is hereby proposed:
>
> All command-line software which outputs text with ANSI color added should check
> for the presence of a `NO_COLOR` environment variable that, when present
> (regardless of its value), prevents the addition of ANSI color.
2020-08-12 09:37:45 -04:00
Nikita Popov
a41cf3e1d2 Revert "Make diff section contents red(-)/green(+) in run-tests.php"
This reverts commit 58489bd3b9.

This also uses ANSI escape code coloring for .diff files, where it
is not meaningful.
2020-08-10 09:38:48 +02:00
Tyson Andre
58489bd3b9 Make diff section contents red(-)/green(+) in run-tests.php
This uses green for lines with `+` and red for lines with `-`.

Colors(Red and Green) would make the failure causes stand out visually when scrolling through errors.

Closes GH-5965
2020-08-09 20:59:43 -04:00
Tyson Andre
e79adf6381 Use short array values in run-tests.php
The run-tests.php file already requires php 7 syntax.
Done automatically with phpcbf with no other changes done/planned.

This file is already using short arrays in other places.

run-tests has already been substantially refactored in PHP 8.0
compared to PHP 7.4.

This makes merge conflicts more likely, but changes to run-tests.php on older
versions are less frequent bug fixes.
2020-08-09 16:09:55 -04:00
George Peter Banyard
5d72e40dfb Add colours to test results in run-tests.php
The test runner will attempt to colourise the result of each test

The --no-color option is introduced to disable this feature.

Closes GH-5901
2020-08-07 18:17:34 +02:00
George Peter Banyard
11274f53e7 Add type information to test-runner
Closes GH-5942
2020-08-06 20:37:36 +02:00
Tyson Andre
07db64156e [RFC] Make string length for getTraceAsString() configurable
Add a `zend.exception_string_param_max_len` ini setting.
(same suffix as `log_errors_max_len`)

Allow values between 0 and 1000000 bytes.
For example, with zend.exception_string_param_max_len=0,
"" would represent the empty string, and "..." would represent something
longer than the empty string.
Previously, this was hardcoded as exactly 15 bytes.

Discussion: https://externals.io/message/110717

Closes GH-5769
2020-07-25 09:50:57 -04:00
Dmitry Stogov
69b1f3d53d Use small counter values for testing 2020-07-08 16:49:51 +03:00
Paul Dragoonis
5d7ff25311 Removing HTML Functionality from run-tests.php
As discussed on GH-5632, the HTML functionality does not appear
to be in active use. For HTML rendering of test results, it is
suggested to instead use the JUnit integration, in combination
with your favorite JUnit viewer.

Closes GH-5705.
2020-06-19 09:59:38 +02:00
Nikita Popov
c260613c6f Optimize parallel test runner startup
Don't connect to each worker individually: First spawn all processes
and then establish connections in any order.

This avoids a ~1s wait when running on many cores.
2020-06-16 11:39:03 +02:00
Gerard Roche
d1dde094f2 run-tests: refactor 2020-05-22 09:58:04 +02:00
Gerard Roche
2ef88f517d run-tests: echo message and exit if no tests found
The test runner currently defaults to running the entire test suite if
no selected tests can be found. This can be unexpected.

For example the ext/mysqlnd/ directory has no tests, if you specify that
directory when testing the entire test suite will be run.

    run-tests.php [options] ext/mysqlnd/

Closes GH-5605.
2020-05-22 09:37:46 +02:00
Nikita Popov
0374cf08c7 Add --file-cache-prime/use options to run-tests
--file-cache-prime populates the file cache,
--file-cache-use uses the file cache.

And fix a number of tests to run under file cache or disabled
timestamp validation.
2020-05-20 15:29:32 +02:00
Nikita Popov
ec1e8540d8 Slightly deduplicate run-tests code 2020-05-20 11:58:42 +02:00
Derick Rethans
3707d75f4b Merge branch 'PHP-7.4' 2020-05-18 13:51:47 +01:00
Derick Rethans
d2797505cd Put Xdebug 3 into 'off' mode for run-tests.php 2020-05-18 13:51:40 +01:00
Derick Rethans
74a4a5ae55 Put Xdebug 3 into 'off' mode for run-tests.php 2020-05-18 13:49:12 +01:00
Gerard Roche
ecc0a87ff2 run-tests: extract usage message
Put the usage message near the top of the script,
into a separate function.

Closes GH-5558.
2020-05-13 09:54:55 +02:00
Gerard Roche
10edee7f03 run-tests: cs fixes (cleanup)
I used php-cs-fixer to do the cs fixes. The configuration I used is
posted below. The reason I disabled some of the rules is because they
create too much noise and would make it difficult to review. But please
feel free to close this PR and run the php-cs-fixer yourself.

    <?php

    $config = PhpCsFixer\Config::create();
    $config->setRiskyAllowed(false);
    $config->setRules([
        '@PSR2' => true,
        '@Symfony' => true,
        'array_syntax' => false,
        'binary_operator_spaces' => false,
        'blank_line_before_statement' => false,
        'concat_space' => false,
        'increment_style' => false,
        'phpdoc_align' => false,
        'single_quote' => false,
        'trailing_comma_in_multiline_array' => false,
        'unary_operator_spaces' => false,
        'yoda_style' => false,
    ]);

    $finder = PhpCsFixer\Finder::create();
    $finder->in(getcwd());

    $finder->exclude('Zend');
    $finder->exclude('build');
    $finder->exclude('ext');
    $finder->exclude('pear');
    $finder->exclude('sapi');
    $finder->exclude('scripts');
    $finder->exclude('win32');

    $config->setFinder($finder);

    return $config;

Closes GH-5557.
2020-05-13 09:52:44 +02:00
Gerard Roche
16f23cd159 run-tests: remove use of FILE_BINARY constant
The FILE_BINARY (and FILE_TEXT) constants are not really valid or useful
constants. It looks like they were added in 5.2.7 and have "no effect,
and are only available for forward compatibility."

See: https://www.php.net/manual/en/filesystem.constants.php

The default value of the file_put_contents() flags parameter is 0 and
FILE_BINARY is set to 0, so removing it doesn't change functionality.

P.S. Maybe those constants should be deprecated or removed in 8.0.

Closes GH-5556.
2020-05-12 16:59:25 +02:00