JIT doesn't recognize that variables may be used after returning from a
trace due to YIELD, so some effects may never be stored to memory.
YIELD ops terminate trace recordings with ZEND_JIT_TRACE_STOP_RETURN, and are
handled mostly like RETURN. Here I change zend_jit_trace_execute() so that
YIELD terminates recordings with ZEND_JIT_TRACE_STOP_INTERPRETER instead,
to ensure that we recognize that variables may be used after returning from
the trace due to YIELD.
Fixes GH-19493
Closes GH-19515
Property hooks were not handled for JIT+trait+preloading.
Split the existing functions that handle op arrays, and add iterations
for property hooks.
Closes GH-18923.
During persisting, the JIT may trigger and fill in the call graph.
The call graph info is allocated on the arena which will be gone after preloading.
To prevent invalid accesses during normal requests, the arena data should be cleared.
This has to be done after all scripts have been persisted because shared op arrays between
scripts can change the call graph.
Closes GH-18916.
ZEND_FUNC_INFO() can not be used on internal CE's. If preloading makes a
CE that's an alias of an internal class, the invalid access happens when
setting the FUNC_INFO.
While we could check the class type to be of user code, we can just skip
aliases altogether anyway which may be faster.
Closes GH-18915.
The trait handling for property hooks in preloading did not exist, we
add a check to skip trait clones and we add the necessary code to update
the op arrays.
Closes GH-18586.
The assertion is imprecise now, and the code assumed that from the
moment an internal class was encountered that there were only internal
classes remaining. This is wrong now, and we still have to continue if
we encounter an internal class. We can only skip the remaining iterations
if the entry in the hash table is not an alias.
Closes GH-18575.
Add a new exit flag (ZEND_JIT_EXIT_CHECK_EXCEPTION) that enables exception
checking during exit/deoptimization.
We already checked for exceptions during exit/deoptimization, but only when
ZEND_JIT_EXIT_FREE_OP1 or ZEND_JIT_EXIT_FREE_OP2 were set (presumably to
handle exceptions thrown during dtor). The new flag makes it possible to request
it explicitly.
This also fixes two issues in zend_jit_trace_exit():
- By returning 1, we were telling the caller (zend_jit_trace_exit_stub()) to
execute the original op handler of EG(current_execute_data)->opline, but in
reality we want to execute EX(opline), which should be EG(exception_op).
- EX(opline) is set to the value of %r15 in zend_jit_trace_exit_stub() before
calling zend_jit_trace_exit(), but this may be the address of a
zend_execute_data when the register is being reused to cache EX(call).
Fixes GH-18262
Closes GH-18297
When a first PHP process launches, Opcache creates a shared file mapping
to use as a shm region. The size of this mapping is set by
opcache.memory_consumption.
When a new PHP process launches while the old one is still running,
Opcache tries to reattach to the shm.
When reattaching it tries to map the requested size (i.e. set by
opcache.memory_consumption). However, if the new requested size is
larger than the size used in the original file mapping, then the call
to VirtualProtect() will fail and the new PHP process will fail to
launch.
It's not possible to resize the virtual region on Windows, unless
relying on undocumented APIs like `NtExtendSection` but then we would
sitll need to communicate that to the first process.
This issue is the root cause of Psalm end-to-end tests failing in
GH-18417: Psalm estimates the required memory sizes and relaunches itself
with more memory requested, if its estimate is below the currently allocated
shared memory. This causes a crash on startup and the tests fail.
To solve this, we need to make the mappings unique per requested size.
There are two ideas:
1. Include in zend_system_id. However, this also affects other things
and may be too overkill.
2. Include it in the filename, this is an easy local change.
I went with this option.
Closes GH-18443.
On win64, xmm6-xmm15 are preserved registers, but the prologues and
epilogues of JITted code don't handle these. The issue occurs when
calling into the JIT code again via an internal handler
(like call_user_func). Therefore, we want to save/restore xmm registers
upon entering/leaving execute_ex. Since MSVC x64 does not support inline
assembly, we create an assembly wrapper around the real execute_ex
function.
The alternative is to always save/restore these xmm registers into the
fixed call frame, but this causes unnecessary overhead.
The same issue occurs for ARM64 platforms for floating point register
8 to 15. However, there we can use inline asm to fix this.
Closes GH-18352.
The JIT helper `zend_jit_assign_op_to_typed_ref` expects a `zval*` as an
argument, so we have to store to the stack if OP1_DATA(=op3) is in a
register.
Closes GH-18299.
If there's a try-finally where the try_op starts on a basic block with a
single JMP, and the JMP optimization causes that basic block to become
unreachable, then we update try_op.
In this case, there is no catch_op, so try_op is erroneously set to 0,
we should instead set it to `b->start`.
Closes GH-18110.
in_array() calls are compiled to frameless calls. Adjust the
optimization appropriately. Luckily, frameless opcodes simplify the
optimization quite a bit.
Fixes GH-18050
Closes GH-18066
A frameless icall with 3 arguments is a special case because it uses
OP_DATA, but this was not added to the list, so the opline pointed to
the wrong address resulting in UBSAN report or crash.
Closes GH-18048.
The FETCH_OBJ_R VM handler has an optimization that directly enters into
a hook if it is a simpler getter hook. This is not compatible with the
minimal JIT because the minimal JIT will try to continue executing the
opcodes after the FETCH_OBJ_R.
To solve this, we check whether the opcode is still the expected one
after the execution of the VM handler. If it is not, we know that we are
going to execute a simple hook. In that case, exit to the VM.
Closes GH-17909.
This solely affects the builtin enum functions currently.
Given that these are stored in SHM, we cannot simply hardwire a pointer into the internal function runtime cache on NTS too, but have to use a MAP_PTR (like on ZTS).
Now, by design, the runtime cache of internal functions no longer is reset between requests, hence we need to store them explicitly as static runtime cache.
On NTS builds we cannot trivially move the pointers into CG(internal_run_time_cache) as they're directly stored on the individual functions (on ZTS we could simply iterate the static map_ptrs).
Hence, we have the choice between having opcache managing the internal run_time_cache for its preloaded functions itself or realloc CG(internal_run_time_cache) and iterate through all functions to assign the new address. We choose the latter for simplicity and initial speed.
When read_property fails, it may return `&EG(uninitialized_zval)`, and
the exception is handled in the VM. The VM will try to
`zval_ptr_dtor_nogc` the result, but the result was never set, resulting
in dtor'ing garbage data. To solve this, we check when a different zval*
was returned and initialize the result with UNDEF. We don't need to copy
as the slow_ex handler return values are used directly in a register.
Closes GH-17749.
When a guard check is created for a variable to check if it's a packed array,
it is possible that there was no prior type check for that variable.
This happens in the global scope for example when the variable aliases.
In the test, this causes a dereference of address 8 because the integer
element in `$a` is interpreted as an array address.
This patch adds a check to see if the guard is handled.
If we were not able to determine or guard the type then we also cannot know the array is packed.
Closes GH-17584.
This test has two classes that use the same trait. In function JIT mode
the same cache slot will be used. This causes problems because it is
primed for the first class and then reused for the second class,
resulting in an incorrect type check failure.
The current check for a megamorphic trait call requires current_frame to
not be NULL, but this is only set in tracing mode and not in function
mode.
This patch corrects the check.
Closes GH-17660.
The code to update the call_level in that case skips the opline itself,
as that's handled by the tail handler, and then wants to set the opline
to the last opline of the block because the code below the switch will
update the call_level for that opline.
However, the test has a block with a single opline (THROW). The block
after that has ZEND_INIT_FCALL, because `i` points to ZEND_INIT_FCALL
now, it erroneously causes the call_level after the switch.
Closes GH-17438.
`bcadd(...)` is a closure for an internal function, and
`zend_jit_push_call_frame` takes into account both last_var and the
difference in argument numbers not only for user code but also for
internal code. However, this is inconsistent with
`zend_vm_calc_used_stack`, causing argument corruption.
Making this consistent fixes the issue.
I could only reproduce the assertion failure when using Valgrind.
Closes GH-17319.
Minimal JIT shouldn't generate a call to the complex handler, but
instead rely on the VM and then check for a two-way jump.
This moves the frameless codegen under the check `JIT_G(opt_level) >=
ZEND_JIT_LEVEL_INLINE`.
EX(opline) / opline can be stale if the IP is not stored, like in this
case on a trace enter. We always need to make sure that the opline is up
to date to make sure we don't use stale data.
Closes GH-17260.
This bug happens because of a nested `SHM_UNPROTECT()` sequence.
In particular:
```
unprotect memory at ext/opcache/ZendAccelerator.c:2127
protect memory at ext/opcache/ZendAccelerator.c:2160
unprotect memory at ext/opcache/ZendAccelerator.c:2164
unprotect memory at ext/opcache/jit/zend_jit_trace.c:7464
^^^ Nested
protect memory at ext/opcache/jit/zend_jit_trace.c:7591
^^^ Problem is here: it should not protect again due to the nested unprotect
protect memory at ext/opcache/ZendAccelerator.c:2191
^^^ This one should actually protect, not the previous one
```
The reason this nesting happen is because:
1. We try to include the script, this eventually calls `cache_script_in_shared_memory`
2. `zend_optimize_script` will eventually run SCCP as part of the DFA pass.
3. SCCP will try to replace constants, but can also run destructors when a partial array is destructed here:
4e9cde758e/Zend/Optimizer/sccp.c (L2387-L2389)
In this case, this destruction invokes the GC which invokes the tracing JIT,
leading to the nested unprotects.
This patch disables the GC to prevent invoking user code, as user code
is not supposed to run during the optimizer pipeline.
Closes GH-17249.
Co-authored-by: Dmitry Stogov <dmitry@zend.com>
ZEND_FETCH_DIM_FUNC_ARG should also be repeated on undefined access,
consistent to how ZEND_FETCH_DIM_R is handled. The opcode was just
missing from the assertion list.
Closes GH-17148.
Co-authored-by: Dmitry Stogov <dmitry@zend.com>
op1 of ZEND_MATCH_ERROR, which refers to the match expression, is not freed by
MATCH_ERROR itself. Instead, it is freed by ZEND_HANDLE_EXCEPTION. For normal
control flow, a FREE is placed at the end of the match expression.
Since FREE may appear after MATCH_ERROR in the opcode sequence, we need to
correctly handle op1 of MATCH_ERROR as alive.
Fixes GH-17106
Closes GH-17108
The `jit_prof_threshold` is a float, supposed to be in range [0, 1],
and usually very small (the default is 0.005). Reporting it as int
is meaningless.
Closes GH-17077.
