When a smart branch and jump live in separate basic blocks, the JIT can't skip
the jitting of the jump, as it may be reachable through another predecessor.
When the smart branch is executed using zend_jit_handler(), we're manually
writing the result of the branch to the given temporary so that the jump will
work as expected. That happens in zend_jit_set_cond().
However, this was only correctly handled for JMPZ branches. The current opline
was compared to opline following the jump, which would set the var to 1 if
equal, i.e. the branch was not taken, meaning var was not zero. For JMPNZ we
need to do the opposite.
Fixes GH-21593
Add a missing `t->stack_map[...].reg = ZREG_NONE` in `zend_jit_snapshot_handler`.
This is needed when reg is `ZREG_NONE`, otherwise side traces will have wrong
assumptions.
Fixes GH-21158
Closes GH-21531
When the JIT defers the IS_UNDEF check for FETCH_OBJ_R to the result
type guard, the deoptimization escape path dispatches to opline->handler
via the trace_escape stub. If opline->handler has been overwritten with
JIT code (e.g. a function entry trace), this creates an infinite loop.
Fix by dispatching to the original VM handler (orig_handler from the
trace extension) instead of going through the trace_escape stub. This
avoids the extra IS_UNDEF guard on every property read while correctly
handling the rare IS_UNDEF case during deoptimization.
Also set current_op_array in zend_jit_trace_exit_to_vm so that the
blacklisted exit deoptimizer can resolve orig_handler, covering the
case where side trace compilation is exhausted.
Closes GH-21368.
Insert type guards (CHECK_OP1_TRACE_TYPE / CHECK_OP2_TRACE_TYPE) on the
sensitive bailout paths in ADD/SUB/MUL JIT compilation: the MAY_BE_UNDEF
and non-numeric operand breaks. Guards are only emitted when the traced
operand type is IS_LONG or IS_DOUBLE, ensuring TSSA result type
predictions stay valid for side traces without affecting the normal
numeric fast path.
Fixes GH-20838
Co-authored-by: Dmitry Stogov <dmitrystogov@gmail.com>
When FE_RESET_RW executes, it converts the CV to a reference before
checking if the array/object is empty. However, when the JIT creates
exit points for FE_RESET_RW in zend_jit_trace_handler(), it wasn't
updating the stack type for op1 to reflect this change.
This caused side traces compiled from these exit points to have
incorrect type information. The side trace's CV cleanup code would
see IS_OBJECT and generate a direct call to zend_objects_store_del(),
but the actual value was a zend_reference*, causing a segfault.
The fix adds ZEND_FE_RESET_RW to the list of opcodes that temporarily
set their op1 stack type to IS_UNKNOWN before creating exit points.
This follows the same pattern used for ZEND_BIND_INIT_STATIC_OR_JMP.
When IS_UNKNOWN, the JIT falls back to SSA type info which correctly
includes MAY_BE_REF for FE_RESET_RW's op1_def.
Fixes GH-20818
Closes GH-20948
This is similar to f6c2e40a11 but for minimal JIT + tracing JIT.
Most of the times the tracing JIT shouldn't rely on going to the VM, but
in some cases, like in minimal JIT, it can and then it hits the same
bug.
Closes GH-20897.
zend_jit_fetch_obj_r_slow_ex() may be used by the function JIT, which doesn't
rely on guards to handle references. Therefore it must deref the property value.
Other variants of zend_jit_fetch_obj_*_slow_ex can not be used used in function
JIT.
Fixes GH-19831
Closes GH-19838
When the assumption that (PRE|POST)_(INC|DEC) overflows turns out to be
false and we exit, effects are lost if op1 or result were in regs.
Fix by updating the stack map before creating the exit point.
Fixes GH-19669
Closes GH-19680
When the assumption that (PRE|POST)_(INC|DEC) overflows turns out to be
false and we exit, effects are lost if op1 or result were in regs.
Fix by updating the stack map before creating the exit point.
Fixes GH-19669
Closes GH-19680
Blacklisted side traces (aka JIT'ed exits) may return the previous opline
after calling the original op handler. As a result, the op handler is called
again by the VM.
Fix this by always returning the opline returned by the original op handler.
Always use zend_jit_vm_enter(jit, ref) to signal the VM that it must reload
EG(current_execute_data) as it may have changed during the execution of
the trace.
Fixes GH-19486
Closes GH-19535
JIT doesn't recognize that variables may be used after returning from a
trace due to YIELD, so some effects may never be stored to memory.
YIELD ops terminate trace recordings with ZEND_JIT_TRACE_STOP_RETURN, and are
handled mostly like RETURN. Here I change zend_jit_trace_execute() so that
YIELD terminates recordings with ZEND_JIT_TRACE_STOP_INTERPRETER instead,
to ensure that we recognize that variables may be used after returning from
the trace due to YIELD.
Fixes GH-19493
Closes GH-19515
Property hooks were not handled for JIT+trait+preloading.
Split the existing functions that handle op arrays, and add iterations
for property hooks.
Closes GH-18923.
During persisting, the JIT may trigger and fill in the call graph.
The call graph info is allocated on the arena which will be gone after preloading.
To prevent invalid accesses during normal requests, the arena data should be cleared.
This has to be done after all scripts have been persisted because shared op arrays between
scripts can change the call graph.
Closes GH-18916.
Add a new exit flag (ZEND_JIT_EXIT_CHECK_EXCEPTION) that enables exception
checking during exit/deoptimization.
We already checked for exceptions during exit/deoptimization, but only when
ZEND_JIT_EXIT_FREE_OP1 or ZEND_JIT_EXIT_FREE_OP2 were set (presumably to
handle exceptions thrown during dtor). The new flag makes it possible to request
it explicitly.
This also fixes two issues in zend_jit_trace_exit():
- By returning 1, we were telling the caller (zend_jit_trace_exit_stub()) to
execute the original op handler of EG(current_execute_data)->opline, but in
reality we want to execute EX(opline), which should be EG(exception_op).
- EX(opline) is set to the value of %r15 in zend_jit_trace_exit_stub() before
calling zend_jit_trace_exit(), but this may be the address of a
zend_execute_data when the register is being reused to cache EX(call).
Fixes GH-18262
Closes GH-18297
On win64, xmm6-xmm15 are preserved registers, but the prologues and
epilogues of JITted code don't handle these. The issue occurs when
calling into the JIT code again via an internal handler
(like call_user_func). Therefore, we want to save/restore xmm registers
upon entering/leaving execute_ex. Since MSVC x64 does not support inline
assembly, we create an assembly wrapper around the real execute_ex
function.
The alternative is to always save/restore these xmm registers into the
fixed call frame, but this causes unnecessary overhead.
The same issue occurs for ARM64 platforms for floating point register
8 to 15. However, there we can use inline asm to fix this.
Closes GH-18352.
Ilija Tovilo