Insert type guards (CHECK_OP1_TRACE_TYPE / CHECK_OP2_TRACE_TYPE) on the
sensitive bailout paths in ADD/SUB/MUL JIT compilation: the MAY_BE_UNDEF
and non-numeric operand breaks. Guards are only emitted when the traced
operand type is IS_LONG or IS_DOUBLE, ensuring TSSA result type
predictions stay valid for side traces without affecting the normal
numeric fast path.
Fixes GH-20838
Co-authored-by: Dmitry Stogov <dmitrystogov@gmail.com>
When FE_RESET_RW executes, it converts the CV to a reference before
checking if the array/object is empty. However, when the JIT creates
exit points for FE_RESET_RW in zend_jit_trace_handler(), it wasn't
updating the stack type for op1 to reflect this change.
This caused side traces compiled from these exit points to have
incorrect type information. The side trace's CV cleanup code would
see IS_OBJECT and generate a direct call to zend_objects_store_del(),
but the actual value was a zend_reference*, causing a segfault.
The fix adds ZEND_FE_RESET_RW to the list of opcodes that temporarily
set their op1 stack type to IS_UNKNOWN before creating exit points.
This follows the same pattern used for ZEND_BIND_INIT_STATIC_OR_JMP.
When IS_UNKNOWN, the JIT falls back to SSA type info which correctly
includes MAY_BE_REF for FE_RESET_RW's op1_def.
Fixes GH-20818
Closes GH-20948
This is similar to f6c2e40a11 but for minimal JIT + tracing JIT.
Most of the times the tracing JIT shouldn't rely on going to the VM, but
in some cases, like in minimal JIT, it can and then it hits the same
bug.
Closes GH-20897.
The dynamic loader, starting around version 1284, patches the thunk emitted for
thread local variables by the compiler, so that its format changes from
struct Thunk {
void *func;
size_t module;
size_t offset;
}
to
struct Thunk_v2 {
void *func;
uint32_t module;
uint32_t offset;
// other fields
}
which has the same size, but not the same layout.
This is mentionned in
9307719dd8/libdyld/ThreadLocalVariables.h (L90)
As a result, access to thread specific variables in JIT is broken.
Fix by using the new layout when the new dynamic loader is in use.
Closes GH-20121
zend_runtime_jit() prevents concurrent compilation with
zend_shared_alloc_lock(), but this doesn't prevent blocked threads from
trying to compile the function again after they acquire the lock.
In the case of GH-19889, one of the function entries is compiled with
zend_jit_handler(), which fails when the op handler has already been replaced by
a JIT'ed handler.
Fix by marking compiled functions with a new flag ZEND_FUNC_JITED, and
skipping compilation of marked functions. The same fix is applied to
zend_jit_hot_func().
Fixes GH-19889
Closes GH-19971
zend_jit_fetch_obj_r_slow_ex() may be used by the function JIT, which doesn't
rely on guards to handle references. Therefore it must deref the property value.
Other variants of zend_jit_fetch_obj_*_slow_ex can not be used used in function
JIT.
Fixes GH-19831
Closes GH-19838
When the assumption that (PRE|POST)_(INC|DEC) overflows turns out to be
false and we exit, effects are lost if op1 or result were in regs.
Fix by updating the stack map before creating the exit point.
Fixes GH-19669
Closes GH-19680
JIT doesn't recognize that variables may be used after returning from a
trace due to YIELD, so some effects may never be stored to memory.
YIELD ops terminate trace recordings with ZEND_JIT_TRACE_STOP_RETURN, and are
handled mostly like RETURN. Here I change zend_jit_trace_execute() so that
YIELD terminates recordings with ZEND_JIT_TRACE_STOP_INTERPRETER instead,
to ensure that we recognize that variables may be used after returning from
the trace due to YIELD.
Fixes GH-19493
Closes GH-19515
Property hooks were not handled for JIT+trait+preloading.
Split the existing functions that handle op arrays, and add iterations
for property hooks.
Closes GH-18923.
ZEND_FUNC_INFO() can not be used on internal CE's. If preloading makes a
CE that's an alias of an internal class, the invalid access happens when
setting the FUNC_INFO.
While we could check the class type to be of user code, we can just skip
aliases altogether anyway which may be faster.
Closes GH-18915.
GAS started checking the relocation for tlsgd: it must use the %rdi
register. However, the inline assembly now uses %rax instead.
Fix it by changing the "=a" output register to "=D".
Source: ec181e1710/gas/config/tc-i386.c (L6793)
gottpoff is unaffected.
Closes GH-18779.
Polymorphic calls pass this and the function to side traces via snapshotting.
However, we assume that this/func are in registers, when in fact they may be
spilled.
Here I update snapshotting of poly_func/poly_this to support spilling:
- In zend_jit_snapshot_handler, keep track of the C stack offset
of the spilled register, in a way similar to how stack variables.
- In zend_jit_start, do not pre-load the registers if they were spilled.
- In zend_jit_trace_exit / zend_jit_trace_deoptimization, load from the
stack if the register was spilled.
- Store a reference to poly_func/poly_this in zend_jit_ctx so we can use that
directly in the side trace.
Closes GH-18408
Add a new exit flag (ZEND_JIT_EXIT_CHECK_EXCEPTION) that enables exception
checking during exit/deoptimization.
We already checked for exceptions during exit/deoptimization, but only when
ZEND_JIT_EXIT_FREE_OP1 or ZEND_JIT_EXIT_FREE_OP2 were set (presumably to
handle exceptions thrown during dtor). The new flag makes it possible to request
it explicitly.
This also fixes two issues in zend_jit_trace_exit():
- By returning 1, we were telling the caller (zend_jit_trace_exit_stub()) to
execute the original op handler of EG(current_execute_data)->opline, but in
reality we want to execute EX(opline), which should be EG(exception_op).
- EX(opline) is set to the value of %r15 in zend_jit_trace_exit_stub() before
calling zend_jit_trace_exit(), but this may be the address of a
zend_execute_data when the register is being reused to cache EX(call).
Fixes GH-18262
Closes GH-18297
The JIT helper `zend_jit_assign_op_to_typed_ref` expects a `zval*` as an
argument, so we have to store to the stack if OP1_DATA(=op3) is in a
register.
Closes GH-18299.
A frameless icall with 3 arguments is a special case because it uses
OP_DATA, but this was not added to the list, so the opline pointed to
the wrong address resulting in UBSAN report or crash.
Closes GH-18048.
This is a bit of a theoretical issue, but the maximum string length is
actually ZSTR_MAX_LEN instead of SIZE_MAX. The resulting check is a bit
slower but should still be relatively cheap.
Closes GH-18049.
