The dynamic loader, starting around version 1284, patches the thunk emitted for
thread local variables by the compiler, so that its format changes from
struct Thunk {
void *func;
size_t module;
size_t offset;
}
to
struct Thunk_v2 {
void *func;
uint32_t module;
uint32_t offset;
// other fields
}
which has the same size, but not the same layout.
This is mentionned in
https://github.com/apple-oss-distributions/dyld/blob/9307719dd8dc9b385daa412b03cfceb897b2b398/libdyld/ThreadLocalVariables.h#L90
As a result, access to thread specific variables in JIT is broken.
Fix by using the new layout when the new dynamic loader is in use.
Closes GH-20121
zend_runtime_jit() prevents concurrent compilation with
zend_shared_alloc_lock(), but this doesn't prevent blocked threads from
trying to compile the function again after they acquire the lock.
In the case of GH-19889, one of the function entries is compiled with
zend_jit_handler(), which fails when the op handler has already been replaced by
a JIT'ed handler.
Fix by marking compiled functions with a new flag ZEND_FUNC_JITED, and
skipping compilation of marked functions. The same fix is applied to
zend_jit_hot_func().
Fixes GH-19889
Closes GH-19971
This adds a new flag: ZEND_JIT_DEBUG_TRACE_EXIT_INFO_SRC. When the flag is set,
zend_jit_dump_exit_info() exposes the source of exit points, in debug builds.
Closes GH-19700
zend_jit_fetch_obj_r_slow_ex() may be used by the function JIT, which doesn't
rely on guards to handle references. Therefore it must deref the property value.
Other variants of zend_jit_fetch_obj_*_slow_ex can not be used used in function
JIT.
Fixes GH-19831
Closes GH-19838
When the assumption that (PRE|POST)_(INC|DEC) overflows turns out to be
false and we exit, effects are lost if op1 or result were in regs.
Fix by updating the stack map before creating the exit point.
Fixes GH-19669
Closes GH-19680
When the assumption that (PRE|POST)_(INC|DEC) overflows turns out to be
false and we exit, effects are lost if op1 or result were in regs.
Fix by updating the stack map before creating the exit point.
Fixes GH-19669
Closes GH-19680
On aarch64 we must set IR_USE_FRAME_POINTER to ensure that LR/x30 is
saved. Also, fixed_stack_frame_size must be n*16, not n*16+8 like on x86.
Fixes GH-19601
Closes GH-19630
Blacklisted side traces (aka JIT'ed exits) may return the previous opline
after calling the original op handler. As a result, the op handler is called
again by the VM.
Fix this by always returning the opline returned by the original op handler.
Always use zend_jit_vm_enter(jit, ref) to signal the VM that it must reload
EG(current_execute_data) as it may have changed during the execution of
the trace.
Fixes GH-19486
Closes GH-19535
Introduce the TAILCALL VM, a more efficient variant of the CALL VM:
* Each opcode handler tailcalls the next opcode handler directly instead of
returning to the interpreter loop. This eliminates call and interpreter loop
overhead.
* Opcode handlers use the preserve_none calling convention to eliminate
register saving overhead.
* preserve_none uses non-volatile registers for its first arguments, so
execute_data and opline are usually kept in these registers and no code is
required to forward them to the next handlers.
Generated machine code is similar to a direct-threaded VM with register pinning,
like the HYBRID VM.
JIT+TAILCALL VM also benefits from this compared to JIT+CALL VM:
* JIT uses the registers of the execute_data and opline args as fixed regs,
eliminating the need to move them in prologue.
* Traces exit by tailcalling the next handler. No code is needed to forward
execute_data and opline.
* No register saving/restoring in epilogue/prologue.
The TAILCALL VM is used when the HYBRID VM is not supported, and the compiler
supports the musttail and preserve_none attributes: The HYBRID VM is used when
compiling with GCC, the TAILCALL VM when compiling with Clang>=19 on x86_64 or
aarch64, and the CALL VM otherwise.
This makes binaries built with Clang>=19 as fast as binaries built with GCC.
Before, these were considerably slower (by 2.8% to 44% depending on benchmark,
and by 5% to 77% before 76d7c616bb).
Closes GH-17849
Closes GH-18720
JIT doesn't recognize that variables may be used after returning from a
trace due to YIELD, so some effects may never be stored to memory.
YIELD ops terminate trace recordings with ZEND_JIT_TRACE_STOP_RETURN, and are
handled mostly like RETURN. Here I change zend_jit_trace_execute() so that
YIELD terminates recordings with ZEND_JIT_TRACE_STOP_INTERPRETER instead,
to ensure that we recognize that variables may be used after returning from
the trace due to YIELD.
Fixes GH-19493
Closes GH-19515
Dmitry Stogov