This fixes the PHP deprecation warning:
PHP Deprecated: Implicit conversion from float 2048.96875 to int
loses precision in .../ext/mbstring/gen_rare_cp_bitvec.php on line 9
The current function `CHECK_HEADER_ADD_INCLUDE()` automatically defines
`HAVE_<HEADER_NAME_H>` preprocessor macros, which makes it difficult to
sync with other build systems. Specially, if some `HAVE_` macro is used
in the code and this function defines this macro but Autotools doesn't.
The new `CHECK_HEADER()` function behaves similar except it doesn't
define the `HAVE_<HEADER_NAME_H>` preprocessor macro.
This removes the following unused compile definitions:
HAVE_ARGON2_H
HAVE_AVIF_H
HAVE_BZLIB_H
HAVE_CAPSTONE_CAPSTONE_H
HAVE_CURL_EASY_H
HAVE_DB_H
HAVE_DECODE_H
HAVE_DEPOT_H
HAVE_EDITLINE_READLINE_H
HAVE_ENCHANT_H
HAVE_ENCODE_H
HAVE_FFI_H
HAVE_FIREBIRD_INTERFACE_H
HAVE_FT2BUILD_H
HAVE_GD_H
HAVE_GLIB_H
HAVE_GMP_H
HAVE_HTTPD_H
HAVE_IBASE_H
HAVE_IR_IR_H
HAVE_KECCAKHASH_H
HAVE_LBER_H
HAVE_LDAP_H
HAVE_LIBEXSLT_EXSLT_H
HAVE_LIBINTL_H
HAVE_LIBPQ_FE_H
HAVE_LIBTIDY_TIDY_H
HAVE_LIBXML_PARSER_H
HAVE_LIBXML_TREE_H
HAVE_LIBXML_XMLWRITER_H
HAVE_LIBXSLT_XSLT_H
HAVE_LMDB_H
HAVE_MBSTRING_H
HAVE_MYSQL_H
HAVE_ONIGURUMA_H
HAVE_OPENSSL_SSL_H
HAVE_PNG_H
HAVE_SNMP_H
HAVE_SODIUM_H
HAVE_SQLITE3_H
HAVE_SQLITE3EXT_H
HAVE_SYBFRONT_H
HAVE_TIDY_H
HAVE_TIDY_TIDY_H
HAVE_TIDYBUFFIO_H
HAVE_TIMELIB_CONFIG_H
HAVE_UNICODE_USPOOF_H
HAVE_UNICODE_UTF_H
HAVE_XPM_H
HAVE_ZIP_H
HAVE_ZIPCONF_H
HAVE_ZLIB_H
The following compile definitions are defined explicitly:
- HAVE_ICONV_H
- HAVE_MSCOREE_H
- HAVE_SQL_H
- HAVE_SQLEXT_H
Additionally, the `SETUP_OPENSSL()` function doesn't accept the 6th
argument anymore.
This avoids a crash in cases where the list of candidate encodings is so huge
that alloca would fail. Such crashes have been observed when the list of
encodings was larger than around 208,000 entries.
The aim of this PR is twofold:
- Reduce the number of highly similar TMP|VAR handlers
- Avoid ZVAL_DEREF in most of these cases
This is achieved by guaranteeing that all zend_compile_expr() calls, as well as
all other compile calls with BP_VAR_{R,IS}, will result in a TMP variable. This
implies that the result will not contain an IS_INDIRECT or IS_REFERENCE value,
which was mostly already the case, with two exceptions:
- Calls to return-by-reference functions. Because return-by-reference functions
are quite rare, this is solved by delegating the DEREF to the RETURN_BY_REF
handler, which will examine the stack to check whether the caller expects a
VAR or TMP to understand whether the DEREF is needed. Internal functions will
also need to adjust by calling the zend_return_unwrap_ref() function.
- By-reference assignments, including both $a = &$b, as well as $a = [&$b]. When
the result of these expressions is used in a BP_VAR_R context, the reference
is unwrapped via a ZEND_QM_ASSIGN opcode beforehand. This is exceptionally
rare.
Closes GH-20628
Thanks to the GitHub user vi3tL0u1s (Viet Hoang Luu) for reporting this issue.
The MacJapanese legacy text encoding has a very unusual property; it is possible for a string
to encode more codepoints than it has bytes. In some corner cases, this resulted in a situation
where the implementation code for mb_substr() would allocate a buffer of size -1. As you can
probably imagine, that doesn't end well.
Fixes GH-20832.
Over the last few years, I refactored mbstring to perform encoding conversion
a buffer at a time, rather than a single byte at a time. This resulted in a
huge performance increase.
After the refactoring, the old "byte-at-a-time" code was retained for two
reasons:
1) It was used by the mailparse PECL extension.
2) It was used to implement mb_strcut for some text encodings.
However, after reviewing mailparse's use of mbstring, it is clear that
mailparse only relies on mbstring for decoding of QPrint, and possibly
Base64. It does not use the byte-at-a-time conversion code for any
other encoding.
Further, mb_strcut only relies on the byte-at-a-time conversion code
for a limited number of legacy text encodings, such as ISO-2022-JP,
HZ, UTF-7, etc.
Hence, we can remove over 5000 lines of unused code without breaking
anything. This will help to reduce binary size, and make the mbstring
codebase easier to navigate for new contributors.
The legacy mbfl_strcut function is only used to implement mb_strcut
for legacy text encodings which 1) do not use a fixed number of bytes
per codepoint, 2) do not have an 'mblen_table' which can be used to
quickly determine the codepoint length of a byte sequence, and 3) do
not have a specialized 'mb_cut' function which implements mb_strcut
for that text encoding.
Remove unused code from mbfl_strcut, and leave only what is currently
needed for the implementation of mb_strcut.
mbstring's Unicode case conversion is table-driven, using Minimal Perfect Hash tables.
However, for small codepoint values, we bypass the hashtable lookup and just use
hard-coded conversion logic (i.e. adding or subtracting 0x20 from the appropriate
ASCII range).
For upcasing and downcasing, we had already optimized the conditional which sends
execution down this fast path, to use the fast path for as many codepoint values
as possible. However, for case folding, this had not been done.
This will give a small performance boost for case-folding Unicode text which
includes non-breaking spaces, symbols like ¥ or ™, or accented Latin
characters (used in many European languages).
GB18030-2022 is the current official standard, superseding the previous 2005 and 2000 versions. It is essential for modern Chinese text processing for the following reasons:
1. Superset Relationship: GB18030 is a strict superset of CP936 (GBK) and EUC-CN (GB2312). Using GB18030 as the detection target covers all characters in these older encodings while enabling support for a much wider range of characters.
2. Extended Character Coverage: The 2022 standard includes significant updates, covering over 87,000 characters. It adds support for CJK Extensions (C, D, E, F, G) and updates mappings for rare characters that were previously mapped to the Private Use Area (PUA) in the 2005 version. This is critical for correctly handling names containing rare characters (e.g., in banking or government data).
3. Backward Compatibility: It is safe to promote GB18030-2022 as the preferred encoding. Files encoded in EUC-CN or CP936 are valid GB18030 streams.
This PR adds GB18030-2022 to the default encoding list for CN.
The issue is specific to SLES15.
Arguably this should be reported to them as it seems to me they meddled
with the oniguruma source code.
The definition in oniguruma.h on that platform looks like this (same as upstream):
```c
ONIG_EXTERN
int onig_error_code_to_str PV_((OnigUChar* s, int err_code, ...));
```
Where `PV_` is defined as (differs):
```c
#ifndef PV_
#ifdef HAVE_STDARG_PROTOTYPES
# define PV_(args) args
#else
# define PV_(args) ()
#endif
#endif
```
So that means that `HAVE_STDARG_PROTOTYPES` is unset.
This can be set if we define `HAVE_STDARG_H`,
which we can do because PHP requires at least C99 in which the header
is always available.
We could also use an autoconf check, but this isn't really necessary as
it will always succeed.
Moves the usage of `mb_internal_encoding()` to INI section for the tests not testing the encoding/function itself, but the other mbstring/iconv functions.
