Sigh. I included tests which were intended to check this case in the
test suite for ISO-2022-JP-MS, but those tests were faulty and didn't
actually test what they were supposed to.
Fixing the tests revealed that there were still bugs in this area.
There was a bit of legacy code here which looks like the original author
of mbstring intended to allow conversion of Unicode Private Use Area
codepoints to ISO-2022-JP-KDDI. However, that code never worked.
It set the output variable to values which were not matched by any
of the 'if' clauses below, which meant that nothing was actually
emitted to the output. In other words, if one tried to convert Unicode
to ISO-2022-JP-KDDI, and the Unicode string contained PUA codepoints,
they would be quietly 'swallowed' and disappear.
I don't know what ISO-2022-JP-KDDI byte sequences the author wanted
to map those PUA codepoints to, and anyways, this use case is so obscure
that there is little point in worrying about it. However, it is better
to remove the non-functioning code than to leave it in.
This means that if now one tries to convert PUA codepoints to
ISO-2022-JP-KDDI, those codepoints will be treated as erroneous rather
than silently ignored.
After mb_substitute_character("long"), mbstring will respond to
erroneous input by inserting 'long' error markers into the output.
Depending on the situation, these error markers will either look like
BAD+XXXX (for general bad input), U+XXXX (when the input is OK, but it
converts to Unicode codepoints which cannot be represented in the
output encoding), or an encoding-specific marker like JISX+XXXX or
W932+XXXX.
We have almost no tests for this feature. Add a bunch of tests to
ensure that all our legacy encoding handlers work in a reasonable
way when 'long' error markers are enabled.
Some text encodings supported by mbstring (such as UCS-4) accept 4-byte
characters. When mbstring encounters an illegal byte sequence for the
encoding it is using, it should emit an 'illegal character' marker,
which can either be a single character like '?', an HTML hexadecimal
entity, or a marker string like 'BAD+XXXX'.
Because of the use of signed integers to hold 4-byte characters,
illegal 4-byte sequences with a 'negative' value (one with the high
bit set) were not handled correctly when emitting the illegal char
marker. The result is that such illegal sequences were just skipped
over (and the marker was not emitted to the output). Fix that.
The "wchar" encoding isn't really an encoding -- it's what we
internally use as the representation of decoded characters.
In practice, it tends to behave a lot like the 8bit encoding when
used from userland, because input code units end up being treated
as code points.
This patch removes the wchar encoding from the public encoding
list and reserves it for internal use only.
Store the result of sqlite3_data_count() into a variable and check that inside a loop instead calling it directly all the time. GCC is not brave enough to figure out the function produces the same result every time and call it repeatedly. This change produces fairly small but measurable and consistent speedup.
Avoid code genearation for destructors of immutable parameters of internal functions that cannot be changed in place.
Keep destructors for immutable array parameters, because few function may modufy them in-pace.
basic_functions.c already has a macro for returning an ini
value to userland, so make use of it for get_cfg_var() as well,
after generalizing it to not write to return_value in particular.
In this case we should use the original internal handler. Otherwise
the trampoline will attempt to free the closure, but the function
being used is not actually part of a closure anymore.
We would end up freeing the function name twice here, once for
the original closure, and once for the rebound one.
Rather than further special casing the zend_closure_call_magic
case, always addref the function_name for internal functions,
the same we do for userland functions. To compensate, we need to
release the original function name when creating from callable
or call frame.
Fixes oss-fuzz #37695.
Alex Dowad