"auto" is only meaningful in functions which accept an encoding
*list* and support encoding detection. These functions have
explicit checks for "auto". It cannot be used as a standalone
encoding in any meaningful capacity, so I'm dropping it entirely.
Implements 8bit conversions equivalently to iso-8859-1 conversions.
This seems quite dubious to me, but seems to match the previous
behavior.
It might make more sense to map the characters into a private area
instead, so that the 8bit encoding is treated as binary data with
no case conversions (including no case conversions in the ascii
range).
This patch adds missing newlines, trims multiple redundant final
newlines into a single one, and trims redundant leading newlines.
According to POSIX, a line is a sequence of zero or more non-' <newline>'
characters plus a terminating '<newline>' character. [1] Files should
normally have at least one final newline character.
C89 [2] and later standards [3] mention a final newline:
"A source file that is not empty shall end in a new-line character,
which shall not be immediately preceded by a backslash character."
Although it is not mandatory for all files to have a final newline
fixed, a more consistent and homogeneous approach brings less of commit
differences issues and a better development experience in certain text
editors and IDEs.
[1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_206
[2] https://port70.net/~nsz/c/c89/c89-draft.html#2.1.1.2
[3] https://port70.net/~nsz/c/c99/n1256.html#5.1.1.2
This patch adds missing newlines, trims multiple redundant final
newlines into a single one, and trims redundant leading newlines.
According to POSIX, a line is a sequence of zero or more non-' <newline>'
characters plus a terminating '<newline>' character. [1] Files should
normally have at least one final newline character.
C89 [2] and later standards [3] mention a final newline:
"A source file that is not empty shall end in a new-line character,
which shall not be immediately preceded by a backslash character."
Although it is not mandatory for all files to have a final newline
fixed, a more consistent and homogeneous approach brings less of commit
differences issues and a better development experience in certain text
editors and IDEs.
[1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_206
[2] https://port70.net/~nsz/c/c89/c89-draft.html#2.1.1.2
[3] https://port70.net/~nsz/c/c99/n1256.html#5.1.1.2
If we don't know how to convert between two encodings, make sure
we error instead of ignoring the issue.
Explicitly use vtbl_pass if we are round-tripping wchar->wchar or
8bit->8bit. Fingers crossed that nothing else relies on the
vtbl_pass fallback...
The C89 standard and later defines the `<string.h>` header as part of
the standard headers [1] and on current systems it is always present.
Code included also `<strings.h>` header as an alterinative in some
files. This kind of check was relevant on some older systems where the
`<strings.h>` file included definitions for the C89 compliant
`<string.h>`. Today such alternative check is not required anymore. The
`<strings.h>` file is part of the POSIX definition these days.
Also Autoconf suggests doing this and relying on C89 or above [2] and [3].
This patch also cleans few unused `<strings.h>` inclusions in the libmbfl.
[1]: https://port70.net/~nsz/c/c89/c89-draft.html#4.1.2
[2]: http://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/autoconf/headers.m4
[3]: https://www.gnu.org/software/autoconf/manual/autoconf-2.69/autoconf.html
The `<assert.h>` header file is part of the standard C89 headers [1] and
on older systems there needed to be also a manual check if header is
present.
Since PHP requires at least C89 manual check and the `HAVE_ASSERT_H`
symbol defined by Autoconf in configure.ac can be both removed [2].
This patch also removes unused <assert.h> includes where c files don't
use the `assert()` macro.
Refs:
[1] https://port70.net/~nsz/c/c89/c89-draft.html#4.2
[2] https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/autoconf/headers.m4
The `<stddef.h>` header file is part of the standard C89 headers [1] and
on current systems there is no need for a manual check if header is
present.
Since PHP requires at least C89 the `HAVE_STDDEF_H` symbol isn't defined
by Autoconf anywhere else anymore [2] and accross the PHP source code
the header is included unconditionally already.
This patch syncs this also for the bundled libmbfl which is maintaned as
a fork in php-src.
Refs:
[1] https://port70.net/~nsz/c/c89/c89-draft.html#4.1.2
[2] https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/autoconf/headers.m4
Autoconf 2.59d (released in 2006) [1] started promoting several macros
as not relevant for newer systems, including the `AC_C_CONST`.
The `const` keyword is used in C since C89. On old systems some compilers
lacked the `const` and this macro defined it to be empty. This check was
relevant on systems with compilers before C89 and on current systems it
can be omitted. [2]
PHP also requires at least C89 so `const` is always available.
Refs:
[1] http://git.savannah.gnu.org/cgit/autoconf.git/tree/NEWS
[2] https://www.gnu.org/software/autoconf/manual/autoconf-2.69/autoconf.html
The $Id$ keywords were used in Subversion where they can be substituted
with filename, last revision number change, last changed date, and last
user who changed it.
In Git this functionality is different and can be done with Git attribute
ident. These need to be defined manually for each file in the
.gitattributes file and are afterwards replaced with 40-character
hexadecimal blob object name which is based only on the particular file
contents.
This patch simplifies handling of $Id$ keywords by removing them since
they are not used anymore.
One set of parenthesis was missing, causing a legitimate compiler
warnings. In the end it doesn't actually matter, because it just
ends up doing an unnecessary check in the w > 0 case.
This fixes the logic and moves it out into a separate functions,
to be a bit more readable.
The libmbfl library is bundled with PHP and has its own repository for
development and bug fixes. To avoid confusion and faster development the
README has been updated to include the information of the original library and
to use the bundled library as a fork of the upstream repository instead.
For functions like mb_chr() and mb_ord() just looking up the
input/output filter for the encoding dominates the runtime. This
commit stores the input/output filter for an encoding in the
mbfl encoding structure, so it can be looked up directly, rather
than scanning through filter function lists.
The introduced checks were not correct in two respects:
* It was checked whether the source encoding of the string matches
the internal encoding, while the actually relevant encoding is
the *target* encoding.
* Even if the correct encoding is used, the checks are still too
conservative. Just because something is not a "Unicode-encoding"
does not mean that it does not map any non-ASCII characters.
I've reverted the added checks and instead adjusted mbfl_convert
to first try to use the provided substitution character and if
that fails, perform the fallback to '?' at that point. This means
that any codepoint mapped in the target encoding should now be
correctly supported and anything else should fall back to '?'.
This takes the substr from the offset to the end of the string.
This avoids pointless searching for the end position and also
saves us a length calculation in the strstr family of functions.
The HTML decoding filter uses the `opaque` member of mbfl_convert_filter
as buffer, but there was no copy constructor defined, what caused double
frees when the filter is copied (what happens multiple times in mb_strcut(),
for instance).
Also prune out some duplicate code and use strlen() and memcpy()
instead of ad-hoc reimplementations. Remove multiplications by
sizeof(unsigned char), which wrongly imply that this can be
anything but 1.
