fbdcab953d
To give the background on this issue, here is an excerpt from JAPANESE.txt,
from the Unicode Consortium:
Apple has defined a block of 32 corporate characters as "transcoding
hints." These are used in combination with standard Unicode characters
to force them to be treated in a special way for mapping to other
encodings; they have no other effect. Sixteen of these transcoding
hints are "grouping hints" - they indicate that the next 2-4 Unicode
characters should be treated as a single entity for transcoding. The
other sixteen transcoding hints are "variant tags" - they are like
combining characters, and can follow a standard Unicode (or a sequence
consisting of a base character and other combining characters) to
cause it to be treated in a special way for transcoding. These always
terminate a combining-character sequence.
The transcoding coding hints used in this mapping table are:
0xF860 group next 2 characters as a single entity for transcoding
0xF861 group next 3 characters as a single entity for transcoding
0xF862 group next 4 characters as a single entity for transcoding
0xF87A variant tag for "negative" (i.e. black & white reversed)
0xF87E variant tag for vertical form
0xF87F variant tag for other alternate form
For example, the Apple addition character 0x85AB is Roman numeral
thirteen. There is no single Unicode for this (although there are
standard Unicodes for Roman numerals 1-12). Using the grouping hint
0xF862 in combination with standard Unicodes, we can map this as
0xF862+0x0058+0x0049+0x0049+0x0049 (i.e. X + I + I + I).
Our SJIS-mac conversion code actually recognizes some special sequences
which start with an Apple 'transcoding hint'. However, if a transcoding
hint is misplaced and is not followed by one of the expected sequences,
we can just emit one error marker for the bad transcoding hint and then
process the following codepoint as normal.
libmbfl
This is libmbfl, a streamable multibyte character code filter and converter library, written by Shigeru Kanemoto.
The original version of libmbfl is developed and distributed at https://github.com/moriyoshi/libmbfl under the LGPL 2.1 license. See the LICENSE file for licensing information.
The libmbfl library is bundled with PHP as a fork of the original repository and is not in sync with the upstream. As such, the libmbfl directory is directly modified in the php-src repository.
Changelog
October 2017
- Since 2017, it is forked and bundled in the php-src repository. For the list of changes related to PHP see the PHP NEWS change logs.
Version 1.3.2 August 20, 2011
- Added JISX-0213:2004 based encoding : Shift_JIS-2004, EUC-JP-2004, ISO-2022-JP-2004 (rui).
- Added gb18030 encoding (rui).
- Added CP950 with user user defined area based on Big5 (rui).
- Added mapping for user defined character area to CP936 (rui).
- Added UTF-8-Mobile to support the pictogram characters defined by mobile phone carrier in Japan (rui).
Version 1.3.1 August 5, 2011
- Added check for invalid/obsolete utf-8 encoding (rui).
Version 1.3.0 August 1, 2011
-
Added encoding conversion between Shift_JIS and Unicode (6.0 or PUA) for pictogram characters defined by mobile phone carrier in Japan (rui).
-
Fixed encoding conversion of cp5022x for user defined area (rui).
-
Added MacJapanese (SJIS-mac) for legacy encoding support (rui).
-
Backport from PHP 5.2 (rui).
Version 1.1.0 March 02, 2010
- Added cp5022x encoding (moriyoshi)
- Added ISO-2022-JP-MS (moriyoshi)
- Moved to github.com from sourceforge.jp (moriyoshi)
Earlier versions
- 1998/11/10 sgk implementation in C++
- Rewriting with sgk C 1999/4/25.
- 1999/4/26 Implemented sgk input filter. Add filter while estimating kanji code.
- 1999/6 Unicode support.
- 1999/6/22 Changed sgk license to LGPL.
Credits
Marcus Boerger helly@php.net Hayk Chamyan hamshen@gmail.com Wez Furlong wez@thebrainroom.com Rui Hirokawa hirokawa@php.net Shigeru Kanemoto sgk@happysize.co.jp U. Kenkichi kenkichi@axes.co.jp Moriyoshi Koizumi moriyoshi@php.net Hironori Sato satoh@jpnnet.com Tsukada Takuya tsukada@fminn.nagano.nagano.jp Tateyama tateyan@amy.hi-ho.ne.jp Den V. Tsopa tdv@edisoft.ru Maksym Veremeyenko verem@m1stereo.tv Haluk AKIN halukakin@gmail.com