Re: windows 936

[Martin Duerst <duerst@it.aoyama.ac.jp>]

Dealing with minor differeces and variants as notes is definitely
one possibility. However, I think sooner rather than later, we
should look at a more syntematic way of indicating variants
and extensions.

Here is an extremely rough strawman:

a) Identify a character that's okay in charset tags but rarely
   used (e.g. '+', don't even know whether that's okay)
b) Use this character to separate base tag and variants, e.g.
   base tag: Shift_jis
   tag with variant: Shift_jis+cp932

Shift_jis would only indicate that this is some kind of shift_jis.
Applications that don't care too much about variants would just
use this. Shift_jis+cp932 indicates the variant with the Microsoft
additions. Applications on the receiving end not interested in
variants would have to cut off trailing '+' and what's after.

The above proposal isn't without problems, but addresses the
second most fundamental problem in the current scheme.

(The first most fundamental problem is that stuff is often
tagged wrongly. But that's a much harder problem than the variants.)

Regards,    Martin.

At 10:51 07/05/22, Erik van der Poel wrote:
>Most of the Windows code pages are "supersets" of other standard sets.
>But rather than adding new charset names for these supersets, it might
>be better to add comments to the existing registrations to point out
>the relationships between the various sets.
>
>For example, the windows-936 registration might refer to the gb2312
>one, the windows-31j registration might refer to Windows Code Page 932
>and the Shift_JIS registration, the EUC-KR registration might refer to
>CP 949 and the Big5 registration to CP 950. All as informative
>references, rather than normative, I think.
>
>This promotes interoperability while avoiding the addition of more
>names and "virtual" aliases.
>
>Erik
>
>On 5/21/07, Shawn Steele <Shawn.Steele@microsoft.com> wrote:
>>
>>
>>
>>
>> I am looking at the registrations for the remaining 4 "system" code pages:
>> 932, 936, 949 & 950.  This seems complicated since IE uses other names for
>> them.
>>
>>
>>
>> For example, for 936, IE recognizes Chinese, CN-GB, csGB2312, csGB231280,
>> csISO58GB231280, GB2312, GB2312-80, GB231280, GBK, GB_2312_80, iso-ir-58,
>> and, of course its known to the system as 936.
>>
>>
>>
>> Our APIs report this code page as being "gb2312"
>>
>>
>>
>> There is an existing registration for GBK, aliases of CP936, MS936 and
>> windows-936, but not of the gb2312 name.  The existing registration points
>> to broken links at Microsoft and IBM.  This should probably be updated to
>> point to:
>>
>>
>>
>> http://www.microsoft.com/globaldev/reference/dbcs/936.mspx
>>
>> http://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP936.TXT
>> and
>>
>> http://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WindowsBestFit/bestfit936.txt
>>
>>
>>
>> I am a bit uncertain that GBK == 936, although this is what the existing
>> registration implies.
>>
>>
>>
>> The alternative solution would seem to be to register a new charset as
>> "windows-936" with the same additional aliases as the GBK registration and
>> point to the above tables.  This would then also lead to the question of
>> whether GBK and gb2312 should be listed as aliases for any such windows-936
>> code page although the interpretation of those aliases could differ for
>> other systems.
>>
>>
>>
>> My goal is to clarify the Microsoft system code page mappings such as for
>> 932, 936, 949 & 950, and I'd appreciate any suggestions about how to best do
>> that J
>>
>>
>>
>> Thanks,
>>
>>
>>
>> - Shawn
>>
>>
>>
>> Shawn Steele
>>
>> SDE
>>
>> Windows International
>>
>> Microsoft
>>
>>


#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:duerst@it.aoyama.ac.jp