Unicode support

[Ang Bodhi <bodhi@xxxxxxxxxxxxxx>]

Hi,

In the SoText2 et al code, it says i18n only work with Irix 6,
and Linux is dumped in the non-i18n category with Irix 5.
Firstly, I think this shouldn't be as Linux has iconv.
Secondly, it seems the only i18n requirement is the use of
iconv to convert UTF8 string to UCS2 string. If this is so,
I think the conversion can be replaced by an algorithm
published in the book The Unicode Standard, and the iconv
code be removed (thus Irix 5 can also benefit). I suggest
the use of a UCS2 string class SbUCS2String to replace the
existing UCS string representation in
SoBitmapFontCache/SoOutlineFontCache:

<   // char* pointers of UCS-2 strings:
<   SbPList UCSStrings;
<   // size of these strings, in UCS-2 characters:
<   SbPList UCSNumChars;
----------
>   // SbUCS2String* pointers of UCS-2 strings:
>   SbPList UCSStrings;

The UTF8 to UCS2 conversion algorithm can be embedded in the
SbUCS2String class, thus conversion can be more easily done:

<   UCSStrings[i] = new char[2*strings[i].getLength()+2];
<
<   char* input = (char *)strings[i].getString();
<   size_t inbytes = strings[i].getLength();
<   size_t outbytes = 2*inbytes+2;
<   char* output = (char*)UCSStrings[i];
<
<   if ((iconv(conversionCode, &input, &inbytes, &output, &outbytes) == (size_t)-1)){
<       // error
<   }
<
<   if (inbytes){
<       // error
<       return;
<   }
<
<   UCSNumChars[i] = (void*)((2*strings[i].getLength()+2 - outbytes)>>1);
----------
>   UCSStrings[i] = new SbUCS2String (strings[i].getString());

Subsequently string processing will become easy:

<   const char *str = getUCSString(line);
<
<   for (int i = 0; i < getNumUCSChars(line); i++) {
<       GLuint c = str[2*i]<<8 | str[2*i+1];
<       // hard working
<   }
----------
>   const SbUCS2 *str = getUCSString(line)->getString();
>   SbUCS2 c;
>
>   while ((c = *str++)) {
>       // hard working
>   }

and conversion back to UTF8 string (in case needed) can be
done with:

    char *str = getUCSString(line)->getUTF8String();

What do you think? :) To prove this concept, I've wrote the
SbUCS2String class complete with the conversion algorithm
(cut'n'paste from the Unicode book), and tested it with a
modified version of SoText2, as shown here displaying
Chinese character strings.

I've also updated libFL-src2 (support Unicode string, and
clean up (more to do)). Is there a public dir to upload files
for sharing? :)

Ang Bodhi