Subject: Strings Conversions

From: Olivier Chapuis <[email-masked]>
Date: Tue, 2 Jul 2002 18:50:32 +0200

I recall that now under certain situation fvwm or a module
needs to perform some strings conversions. At present time
the conversions are performed each time you draw the string.

Here some examples (when you need conversion):
(1) you have a menu encoded in UTF-8 and to display this menu in
the good way you use an iso10646-1 (core) font. To be able to display
the menu, fvwm should convert the menu labels into a (XChar2b *) struct
in USC-2 encoding [1].
(2)  you have a menu encoded in UTF-8 and want to use an iso8859-1
(core) font, so you indicate to fvwm that the StringEncoding is UTF-8.
To be able to display the menu, fvwm converts (using iconv) the menu
labels from UTF-8 to iso8859-1.
(3) xft font leads to various conversion. Basically if your locale
charset is not iso8859-1, fvwm should convert into USC-2.
(4) bidi conversions are some times needed

So if we do not want to perform some str conversions each time we
draw a string we should keep the result of the conversions in memory.

The point of this mail is that I'm not sure that we should do
that and I would like to have some advice. I've made some
experimentation so that we can take a decision. It is clear
that the advantage of keeping the result of the conversion
is to speed up the code and to save some CPU. The disadvantage
is that more memory is used and that some special work should be
done when the font change (or at least when the string/font encoding
change).

I've (partially) implemented this "keep in memory converted
string" idea on my machine by introducing a new struct

typedef struct
{
	char *str;                  /* this original string */
	char *e_str;                /* the encoded string or str */
	XChar2b *str_2b;            /* the encoded string or NULL */
	int e_len;                  /* lenght of the encoded string */
	unsigned is_encoded : 1;
} FlocaleString;

and by replacing all the (char *)str which should be displayed by
fvwm by such a structure. The first time an FlocaleString should be
displayed conversions are performed (if needed) and the next times
we do not needed to do any conversion (if the font change the encoded
part of the structure is freed and is_encoded is set to zero). 
I've fully implemented this in menus.
 
So my first test is to start fvwm with fvwm-themes and to use an
iso10646-1 font for the menus. Then, pop up all the fvwm-themes
root menus with depth =< 2 (I've two generated menus: kde3 menu
and the xlock menus).
At start up the new code uses ~ 44 Kb of additional memory, at the end
of the test it uses ~ 136 Kb of additional memory. This does not seem
dramatic. On the other hands I see no visible difference.

My second test is about speed up. I've wrote an "hello world" program
using libfvwm.a. Each times you click on the window it draws 1000
times a given string with a given fvwm font using the functions of
the libs.

Here some results (I've a 256Mhz AMD k6 processor :o( )

(a) An iso-10646-1 core font and an UTF-8 strings of 240 characters
(repetition of "test éè" in UTF-8, so the converted string is of length
< 240):
1000 drawing take 0.25 sec with the new code
1000 drawing take 0.50 sec with the current code

(b) An iso8859-15 core font with StringEncoding=iso8859-15 and the
same UTF-8 string as (a):
1000 drawing take 1.5 sec with the new code
1000 drawing take 1.7 sec with the current code

(c) the same test  but replace iso8859-15 with iso8859-1
1000 drawing take 0.20 sec with the new code
1000 drawing take 0.30 sec with the current code

(d) An iso8859-15 core font with an ascii strings of 240 characters
(non conversion):
1000 drawing take 0.34 sec with both code

(e) An iso8859-1 core font with the same string than (d)
(non conversion):
1000 drawing take 0.28 sec with both code

(f) Drawing with an xft font a "locale" string of length 240 (needs
a conversion from conversion iso8859-15 to USC-2 here).
1000 drawing take 4.63 sec with the new code
1000 drawing take 5.00 sec with the current code

For me the result of the test (b) vs the result of test (c) was
totally mysterious ... As my locale charset is iso8859-15
(b) use a FontSet as (c) use a FontStruct. (d) and (e) show that
Xmb drawing is slow when you have a alternation of ascii and non
ascii characters and not so slow if it is not the case.

So, the conclusion is that the new code is a bit faster, but nothing
visible or important. Other factors than conversion seems more
important [2] and any way the Xmb* functions do a lot of conversions
each times. The additional memory seems to me not very important too.

So, I do not know what to do. Some suggestions?

Regards, Olivier

Note: [1]
typedef struct {
    unsigned char byte1;
    unsigned char byte2;
} XChar2b;

[2] It seems that some part of the Xlib i18n code is broken.
In fact I begun to work on this subject one week ago. But I
spent a lot of time on the Xlib code... The first thing is that
there are memory leaks in XCreateFontSet/XFreeFontSet (a FontStruct
is not freed at some point + other minor leaks). So the fvwm-2.4.x
memory problems with --enable-multibyte is not fvwm fault but Xlib fault.
These memory leaks are less visible in the 2.5.x series because we
have a font cache. I've sent a patch to the XFree team that fixes the
leaks but I get no feedback yet. Moreover, it seems that too many
FontStruct is loaded when you create a FontSet so I think that the
Xmb* functions can be speed up. Basically, the Xlib i18n code needs
a clean up.
--
Visit the official FVWM web page at <URL:http://www.fvwm.org/>.
To unsubscribe from the list, send "unsubscribe fvwm-workers" in the
body of a message to [email-masked].
To report problems, send mail to [email-masked]