Discussion:
[Patch] fix non-english letters in root-menu -> windows and window-menu->In group issue
(too old to reply)
Wang Diancheng
2009-05-31 09:09:00 UTC
Permalink
Hi,

attachments fix non-english letters in non-root-menu -> windows and
window-menu->In group issue.

related Proposed Goals:

* Non-English Letters
* CJK Letters

related bug:

577913

This bug caused by function substring, when using this function, it can
lead to malformed UTF-8 characters.

I add a module to librep named utf8 located in 'rep.util.utf8', which
provide two function "utf8-string-length" and "utf8-substring", then
call them when extracting substring to window-name.

Please review my patches, thanks.

Wang Diancheng
Christopher Roy Bratusek
2009-05-31 10:16:37 UTC
Permalink
Am Sun, 31 May 2009 17:09:00 +0800
Post by Wang Diancheng
Hi,
attachments fix non-english letters in non-root-menu -> windows and
window-menu->In group issue.
* Non-English Letters
* CJK Letters
577913
This bug caused by function substring, when using this function, it
can lead to malformed UTF-8 characters.
I add a module to librep named utf8 located in 'rep.util.utf8', which
provide two function "utf8-string-length" and "utf8-substring", then
call them when extracting substring to window-name.
Please review my patches, thanks.
Wang Diancheng
Hi Wang,

long time ago I heard something from you :)

I Just tried it: and it's working!

While you're at UTF-8, could you try to fix the same issue in
Sawfish-UI?

That would be great!

Thanks in advance,
Chris
Wang Diancheng
2009-06-01 04:10:26 UTC
Permalink
Am Sun, 31 May 2009 17:09:00 +0800 schrieb Wang Diancheng
Post by Wang Diancheng
Hi,
attachments fix non-english letters in non-root-menu -> windows and
window-menu-> In group issue.
* Non-English Letters * CJK Letters
577913
This bug caused by function substring, when using this function,
it can lead to malformed UTF-8 characters.
I add a module to librep named utf8 located in 'rep.util.utf8',
which provide two function "utf8-string-length" and
"utf8-substring", then call them when extracting substring to
window-name.
Please review my patches, thanks.
Wang Diancheng
Hi Wang,
long time ago I heard something from you :)
I Just tried it: and it's working!
While you're at UTF-8, could you try to fix the same issue in
Sawfish-UI?
That would be great!
Thanks in advance, Chris
Hi, Christopher

I didn't read sawfish-ui code before, anyway, I'll try it.
Christopher Roy Bratusek
2009-06-01 07:07:29 UTC
Permalink
Am Mon, 01 Jun 2009 12:10:26 +0800
Post by Wang Diancheng
Am Sun, 31 May 2009 17:09:00 +0800 schrieb Wang Diancheng
Post by Wang Diancheng
Hi,
attachments fix non-english letters in non-root-menu -> windows and
window-menu-> In group issue.
* Non-English Letters * CJK Letters
577913
This bug caused by function substring, when using this
function, it can lead to malformed UTF-8 characters.
I add a module to librep named utf8 located in 'rep.util.utf8',
which provide two function "utf8-string-length" and
"utf8-substring", then call them when extracting substring to
window-name.
Please review my patches, thanks.
Wang Diancheng
Hi Wang,
long time ago I heard something from you :)
I Just tried it: and it's working!
While you're at UTF-8, could you try to fix the same issue in
Sawfish-UI?
That would be great!
Thanks in advance, Chris
Hi, Christopher
I didn't read sawfish-ui code before, anyway, I'll try it.
Hi,

... some more infos: I've marked all strings translatable (_ "string")
and they won't show up, if the widget is a label (if it's the GtkTree
or a GtkButton (GtkStock label), they are working properly), no matter
if containing non-utf8 chars or not, so sawfish-ui's i18n support is
badly broken on labels. One more thing: if you start sawfish with
LANG=en and want to start sawfish-ui with LANG=de you get both de and
en strings (I guess that's because sawfish-ui resets the lang not
correctly).

... I made a snapshot tarball for this change (as I don't want to fuck
up GIT): http://www.nanolx.org/free/sawfish-1.5.0-gamma.tar.bz2

make -C po update-po has been ran here.

Thanks for your efforts,
Chris
Wang Diancheng
2009-06-02 02:07:17 UTC
Permalink
Am Mon, 01 Jun 2009 12:10:26 +0800 schrieb Wang Diancheng
Am Sun, 31 May 2009 17:09:00 +0800 schrieb Wang Diancheng >
Post by Wang Diancheng
Hi,
attachments fix non-english letters in non-root-menu ->
windows >> and >> window-menu-> In group issue.
Post by Wang Diancheng
* Non-English Letters * CJK Letters
577913
This bug caused by function substring, when using this >>
function, it can lead to malformed UTF-8 characters.
Post by Wang Diancheng
I add a module to librep named utf8 located in
'rep.util.utf8', >> which provide two function
"utf8-string-length" and >> "utf8-substring", then call them when
extracting substring to >> window-name.
Post by Wang Diancheng
Please review my patches, thanks.
Wang Diancheng
Hi Wang,
long time ago I heard something from you :)
I Just tried it: and it's working!
While you're at UTF-8, could you try to fix the same issue in >
Sawfish-UI?
That would be great!
Thanks in advance, Chris
Hi, Christopher
I didn't read sawfish-ui code before, anyway, I'll try it.
Hi,
... some more infos: I've marked all strings translatable (_
"string") and they won't show up, if the widget is a label (if
it's the GtkTree or a GtkButton (GtkStock label), they are working
properly), no matter if containing non-utf8 chars or not, so
sawfish-ui's i18n support is badly broken on labels. One more
thing: if you start sawfish with LANG=en and want to start
sawfish-ui with LANG=de you get both de and en strings (I guess
that's because sawfish-ui resets the lang not correctly).
... I made a snapshot tarball for this change (as I don't want to
http://www.nanolx.org/free/sawfish-1.5.0-gamma.tar.bz2
make -C po update-po has been ran here.
Thanks for your efforts, Chris
We have mirgrated to intltool for translation, Now, transated strings
displayed in sawfish-ui are translated in two processes(sawfish,
sawfish-ui), so the strings come from sawfish process depend on
sawfish's locale, others depend on sawfish-ui's locale.

the solution is getting the sawfish's locale when sawfish-ui startup,
then set itself locale to same with sawfish process. Then we must modify
sawfish-ui to prevent it to transate again when it got strings from sawfish.
Teika Kazura
2009-06-02 02:31:51 UTC
Permalink
Hi.

I'll work on configurator issue, too. First to try will be:
$ git grep substring

Regards,
Teika (Teika kazura)
Christopher Roy Bratusek
2009-06-09 04:53:28 UTC
Permalink
Any news from sawfish-ui + utf8? (just wanted to ask)

Meanwhile I'll squash some other bugs :)

Chris

P.S.: I hope you fix it as this is -of course- currently blocking 1.5.0

Am Mon, 01 Jun 2009 12:10:26 +0800
Post by Wang Diancheng
Am Sun, 31 May 2009 17:09:00 +0800 schrieb Wang Diancheng
Post by Wang Diancheng
Hi,
attachments fix non-english letters in non-root-menu -> windows and
window-menu-> In group issue.
* Non-English Letters * CJK Letters
577913
This bug caused by function substring, when using this
function, it can lead to malformed UTF-8 characters.
I add a module to librep named utf8 located in 'rep.util.utf8',
which provide two function "utf8-string-length" and
"utf8-substring", then call them when extracting substring to
window-name.
Please review my patches, thanks.
Wang Diancheng
Hi Wang,
long time ago I heard something from you :)
I Just tried it: and it's working!
While you're at UTF-8, could you try to fix the same issue in
Sawfish-UI?
That would be great!
Thanks in advance, Chris
Hi, Christopher
I didn't read sawfish-ui code before, anyway, I'll try it.
Wang Diancheng
2009-06-09 05:12:51 UTC
Permalink
Hi Chris,
Sorry,
I'm so busy recently, so many jobs. And sawfish-ui use many lisp's knowledge,
I'm not good at it, anyway, I'll try to fix these bugs ASAP.

Wang Diancheng
Any news from sawfish-ui + utf8? (just wanted to ask) Meanwhile
I'll squash some other bugs :)
Chris
P.S.: I hope you fix it as this is -of course- currently blocking 1.5.0
Am Mon, 01 Jun 2009 12:10:26 +0800 schrieb Wang Diancheng
Am Sun, 31 May 2009 17:09:00 +0800 schrieb Wang Diancheng >
Post by Wang Diancheng
Hi,
attachments fix non-english letters in non-root-menu ->
windows >> and >> window-menu-> In group issue.
Post by Wang Diancheng
* Non-English Letters * CJK Letters
577913
This bug caused by function substring, when using this >>
function, it can lead to malformed UTF-8 characters.
Post by Wang Diancheng
I add a module to librep named utf8 located in
'rep.util.utf8', >> which provide two function
"utf8-string-length" and >> "utf8-substring", then call them when
extracting substring to >> window-name.
Post by Wang Diancheng
Please review my patches, thanks.
Wang Diancheng
Hi Wang,
long time ago I heard something from you :)
I Just tried it: and it's working!
While you're at UTF-8, could you try to fix the same issue in >
Sawfish-UI?
That would be great!
Thanks in advance, Chris
Hi, Christopher
I didn't read sawfish-ui code before, anyway, I'll try it.
Christopher Roy Bratusek
2009-06-19 23:53:45 UTC
Permalink
one additional info: the reson why the widgets have no label is pretty
simple: the function for creating the widgets does not receive one
(tested it via making the label non-optional), so we need to find the
point there the labels get lost ...

CHris
Teika Kazura
2009-06-20 06:45:19 UTC
Permalink
Hi, Chris.
Post by Christopher Roy Bratusek
one additional info: the reson why the widgets have no label is pretty
simple: the function for creating the widgets does not receive one
(tested it via making the label non-optional), so we need to find the
point there the labels get lost ...
You mean in sawfish.gtk or sawfish.ui, or in rep-gtk?

It sounds strange per se, but the problem in sawfish configurator is
that the text is cut off at the first non-english (non-ascii may be
more correct) letter, so I fear it may not be the direct cause.

Thanks,
Teika (Teika kazura)
Wang Diancheng
2009-07-28 10:22:38 UTC
Permalink
Hi All,

I am debugging this problem recently, but I found this problem caused by
some defect of librep. If someone is familiar with librep, please
continue to debug it.

Attachment is a test program, it gets window name from sawfish wm
through connecting socket directly. you can test it using a window with
a utf-8 window name, eg. use firefox open a website with a utf-8
title(eg. www.163.cn).

compiling:
firstly, please set CLIENT_PATH correctly, then:
gcc -rdynamic -o testclient testclient.c -ldl

usage:
run xwininfo get a window id (with utf-8 window name)
run testclient, input the window id, you will see the window name from
sawfish wm.

following is my result:
This is the actual window name:
Christopher Roy Bratusek
2009-07-28 12:56:31 UTC
Permalink
Hi Wang,

very good. I can fully reproduce it -of course- but well, my rep
knowlege does not allow me to help you here. But I'm sure you'll get
that fixed.

Keep up your good work,
Chris
Post by Wang Diancheng
Hi All,
I am debugging this problem recently, but I found this problem caused by
some defect of librep. If someone is familiar with librep, please
continue to debug it.
Attachment is a test program, it gets window name from sawfish wm
through connecting socket directly. you can test it using a window with
a utf-8 window name, eg. use firefox open a website with a utf-8
title(eg. www.163.cn).
gcc -rdynamic -o testclient testclient.c -ldl
run xwininfo get a window id (with utf-8 window name)
run testclient, input the window id, you will see the window name from
sawfish wm.
xwininfo: Please select the window about which you
would like information by clicking the
mouse in that window.
xwininfo: Window id: 0x1c0004e "163|163邮箱|163博客|163聊天室|163相册|信息龙163.CN--个人娱乐专家 - Mozilla Firefox"
0x1c0004e
window name is "163|163\0.)\/)*\/..\0-/\/..\//)|163\0--\/*-\/,*\0--\/..\/-*|163\0-0\/))\/**\0--\/-,\/.)\0--\/..\/-,|163\0-/\/,+\//0\0--\/).\/*,|\0-,\/0/\/-)\0-.\/))\/./\0.)\/0.\/,)163.CN--\0-,\//0\/.*\0-,\/0*\/0*\0--\/-0\//)\0-,\/0)\/*0\0-,\//0\/++\0--\/..\//. - Mozilla Firefox"
I think the bug must be exist in function "rep_call_lisp1", please see
the source file "src/server.c:110".
Hi Chris, Sorry, I'm so busy recently, so many jobs. And
sawfish-ui use many lisp's knowledge, I'm not good at it, anyway,
I'll try to fix these bugs ASAP.
Wang Diancheng
Any news from sawfish-ui + utf8? (just wanted to ask) Meanwhile
I'll squash some other bugs :)
Chris
P.S.: I hope you fix it as this is -of course- currently blocking 1.5.0
Am Mon, 01 Jun 2009 12:10:26 +0800 schrieb Wang Diancheng
Am Sun, 31 May 2009 17:09:00 +0800 schrieb Wang Diancheng >
Post by Wang Diancheng
Hi,
attachments fix non-english letters in non-root-menu ->
windows >> and >> window-menu-> In group issue.
Post by Wang Diancheng
* Non-English Letters * CJK Letters
577913
This bug caused by function substring, when using this >>
function, it can lead to malformed UTF-8 characters.
Post by Wang Diancheng
I add a module to librep named utf8 located in
'rep.util.utf8', >> which provide two function
"utf8-string-length" and >> "utf8-substring", then call them
when extracting substring to >> window-name.
Post by Wang Diancheng
Please review my patches, thanks.
Wang Diancheng
Hi Wang,
long time ago I heard something from you :)
I Just tried it: and it's working!
While you're at UTF-8, could you try to fix the same issue in
Sawfish-UI?
That would be great!
Thanks in advance, Chris
Hi, Christopher
I didn't read sawfish-ui code before, anyway, I'll try it.
einfaches Textdokument-Anlage (testclient.c)
#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>
#define CLIENT_PATH "/usr/lib/rep/i686-everest-linux-gnu/sawfish/client.so"
int main()
{
void *handle;
int (*client_open)(char *);
char *error;
char *(*client_eval) (char *form, int *lenp, int *errorp);
handle = dlopen(CLIENT_PATH, RTLD_LAZY);
if (!handle) {
fprintf(stderr, "%s\n", dlerror());
exit(EXIT_FAILURE);
}
dlerror(); /* Clear any existing error */
/* Writing: cosine = (double (*)(double)) dlsym(handle, "cos");
would seem more natural, but the C99 standard leaves
casting from "void *" to a function pointer undefined.
The assignment used below is the POSIX.1-2003 (Technical
Corrigendum 1) workaround; see the Rationale for the
POSIX specification of dlsym(). */
*(void **) (&client_open) = dlsym(handle, "client_open");
*(void **) (&client_eval) = dlsym(handle, "client_eval");
if ((error = dlerror()) != NULL) {
fprintf(stderr, "%s\n", error);
exit(EXIT_FAILURE);
}
(*client_open)(NULL);
int errorp;
int lenp;
char window_id[256];
char form[512];
printf("please input a window id(Note, window id must exists!):\n");
scanf("%s", window_id);
sprintf(form,"(window-name (get-window-by-id %s))",window_id);
char *res = client_eval(form, &lenp,&errorp);
printf("window name is %s\n",res);
dlclose(handle);
exit(EXIT_SUCCESS);
}
/*
** compile-command: "gcc -rdynamic -o testclient testclient.c -ldl"
*/
Teika Kazura
2009-08-01 07:39:47 UTC
Permalink
Hi, Wang Diancheng. (How shall I call you? Diancheng, Wang Diancheng,
Diancheng Wang, or maybe Dian-dian?) Thank you for your efforts!

I found how to do almost equivalent thing from sawfish-client:
------------------------------------------------------------------------
$ sawfish-client

sawfish 1.6.0, Copyright (C) 1999-2000 John Harper
sawfish comes with ABSOLUTELY NO WARRANTY; for details see the file COPYING

Enter `,help' to list commands.
user> (window-name (get-window-by-id #x10000a3))
------------------------------------------------------------------------
In rep, 0x... is represented by #x...

Seems that you gave us a big clue. The "\0" 's in output are null
bytes. (You can confirm it by redirect to file.) So it may be the
cause of unprinted text in configurator.


But the picture is not simple, and I'm confused.

Emacs' "sawfish mode" (http://www.davep.org/emacs/#sawfish.el) is an
interface to sawfish-client. When I eval:
(format standard-error "%s\n" (window-name (get-window-by-id #x1000045)))
within sawfish-mode, then the output (stderr) goes into ~/.saw-out (I
run sawfish as: $ sawfish &> ~/.saw-out), and **the window name is
printed there correctly**. Wow!

But when I do the same in sawfish-client in shell, I see what you saw.
(It's not encoding of shell. Redirect does not matter.)

(I thought sawfish-client connects to sawfish WM, so that WM outputs
(format standard-error ...). But no, sawfish-client outputs to shell's
stdout, not stderr.)

That's all I can say now.

Regards,
Teika (Teika kazura)

Teika Kazura
2009-06-02 01:27:43 UTC
Permalink
Hi, Wang Diancheng, thanks for great patch. Most are ok.

Tell me, in librep patch you included some macros, but for what? They
don't seem necessary, and have (minor) some shortcomings as described
below, they may be deleted.

There's a misleading point in UTF8_LENGTH. There (Char) takes ucs (or
Unicode 'codepoint' in more slack terminology) arg, unlike other
'Char's, which are a real byte. And UTF8_LENGTH converts ucs Char to
length in bytes in utf-8 representation, right? We don't seem to need
this macro.

On the other hand, UTF8_GET can be used to check the validity,
that is, it can check if the given string is really utf-8, so
may be of use. (More precisely, if it returns -1, then it's not
utf-8, but nothing can be said otherwise.) Anyway, because sawfish
doesn't handle so many strings, we don't need inline function, so if
it is to be rewritten (currently not necessary), it can be implemented
as a usual function.

Lastly, let me ask a question on copyright notice. You say that you
borrowed codes from glib, but "This file some code come from glib:" is
enough as copyright notice for GPL'd work?

Thanks a lot!

Teika (Teika kazura)
Wang Diancheng
2009-06-02 01:53:23 UTC
Permalink
Hi Teika,
Hi, Wang Diancheng, thanks for great patch. Most are ok. Tell me,
in librep patch you included some macros, but for what? They don't
seem necessary, and have (minor) some shortcomings as described
below, they may be deleted.
These macro can be removed, now. I left them here, just because I want
to provde more utf-8 related functions if needed in future. Anyway I
will send another patch to cleanup the code.
There's a misleading point in UTF8_LENGTH. There (Char) takes ucs
(or Unicode 'codepoint' in more slack terminology) arg, unlike
other 'Char's, which are a real byte. And UTF8_LENGTH converts ucs
Char to length in bytes in utf-8 representation, right? We don't
seem to need this macro.
On the other hand, UTF8_GET can be used to check the validity,
that is, it can check if the given string is really utf-8, so may
be of use. (More precisely, if it returns -1, then it's not utf-8,
but nothing can be said otherwise.) Anyway, because sawfish
doesn't handle so many strings, we don't need inline function, so
if it is to be rewritten (currently not necessary), it can be
implemented as a usual function.
Lastly, let me ask a question on copyright notice. You say that
you borrowed codes from glib, but "This file some code come from
glib:" is enough as copyright notice for GPL'd work?
Because many code come from glib, glib's copyright is GPL, in fact, I
don't know how to process this situation, suggestion and patch welcome! :)
Thanks a lot!
Teika (Teika kazura)
Christopher Roy Bratusek
2009-06-02 19:09:12 UTC
Permalink
Am Tue, 02 Jun 2009 09:53:23 +0800
Post by Wang Diancheng
Hi Teika,
Hi, Wang Diancheng, thanks for great patch. Most are ok. Tell
me, in librep patch you included some macros, but for what?
They don't seem necessary, and have (minor) some shortcomings
as described below, they may be deleted.
These macro can be removed, now. I left them here, just because I want
to provde more utf-8 related functions if needed in future. Anyway I
will send another patch to cleanup the code.
Then we'll keep them, rather than removing and re-adding them.
Post by Wang Diancheng
There's a misleading point in UTF8_LENGTH. There (Char) takes
ucs (or Unicode 'codepoint' in more slack terminology) arg,
unlike other 'Char's, which are a real byte. And UTF8_LENGTH
converts ucs Char to length in bytes in utf-8 representation,
right? We don't seem to need this macro.
On the other hand, UTF8_GET can be used to check the validity,
that is, it can check if the given string is really utf-8, so
may be of use. (More precisely, if it returns -1, then it's not
utf-8, but nothing can be said otherwise.) Anyway, because
sawfish doesn't handle so many strings, we don't need inline
function, so if it is to be rewritten (currently not
necessary), it can be implemented as a usual function.
Lastly, let me ask a question on copyright notice. You say that
you borrowed codes from glib, but "This file some code come from
glib:" is enough as copyright notice for GPL'd work?
Because many code come from glib, glib's copyright is GPL, in fact, I
don't know how to process this situation, suggestion and patch
welcome! :)
librep is also GPL'ed so a copyright notice in the file-header should
be enough (as the GPL is already beeing shipped)
Post by Wang Diancheng
Thanks a lot!
Teika (Teika kazura)
Thanks for your both efforts,
Chris
Loading...