Subject: FVWM: large PipeReads

From: Gregg Dameron <[email-masked]>
Date: Wed, 17 Apr 2002 19:29:25 -0600

Running 2.4.5 on Solaris.

It seems that as a PipeRead script increases in size, fvwm2 struggles to
read and execute it.  The maximum seems to be 1024 bytes, measured by
the string output in debug mode that begins:

<<DEBUG>> about to attempt '<script here>'.

Larger than that, and things get strange.  Reading in the file
containing the PipeRead may give an error such as:

<<DEBUG>> Module switch -1, about to exec: '<truncated PipeRead script>'

<<DEBUG>> Module switch -1, about to exec: '<characters after truncation
point>'
<<ERROR>> No such command '<characters after truncation point>'

Or, at runtime, Bourne may report an unexpected end-of-file, or other
syntax error, where none exists.

I haven't studied the source code in detail, but in read.c, I see:

  char line[1024];

More than a coincidence?

I suppose one workaround is to Exec a separate script and FvwmCommand
the output, but I'm anxious to optimize performance.  Any help is
appreciated.  I guess my preference would be dynamic memory allocation
supporting arbitrarily large PipeReads.

BTW, I noticed that the "about to exec" debug messages chop off the last
character of the line read in.  In read.c:

    if (debugging)
      fvwm_msg(DBG,"ReadSubFunc","Module switch %d, about to exec:
'%.*s'",
        Module,strlen(tline)-1,tline);

Perhaps the "-1" shouldn't be there?

Gregg Dameron

--
Visit the official FVWM web page at <URL: http://www.fvwm.org/>.
To unsubscribe from the list, send "unsubscribe fvwm" in the body of a
message to [email-masked].
To report problems, send mail to [email-masked].

---

From: Dominik Vogt <[email-masked]>
Date: Thu, 18 Apr 2002 12:10:16 +0200

> I haven't studied the source code in detail, but in read.c, I see:

In fvwm, a single command line can not be longer than 1022
characters (1024 minus 1 for the string end minus 1 for the
trailing newline).

> the output, but I'm anxious to optimize performance.

No, the limit is everywhere, not just in piperead.  If that's too
small, the limit can be increased, but I wonder how you manage to
generate single commands that are bigger than 1k.

> Any help is

It's the trailing newline that gets cut off.

Bye

Dominik ^_^  ^_^

-- 
Dominik Vogt, email: [email-masked]
LifeBits Aktiengesellschaft, Albrechtstr. 9, D-72072 Tuebingen
fon: ++49 (0) 7071/7965-0, fax: ++49 (0) 7071/7965-20
--
Visit the official FVWM web page at <URL: http://www.fvwm.org/>.
To unsubscribe from the list, send "unsubscribe fvwm" in the body of a
message to [email-masked].
To report problems, send mail to [email-masked].

---

From: Gregg Dameron <[email-masked]>
Date: Thu, 18 Apr 2002 11:01:20 -0600

> generate single commands that are bigger than 1k.

I would vote in favor of an increase.  It's easy for a PipeRead script to
grow quickly, when you consider Bourne looping and branching constructs (for,
if, case), all of which count toward the limit.  There's a PipeRead in the
distribution (system.fvwm2rc-sample-95) that exceeds 500 bytes before
parameter expansion.  The "for i in ... ; do" statement alone is over 230
bytes.  Once I saw that sample, I jumped to the conclusion that arbitrarily
long PipeReads were not only supported, but tacitly recommended.

Admittedly, a contributing factor is my coding style - I prefer longer shell
variable names and newlines with indentation, all of which count toward the
limit (and certainly can be trimmed).

Perhaps the size limit should be mentioned in the man page (or did I miss
it?).

> It's the trailing newline that gets cut off.

Hmm.  Here's a sample of what I get:

[FVWM.1][ReadSubFunc]: <<DEBUG>> Module switch -1, about to exec:
'DestroyFunc ChangeAppsMa'
[FVWM.1][ReadSubFunc]: <<DEBUG>> Module switch -1, about to exec: 'AddToFunc
ChangeAppsMa'
[FVWM.1][ReadSubFunc]: <<DEBUG>> Module switch -1, about to exec: '+ I
KillModule FvwmTaskBa'
[FVWM.1][ReadSubFunc]: <<DEBUG>> Module switch -1, about to exec: '+ I
KillModule FvwmWinLis'
[FVWM.1][ReadSubFunc]: <<DEBUG>> Module switch -1, about to exec: '+ I
KillModule FvwmIconMa'
[FVWM.1][ReadSubFunc]: <<DEBUG>> Module switch -1, about to exec: '+ I
KillModule FvwmIconBo'
[FVWM.1][ReadSubFunc]: <<DEBUG>> Module switch -1, about to exec: '+ I Exec
echo "AddToFunc StartFunction I $0" > $FVWM_USERDIR/.start-apps-ma'
[FVWM.1][ReadSubFunc]: <<DEBUG>> Module switch -1, about to exec: '+ I $'

It doesn't happen this way on yours and others' systems?

As always, thank you for the quick response.

Gregg Dameron

---

From: [email-masked]
Date: Thu, 18 Apr 2002 13:44:46 -0400

Ahh, you are putting the whole script _IN_ the config file?

I think what Dominik meant was the return value needs to stay under 1022, maybe I'm wrong, can someone clarify?

The way I do it it to keep a script dir in my path and call them like this:

PipeRead "addExecutableOnMenu BrowserPopup opera galeon konqueror mozilla netscape cnetscape"

Where addExecutableOnMenu is it's normal readable self sitting somewhere in ~/somewhere:

#!/bin/bash
# 
#
MENU="$1"
shift
while [ "$1" != "" ]; do

which $1  2> /dev/null | {
        read fullpath
        if [ "$?" = "0" ]
        then
        echo "AddToMenu $MENU $1 Exec $fullpath"
        fi
}
shift
done

So if I have a script over 1024 it will fail even if it's sitting in an external file?

<snip>
> Perhaps the size limit should be mentioned in the man page (or did I miss
<snip>

k a k @ c i s c o . c o m

---

From: Mikhael Goikhman <[email-masked]>
Date: Thu, 18 Apr 2002 17:55:53 +0000

On 18 Apr 2002 11:01:20 -0600, Gregg Dameron wrote:
> grow quickly, when you consider Bourne looping and branching constructs
> (for, if, case), all of which count toward the limit.  There's a PipeRead
> in the distribution (system.fvwm2rc-sample-95) that exceeds 500 bytes
> before parameter expansion.  The "for i in ... ; do" statement alone is
> over 230 bytes.  Once I saw that sample, I jumped to the conclusion that
> arbitrarily long PipeReads were not only supported, but tacitly
> recommended.

Well, the limit was 256 I think, it was changed especially for
system.fvwm2rc-sample-95. :) I have nothing against to double it to 2Kb.
But the tactical recomendation is to use separate (not embedded) scripts.

> 'DestroyFunc ChangeAppsMa'
[...]
> It doesn't happen this way on yours and others' systems?

This was fixed 2 days after 2.4.5 was released, upgrade to 2.4.7.

Regards,
Mikhael.

---

From: Mikhael Goikhman <[email-masked]>
Date: Thu, 18 Apr 2002 18:15:00 +0000

Dear, k a k @ c i s c o . c o m.
Please don't set the From: header to [email-masked], it confuses others.
Setting it to [email-masked] would be ok.

On 18 Apr 2002 13:44:46 -0400, [email-masked] wrote:
> I think what Dominik meant was the return value needs to stay under
> 1022, maybe I'm wrong, can someone clarify?

No, the return value is completely ignored in PipeRead.

> So if I have a script over 1024 it will fail even if it's sitting in an external file?

Yes, of course this is a good solution. The solution in the topic does not
use separate shell scripts, it embeds the script in 1 line:

  PipeRead `for browser in opera galeon konqueror mozilla netscape cnetscape; do fullpath="`which $browser 2>/dev/null`"; if [ -n "$fullpath" ] then echo "AddToMenu BrowserPopup $browser Exec $fullpath; fi; done`

This can't currently exceed 1Kb.

Regards,
Mikhael.

---

From: Dan Espen <[email-masked]>
Date: Thu, 18 Apr 2002 14:14:48 -0400

> be I'm wrong, can someone clarify?

The limit is on the size of the command, not the result.

(The result can produce multiple lines, each one is a command.
So in the result, no line can exceed the limit.)

> PipeRead "addExecutableOnMenu BrowserPopup opera galeon konqueror mozilla net

Thats one way...

> So if I have a script over 1024 it will fail even if it's sitting in an exter

No.

Oops, looks like someone top posted:

>  
> <snip>
> > I would vote in favor of an increase.

You might get your increase, but I don't really favor it.
It just uses up some memory for an extreme case that can be avoided
pretty easily.

Heres FvwmForm-Form dealing with the same limit:

# Caution, this command is getting too long, if you add something,
# you might have some truncation occur:
*FvwmForm-Form: Command !(/bin/echo \
 "# This file last created by FvwmForm-Form on: `/bin/date`."; /bin/echo \
 '*FvwmFormDefault: Font $(Font)'            ; /bin/echo \
 '*FvwmFormDefault: InputFont $(InputFont)'  ; /bin/echo \
 '*FvwmFormDefault: ButtonFont $(ButtonFont)'; /bin/echo \
 '*FvwmFormDefault: TimeoutFont $(TimeoutFont)'; /bin/echo \
 '*FvwmFormDefault: Fore $(Fore)'            ; /bin/echo \
 '*FvwmFormDefault: Back $(Back)'            ; /bin/echo \
 '*FvwmFormDefault: Colorset $(Colorset)'    ; /bin/echo \
 '*FvwmFormDefault: ItemFore $(ItemFore)'    ; /bin/echo \
 '*FvwmFormDefault: ItemBack $(ItemBack)'    ; /bin/echo \
 '*FvwmFormDefault: ItemColorset $(ItemColorset)'       ; /bin/echo \
 '*FvwmFormDefault: ButtonPointer $(ButtonPointer)'     ; /bin/echo \
 '*FvwmFormDefault: ButtonInPointer $(ButtonInPointer)' ; /bin/echo \
 '*FvwmFormDefault: InputPointer $(InputPointer)' ; /bin/echo  \
 '*FvwmFormDefault: ActivateOnPress $(ActivateOnPress)'\
) > ${FVWM_USERDIR}/.FvwmForm
# This would have surely hit a limit so its appended with >>:
*FvwmForm-Form: Command !(\
 /bin/echo '*FvwmFormDefault: ButtonPointerFore $(ButtonPointerFore)';\
 /bin/echo '*FvwmFormDefault: ButtonPointerBack $(ButtonPointerBack)';\
 /bin/echo '*FvwmFormDefault: ButtonInPointerFore $(ButtonInPointerFore)';\
 /bin/echo '*FvwmFormDefault: ButtonInPointerBack $(ButtonInPointerBack)';\
 /bin/echo '*FvwmFormDefault: InputPointerFore $(InputPointerFore)';\
 /bin/echo '*FvwmFormDefault: InputPointerBack $(InputPointerBack)';\
) >> ${FVWM_USERDIR}/.FvwmForm

-- 
Dan Espen                           E-mail: [email-masked]
444 Hoes Lane  Room RRC 1C-214      Phone: (732) 699-5570
Piscataway, NJ 08854
--
Visit the official FVWM web page at <URL: http://www.fvwm.org/>.
To unsubscribe from the list, send "unsubscribe fvwm" in the body of a
message to [email-masked].
To report problems, send mail to [email-masked].

---

From: Gregg Dameron <[email-masked]>
Date: Thu, 18 Apr 2002 13:46:21 -0600

> It just uses up some memory for an extreme case that can be avoided

Thanks to all who have responded to this thread.

You're quite right, Dan.  My original posting campaigned for dynamic memory
allocation.  My motivation is to minimize disk reads after the configuration is
loaded.  True, reading and executing a short script from disk is a small price to
pay; on an idle system you'd never know the difference.  But the system here on
which fvwm runs tends to be anything but idle.  When interactive performance gets
sluggish, our users get cranky.

Bottom line - PipeRead is brilliant, indispensable - I'd like to see it be more
so.

Gregg Dameron

---

From: Dominik Vogt <[email-masked]>
Date: Fri, 19 Apr 2002 12:27:52 +0200

> Bottom line - PipeRead is brilliant, indispensable - I'd like to see it be more

But PipeRead already starts an external program.

Bye

Dominik ^_^  ^_^