Re: nofork command substitution

[Robert Elz]

    Date:        Fri, 19 May 2023 12:03:51 -0400
    From:        Chet Ramey <chet.ramey@case.edu>
    Message-ID:  <0a85095a-1665-d936-b4fa-118dd158e5a8@case.edu>


  | Maybe, and certainly possible, but a more likely use is just a simple
  | assignment to REPLY.

In such cases, the value to be assigned needs to come from somewhere,
and I can't really see a lot of benefit in using this indirect method
of getting that value, rather that simply putting the somewhere directly
in the word, instead of this command substitution, except in the most
unlikely of cases (like when REPLY is built up gradually, via a loop
gradually adding more text).

  | I envision a little more complexity than that; a ${| printf -v REPLY ...; }
  | doesn't really require the command substitution at all.

Nor does a simple assignment to REPLY

  | Probably. The bash implementation is the union of mksh and ksh93's (with
  | one exception, see below), and that part comes from mksh.

Ah, that info wasn't apparent earlier - I wasn't aware this was just
a copy of earlier implementations, it seemed more like something new.

  | The other benefit is not to have to output any text if you don't have to,
  | and for the calling shell not to have to read it.

That's all pseudo-noise --- the shell would need to set REPLY, and then
expand its value.  The alternative is simply to "write" the output into
a buffer in memory which the shell then "reads" when it wants the results
of the command substitution.  Those two are really no different, except
that the REPLY form does all of the work needed to assign to a var, and
then access it (one way or another) - plus everything necessary to make
REPLY a local variable.

Of course all of this only works simply if the commands in the cmdsub
generating the text are built in to the shell, if something like sed
is being used (or awk) then you have to read its output anyway.

  | I would have thought it obvious the space, tab, and newline variants do
  | the execute-command-in-the-current-environment-and-capture-output thing,
  | as described first, where the exceptions are explicitly detailed and
  | everything else is invalid. I can make the former even more explicit.

The question is more (aside from "compat with mksh" which I didn't previously
know was an aim - not sure 100% compat needs to be anyway) why 3 different
but equivalent forms are useful.   The space variant is obvious, that's how
we get this to be a cmdsub rather than a var expansion.   I can see the use
of the newline form, so one can write

        cmd arg arg word${
                sed -n /^p/p /usr/dict/words
                        }more

or something, without needing to put a hard to see space after the '{'.

I can't however see any real rational purpose for the tab variant.

  | I decided not to seek deliberate incompatibility with mksh.

Sure, that makes sense, but you don't need to necessarily implement
everything they implemented, when there is no need for it.

  | Remember where I said this was the union of the mksh and ksh93 features?
  | This one is from ksh93. It's not really any different from $(...). It
  | doesn't cost anything additional to support, either.

It might not cost anything to implement, but it needs to be explained,
documented, and then users forever need to try and understand why there
are two (seemingly equivalent) ways to achieve the same thing, and try
to work out which of them is the right one to use.

Introducing complexity, no matter how cost free it is to implement, is
rarely cost free in reality.

  | You can have that, if you want. x=${ func; } (or x=${ func 1 2 3; }) does
  | the right thing. The ${...;} inherits the positional parameters, so things
  | like `shift' work as expected in the body,

I'm not sure I'd call what happens there "as expected" - I'd just call
it weird.

There's also the issue of in exactly what context these expansions happen.
For words that are the command name, or args, that's fairly simple, the
expansions occur in the context of the parent shell (the one running the
script or whatever).   But for expansions in redirects, as in

        >${ shift 3; echo $2 ;}

or something, then redirects occur in the context of the command about
to be executed (ie: after the fork) and some shells perform the expansions
on the redirect word then as well (others perform them in the parent shell).

In mksh for example (since I now know I can test this using that):

$ set -- a b /tmp/file
$ ls >${ shift 2; echo $1; }
$ echo $1
a

The output goes to /tmp/file, but $1 has not altered in the parent shell.
I don't have this code in bash to test, but it appears as if bash will do
the same.   The NetBSD shell would not, expansions in redirects (but not
the actual redirections) are performed in the parent shell, so side effects
in those are just as visible as any other expansions in the command.

[Aside: when I first tried this, I forgot the need for the ';' before the '}'
 but, surprisingly, it still worked exactly the same way, mksh it seems
simply finds a terminating '}' and uses it - reserved word type matching not
required.]

  | It's to allow local variables and `return'. The existing implementations
  | all agree on that.

Sure, the question isn't that, it is why it doesn't also make local
positional params, just the same as any other function.   (But return
and local can both be made to work in any context you like, there's
nothing that says they are only allowed in a function, or something like
it, other than any rules you impose upon yourself ... it seems like as
described, the "${ " form is more like execution of a '.' script, than
a function.

  | Expansions are performed left-to-right (or, if you prefer, beginning to
  | end) in the word, and the command substitution modifies the current
  | execution environment.

Sure, but last time I looked, it was still unspecified just when side
effects of expansions get applied.

  | There's no reason to be deliberately incompatible.

Sure, understood, but also no reason to implement nonsense, or frills
which are not needed.  Make what is implemented be compatible, by all
means, but that does not mean you need to implement everything.

After all mksh doesn't attempt to be compat with bash wrt brace expansion.
bash does it the sane way (first) - mksh (maybe kshNN as well) the insane
way (after parameter expansions) - nothing at all compatible there.

  | The only thing I just
  | couldn't stomach implementing was making `exit' in the body of this form
  | of command substitution act like `return' and not exit the shell, which
  | ksh93 does.

Fully understand, I wouldn't do that either.

  | Well, that won't do anything since it's in double quotes

Yes, ... those were (probably) just for exposition ... not actually intended.

The issues with all of this are more the increased complexity for the user
trying to work out how it all works, and why.

kre