Re: nofork command substitution

[Chet Ramey]

On 5/19/23 6:11 AM, Robert Elz wrote:
>      Date:        Thu, 18 May 2023 22:14:28 -0400
>      From:        "Dale R. Worley" <worley@alum.mit.edu>
>      Message-ID:  <874jo9kqyj.fsf@hobgoblin.ariadne.com>
>
>    | Chet Ramey <chet.ramey@case.edu> writes:
>    | > Bash allows the close brace to be joined to the remaining
>    | > characters in the word without being followed by a shell metacharacter
>    | > as a reserved word would usually require.
>    |
>    | I had to read this a couple of times to figure out what it means.  In
>    | particular "the word" isn't well-bound here.  Perhaps better is
>
> I'm not sure why this is worth mentioning at all, any more than that
> more text, that is part of the same word, can follow a ${var} expansion.

Because you know someone will eventually ask about it.

> These things (variable expansions, command substitutions, arithmetic
> expansion, tilde exbansion, and in bash and some other shells, brace
> expansion) are all word expansions.   All of them occur anywhere in a
> shell "word", and I don't see this proposed new form as being any different.

It's not. It's command substitution.

> The only oddity is that inside the nofork cmd sub, the closing '}' is being
> treated as a reserved word (as it is when used for grouping lists), but from
> the outside, it is just a part of the word that contains it.

Exactly.

>    | My guess is that the intention is for COMMAND to be "read" with no names:
>    |     ${| read; }
>
> I'd have thought a more likely example would be
>
> 	${| printf -v REPLY 'whatever format' arg... ; 

Maybe, and certainly possible, but a more likely use is just a simple
assignment to REPLY.

> to get specifically formatted text into a shell word, without needing
> a fork, which the alternate
>         $( printf 'whatever format' arg.... )
> would require.

I envision a little more complexity than that; a ${| printf -v REPLY ...; }
doesn't really require the command substitution at all.

> Use of REPLY in particular as the variable to contain the result seems
> like just because it is used that way in read (where it is never really
> needed, that's the ultimate of useless frills) and select, and so is a var
> name already more or less reserved for the shell's internal use (like OPTIND,
> OPTARG, IFS, ...) and was just used for the purpose.  

Probably. The bash implementation is the union of mksh and ksh93's (with
one exception, see below), and that part comes from mksh.


> In the example you gave, it would be.   To me, the sole use of the ${| form
> seems to be that it doesn't delete the trailing \n if any (Chet didn't actually
> say that the other forms do delete trailing newlines,

That's part of the general description of command substitution that I
didn't include in the text describing this feature. But I can make that
point explicitly in this section.

The other benefit is not to have to output any text if you don't have to,
and for the calling shell not to have to read it.


> I'd have thought it better to simply make ${| be the same as the variant
> using the space (there's no hint in the brief spec as to what difference
> it would make using tab or newline, if any, though they were expressly
> mentioned as possible)

I would have thought it obvious the space, tab, and newline variants do
the execute-command-in-the-current-environment-and-capture-output thing,
as described first, where the exceptions are explicitly detailed and
everything else is invalid. I can make the former even more explicit.

> just with newline supression removed.   It would
> be possible to do the same with $(| make that suppress trailing newline
> removal as well).    That is, keep ${| using stdout for its result, rather
> than yet another meaingless use of REPLY.

I decided not to seek deliberate incompatibility with mksh.

> And I fail to see any need at all for the ${( form - I see no difference
> in that one from $( ) except more, and more difficult, syntax, hence
> completely pointless, unless there was something missing I failed to grasp.

Remember where I said this was the union of the mksh and ksh93 features?
This one is from ksh93. It's not really any different from $(...). It
doesn't cost anything additional to support, either.


> Personally, rather than the "nameless function" semantic, I'd prefer the
> exact opposite - a word expansion form which runs a named function (just like
> any other function, without forking) and substitutes its stdout. 

You can have that, if you want. x=${ func; } (or x=${ func 1 2 3; }) does
the right thing. The ${...;} inherits the positional parameters, so things
like `shift' work as expected in the body, but you can certainly call a
shell function there with the expected semantics.

What I mean is that

set -- 1 2
x=${ shift; echo "$@"; }
echo $x "$@"

outputs `2 2'. But you absolutely can have "a word expansion form which
runs a named function (just like any other function, without forking) and
substitutes its stdout" with little effort.


> But to go back to the first few words of the previous para - a command
> substitution is never going to be a "normal" function call, and I cannot
> see any benefit in allowing that form (and only that one - if it were simply
> a possible side effect of a more general mechanism, that would be different)
> to alter the external positional parameter list.

It's to allow local variables and `return'. The existing implementations
all agree on that.

> Trying to explain to someone what
>
>         "$*${ set -- a b c;}$*"
>
> produces, and why that's a good idea, would be kind of difficult I think.

Expansions are performed left-to-right (or, if you prefer, beginning to
end) in the word, and the command substitution modifies the current
execution environment. You expand parameters to the values they have at the
time of the expansion. It's how the existing implementations behave, and
not particularly difficult to understand.


> if the "${ " thing weren't being explained (and presumably implemented) as
> a function, 

I said it superficially resembles function execution, to the extent that
local variables and `return' work.

> then the "set" example above would just be another example.  But
> as it is to be more or less a function, complete with local vars, it should
> (whether implemented as a nameless function, or as a call of a named one)
> be a real function call, with no variant semantics (and so, a local set of
> positional parameters).

There's no reason to be deliberately incompatible. The only thing I just
couldn't stomach implementing was making `exit' in the body of this form
of command substitution act like `return' and not exit the shell, which
ksh93 does.

> Chet: since field splitting and pathname expansion come after word expansions
> you're also going to need to explain what happens with things like
>
>         "${ IFS= ; echo foo; }${ unset IFS; echo bar;}${ IFS=' '; echo a b c;}"

Unless you declare IFS local, it affects the calling execution environment.
The value of IFS when it comes time to do word splitting wins. My advice is
not to do that. But for what it's worth, all the implementations expand
that to

foobara b c

because word splitting takes place after the other word expansions.

> and
>         "${ set -f; echo '*';}${ set +f; echo '???'; }"
>
> and lots more like it.

Well, that won't do anything since it's in double quotes (so it will
output `*???'). But if it were not, mksh and bash output either `*???' or
any three-character or longer filenames in the current directory. ksh93
outputs `*???' for some reason, but I'm not really interested in figuring
out why.

Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU    chet@case.edu    http://tiswww.cwru.edu/~chet/