[PATCHv2] docs: mention another issue with variable expansion

[Eric Blake]

In particular, see http://austingroupbugs.net/view.php?id=221
and http://austingroupbugs.net/view.php?id=255.

* doc/autoconf.texi (Shell Substitutions) <${var+value}>: New
subsection.
<${var=literal}>: Tweak wording.  Add mention of an ambiguity
allowed by POSIX.
* tests/torture.at (Substitute and define special characters):
Make test more robust; here, the outer "" is in a here-doc, and
does not violate the quoting rules of thumb just documented.

Signed-off-by: Eric Blake <address@hidden>
---

Diffs to v1 include some spelling fixes, and expansion of the timing
example to demonstrate that ${a=b} can reuse the existing value of $a,
thus why "" are important even when b would not cause globbing.

I'm pushing this now.

 ChangeLog         |   13 +++++
 doc/autoconf.texi |  147 +++++++++++++++++++++++++++++++++++++++++++++--------
 tests/torture.at  |    4 +-
 3 files changed, 141 insertions(+), 23 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index fa252b3..8097db5 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,16 @@
+2010-08-26  Eric Blake  <address@hidden>
+
+       docs: mention another issue with variable expansion
+       In particular, see http://austingroupbugs.net/view.php?id=221
+       and http://austingroupbugs.net/view.php?id=255.
+       * doc/autoconf.texi (Shell Substitutions) <${var+value}>: New
+       subsection.
+       <${var=literal}>: Tweak wording.  Add mention of an ambiguity
+       allowed by POSIX.
+       * tests/torture.at (Substitute and define special characters):
+       Make test more robust; here, the outer "" is in a here-doc, and
+       does not violate the quoting rules of thumb just documented.
+
 2010-08-25  Eric Blake  <address@hidden>

        m4sh: revert incorrect mix of "${a='b'}"
diff --git a/doc/autoconf.texi b/doc/autoconf.texi
index 28637c5..cf62b0c 100644
--- a/doc/autoconf.texi
+++ b/doc/autoconf.texi
@@ -15392,52 +15392,156 @@ Shell Substitutions
 @ifnotinfo
 @cindex address@hidden@var{var}:address@hidden@}
 @end ifnotinfo
address@hidden address@hidden@address@hidden@}
 Old BSD shells, including the Ultrix @code{sh}, don't accept the
 colon for any shell substitution, and complain and die.
 Similarly for address@hidden@var{var}:address@hidden@}, 
address@hidden@var{var}:address@hidden@}, etc.

address@hidden address@hidden@address@hidden@}
address@hidden address@hidden@address@hidden@}
address@hidden address@hidden@address@hidden@}
address@hidden address@hidden@address@hidden@}
+When using @address@hidden@address@hidden@}} or
address@hidden@address@hidden@address@hidden for providing alternate 
substitutions,
address@hidden must either be a single shell word or be quoted.  Solaris
address@hidden/bin/sh} complains otherwise.
+
address@hidden
+$ @kbd{/bin/sh -c 'echo address@hidden address@hidden'}
+/bin/sh: bad substitution
+$ @kbd{/bin/sh -c 'echo address@hidden'\''b c'\''@}'}
+b c
+$ @kbd{/bin/sh -c 'echo "address@hidden address@hidden"'}
+b c
address@hidden example
+
+According to Posix, if an expansion occurs inside double quotes, then
+the use of unquoted double quotes within @var{value} is unspecified, and
+any single quotes become literal characters; in that case, escaping must
+be done with backslash.
+
address@hidden
+$ @kbd{/bin/sh -c 'echo "address@hidden"b  c"@}"'}
+/bin/sh: bad substitution
+$ @kbd{ksh -c 'echo "address@hidden"b  c"@}"'}
+b c
+$ @kbd{bash -c 'echo "address@hidden"b  c"@}"'}
+b  c
+$ @kbd{/bin/sh -c 'a=; echo address@hidden'\''b  c'\''@}'}
+b  c
+$ @kbd{/bin/sh -c 'a=; echo "address@hidden'\''b  c'\''@}"'}
+'b  c'
+$ @kbd{/bin/sh -c 'a=; echo "address@hidden"b  c\"@}"'}
+"b  c"
+$ @kbd{/bin/sh -c 'a=; echo "address@hidden  address@hidden"'}
+b  c
address@hidden example
+
+Perhaps the easiest way to work around quoting issues in a manner
+portable to all shells is to place the results in a temporary variable,
+then use @samp{$tmp} as the @var{value}, rather than trying to inline
+the expression needing quoting.
+
address@hidden
+$ @kbd{/bin/sh -c 'tmp="a  b\"'\''@}\\"; echo "address@hidden@}"'}
+b  c"'@}\
+$ @kbd{ksh -c 'tmp="a  b\"'\''@}\\"; echo "address@hidden@}"'}
+b  c"'@}\
+$ @kbd{bash -c 'tmp="a  b\"'\''@}\\"; echo "address@hidden@}"'}
+b  c"'@}\
address@hidden example
+
address@hidden address@hidden@address@hidden@}
address@hidden address@hidden@address@hidden@}
 When using @address@hidden@address@hidden@}} to assign a default value
 to @var{var}, remember that even though the assignment to @var{var} does
 not undergo file name expansion, the result of the variable expansion
-does.  In particular, when using @command{:} followed by unquoted
-variable expansion for the side effect of setting a default value, if
-either @samp{value} or the prior contents of @samp{$var} contains
-globbing characters, the shell has to spend time performing file name
+does unless the expansion occurred within double quotes.  In particular,
+when using @command{:} followed by unquoted variable expansion for the
+side effect of setting a default value, if the final value of
address@hidden contains any globbing characters (either from @var{value} or
+from prior contents), the shell has to spend time performing file name
 expansion and field splitting even though those results will not be
-used.  Therefore, it is a good idea to use double quotes when performing
-default initialization.
+used.  Therefore, it is a good idea to consider double quotes when performing
+default initialization; while remembering how this impacts any quoting
+characters appearing in @var{value}.

 @example
-$ time bash -c ': "address@hidden/usr/bin/address@hidden"; echo "$a"'
+$ @kbd{time bash -c ': "address@hidden/usr/bin/address@hidden"; echo "$a"'}
 /usr/bin/*

 real   0m0.005s
 user   0m0.002s
 sys    0m0.003s
-$ time bash -c ': address@hidden/usr/bin/address@hidden; echo "$a"'
+$ @kbd{time bash -c ': address@hidden/usr/bin/address@hidden; echo "$a"'}
 /usr/bin/*

 real   0m0.039s
 user   0m0.026s
 sys    0m0.009s
+$ @kbd{time bash -c 'a=/usr/bin/*; : address@hidden@}; echo "$a"'}
+/usr/bin/*
+
+real   0m0.031s
+user   0m0.020s
+sys    0m0.010s
+
+$ @kbd{time bash -c 'a=/usr/bin/*; : "address@hidden@}"; echo "$a"'}
+/usr/bin/*
+
+real   0m0.006s
+user   0m0.002s
+sys    0m0.003s
 @end example

-Use quotes if @var{literal} contains more than one shell word:
+As with @samp{+} and @samp{-}, you must use quotes when using @samp{=}
+if the @var{value} contains more than one shell word; either single
+quotes for just the @var{value}, or double quotes around the entire
+expansion:

 @example
-: "address@hidden'Some words'@}"
+$ @kbd{: address@hidden'Some words'@}}
+$ @kbd{: "address@hidden address@hidden"}
+$ @kbd{echo $var1 $var2}
+Some words like this
 @end example

 @noindent
-otherwise some shells, such as on Digital Unix V 5.0, die because
-of a ``bad substitution''.
+otherwise some shells, such as Solaris @command{/bin/sh} or on Digital
+Unix V 5.0, die because of a ``bad substitution''.  Meanwhile, Posix
+requires that with @samp{=}, quote removal happens prior to the
+assignment, and the expansion be the final contents of @var{var} without
+quoting (and thus subject to field splitting), in contrast to the
+behavior with @samp{-} passing the quoting through to the final
+expansion.  However, @command{bash} 4.1 does not obey this rule.

address@hidden 1
address@hidden
+$ @kbd{ksh -c 'echo address@hidden \ address@hidden'}
+a  b
+$ @kbd{ksh -c 'echo address@hidden \ address@hidden'}
+a b
+$ @kbd{bash -c 'echo address@hidden \ address@hidden'}
+a  b
address@hidden example
+
+Finally, Posix states that when mixing @address@hidden@}} with regular
+commands, it is unspecified whether the assignments affect the parent
+shell environment.  It is best to perform assignments independently from
+commands, to avoid the problems demonstrated in this example:
+
address@hidden
+$ @kbd{bash -c 'x= address@hidden:address@hidden sh -c "echo +\$x+\$y+";echo 
-$x-'}
++b+b+
+-b-
+$ @kbd{/bin/sh -c 'x= address@hidden:address@hidden sh -c "echo 
+\$x+\$y+";echo -$x-'}
+++b+
+--
+$ @kbd{ksh -c 'x= address@hidden:address@hidden sh -c "echo +\$x+\$y+";echo 
-$x-'}
++b+b+
+--
address@hidden example

-Solaris @command{/bin/sh} has a frightening bug in its interpretation
-of this.  Imagine you need set a variable to a string containing
address@hidden address@hidden@address@hidden@}
address@hidden address@hidden@address@hidden@}
+Solaris @command{/bin/sh} has a frightening bug in its handling of
+literal assignments.  Imagine you need set a variable to a string containing
 @address@hidden  This @address@hidden character confuses Solaris 
@command{/bin/sh}
 when the affected variable was already set.  This bug can be exercised
 by running:
@@ -15458,7 +15562,8 @@ Shell Substitutions

 It seems that @address@hidden is interpreted as matching @address@hidden, even
 though it is enclosed in single quotes.  The problem doesn't happen
-using double quotes.
+using double quotes, or when using a temporary variable holding the
+problematic string.

 @item address@hidden@address@hidden@}
 @cindex address@hidden@address@hidden@}
@@ -15467,7 +15572,7 @@ Shell Substitutions

 @example
 default="yu,yaa"
-: "address@hidden"$default"@}"
+: address@hidden"$default"@}
 @end example

 @noindent
@@ -15493,7 +15598,7 @@ Shell Substitutions

 @example
 default="a b c"
-: "address@hidden"$default"@}"
+: address@hidden"$default"@}
 for c in $list; do
   echo $c
 done
@@ -15755,7 +15860,7 @@ Assignments
 (i.e., it's not a list), then use:

 @example
-: "address@hidden"$default"@}"
+: address@hidden"$default"@}
 @end example

 @item
diff --git a/tests/torture.at b/tests/torture.at
index a8a2aa1..6855da4 100644
--- a/tests/torture.at
+++ b/tests/torture.at
@@ -909,7 +909,7 @@ AC_DEFINE_UNQUOTED([unq1], [$baz], [unquoted, test 1])
 AC_DEFINE_UNQUOTED([unq2], [\$baz], [unquoted, test 2])
 AC_DEFINE_UNQUOTED([unq3], ["$baz"], [unquoted, test 3])
 AC_DEFINE_UNQUOTED([unq4], [${baz+set}], [unquoted, test 4])
-AC_DEFINE_UNQUOTED([unq5], ["${baz+`echo "a b"`}"], [unquoted, test 5])
+AC_DEFINE_UNQUOTED([unq5], ["${baz+`echo "a "' b'`}"], [unquoted, test 5])
 AC_DEFINE_UNQUOTED([unq6], [`echo hi`], [unquoted, test 6])
 AC_DEFINE_UNQUOTED([unq7], ['\\"'], [unquoted, test 7])
 AC_PROG_AWK
@@ -943,7 +943,7 @@ address@hidden@
 #define unq2 $baz
 #define unq3 "bla"
 #define unq4 set
-#define unq5 "a b"
+#define unq5 "a  b"
 #define unq6 hi
 #define unq7 '\"'
 ]])
-- 
1.7.2.2