Re: redirecting a file descriptor to an array variable? Possible? How? RFE?

[Linda Walsh]

Greg Wooledge wrote:
> On Sun, Nov 15, 2015 at 05:26:26AM -0800, Linda Walsh wrote:
> This can be used to serialize an array into a file
> and to read it back:
>
> printf '%s\0' "${array[@]}" > file
----
        So just like you can have [<>|]&[-0-9][0-0] as redirection ops that work
with files or FD's, why not have something like:

out=()
printf '%s\0' "${array[@]}" >&{out[@]} (**)

**: or ">@out" or ">&@out"



using 4.3,   But how is this:

  > mapfile -td '' array < file

That's fine for reading files, but I'm wanting this to be doable
in the absence of a file system, so again, instead of '>' used in printf,
above, use '<' -- but have the object it reads from represent
an array -- so you could have your file in an 'array' with the
nul's trimmed off (or nl's).

As for not wanting to use files... they are an unnecessary
dependency in most cases and file opens are expensive on some
OS's -- more so than linux (ex. cygwin).

If I have GigaBytes of memory, why should I have to use a 
file system to store transient, tmp information (unless it exceed 
all of memory), BUT -- that has often happened when "/tmp" hasn't been
large enough -- with cheaper memory, note the situation on my system
as it's been for a couple of years now...

> df /tmp  /dev/mem
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdc2       7.8G  3.3G  4.6G  42% /tmp
devtmpfs         48G   44K   48G   1% /dev

----
So if you are writing to a tmp file, which will die of
ENOMEM first?

If I thought adding 4.6G to '48G' (statistically), enough
of the the time, I'd certainly arrange for a spill.

But at run time, wouldn't it be smart to check resources --
like disk space in /tmp or free memory?  If free memory is
> 10x the disk space, maybe using the memory should be given
priority in automatic mode, thought for corner cases, 
'tmp' should allow "auto_largest", "mem", "/tmp" or "~/tmp", or 
auto_besteffort (do what one can -- but intially, start w/auto_largest,
then add a spell-to-disk "/tmp", if there is <90% space on it...



> >         But contrary to the manpage under printf:
> > " The -v option causes the  output  to  be
> >  assigned  to  the  variable var rather than being printed to the
> >  standard output. "
> >
> > printf does not print the output of the above into a var instead
> > of stdout.  
> >
> > Seems like that would be a bug?
>
> The command you used above does not have '-v var' in it.
---
1) printf '1\000\n'  | hexdump -C
00000000  31 00 0a                                          |1..|

2) printf '1%c\n' '\x00' | hexdump -C  
00000000  31 5c 0a                                          |1\.|

with/"-v"
1) printf -v v '1\000\n' 
> declare -p v
declare -- v="1"      <<-- note that $v does not have '1\000\n' in it.

Ex2:

1)  printf '"%c"\n' '' | hexdump -C      
00000000  22 00 22 0a 
1b) printf '"%c"\n' $'\x00' | hexdump -C  
00000000  22 00 22 0a  
# Showing, indeed, '' == $'\x00', in this case, however, if we
try printing the result using "-v v" and look at it:

1st, assignment:
> unset v; printf -v v '"%c"\n'  $'\x00' ;declare -p v
declare -- v="\""        <<-- again note that "$v" doesn't contain $'"\x00"'.



> > If I am in a locale using UTF-16, there will be lots of 'NUL'
> > chars if I view that output as binary -- so there has to be one
> > or more ways to encode NUL bytes in a var such that they don't
> > terminate the string.  
>
> I do not have access to a UTF-16 environment.  Does bash even WORK
> in such an environment?  At all?
----
        Actually, on this one, I'll have to find a set of 'locale'
files to describe the charset.  Several files that work with the
"LC_" files end up working due to the way they are specified ..
for example, the 1st go-around for the Chinese to encode their stuff
and everything else ended up considerably larger than UTF-8:

/usr/share/i18n/charmaps> ll --sort size -r | tail -2
-rw-rw-r-- 1 1951778 Aug 11 11:55 UTF-8
-rw-rw-r-- 1 4181789 Nov 17 15:12 GB18030
----

GB18030 _does_ provide a 1:1 mapping of Unicode codepoints there and
back again (same as UTF-8 and UTF-16).  
But since UTF-16 & UTF-8 have the same # of codepoints, they'd
likely be similar in size.


> I don't see the relation between these two issues.  "Being able to
> store NUL bytes in a string" has nothing to do with "being able to
> capture stdout and stderr separately in shell variables" as far as
> I can see.
---
        As I pointed out above, you cannot safely create, save or restore
output from printf that will be redirected into a variable, which
contradicts the manpage specification saying:

    The -v option causes the  output  to  be
    assigned  to  the  variable var rather than being printed to the
    standard output.
> Chet would have to introduce a new feature for the latter to be possible.
----
	I'm sure it's not trivial to handle all the different 
surprise cases.


>