Re: pipes that write to a file

[Nikolay Nechaev]

Hello! In bash there already is a way to do what you want to do (although, not
exactly in a way you're trying to do it). Not sure, how elegant this is
(actually, it doesn't look pretty), but here's how I'd do it:


#!/bin/bash

function prog1()
{
        sed 's/a/b/g'
}

function prog2()
{
        tee
}

function prog3()
{
        # The command, for which we want to inspect the exit code.
        # Uncomment `false` below to make it fail

        tee

        #false
}

echo aaa > f

prog1 < f | prog2 | (prog3 && echo 1>&2 ok || echo 1>&2 failed) | sponge f


When running this, you will see "ok" in the terminal, if the `#false` line is
commented, and "failed" instead, if you uncomment it.

There, however, seems to be no such feature in the standard sh shell

-- 
Best wishes,
Nikolay Nechaev

On Sunday, 27 June 2021 03:13:59 MSK Bruno Haible wrote:
> Hi,
> 
> 'envsubst' is a small program, part of GNU gettext, that is meant to be used
> within pipes (it reads from stdin and writes to stdout).
> 
> Someone proposes to add an --in-place option so that it would read from a
> file and write its output to the same file. [1]
> 
> However, this would not cover all use-cases. People often use 'envsubst'
> and other commands, together, in a pipe, and want the output to overwrite
> the input file.
> 
> So, people come to recommend the 'sponge' program from moreutils [2]:
> 
>    prog1 < FILE | prog2 | prog3 | sponge FILE
> 
> But this program has the drawback that, requiring another step in the
> pipeline, the exit code of the previous command gets lost. While
> 
>    prog1 < FILE | prog2 | prog3 > $tmpfile
>    mv $tmpfile FILE
> 
> allows logic that inspects the exit code prog3,
> 
>    prog1 < FILE | prog2 | prog3 | sponge FILE
> 
> does not.
> 
> What is the right solution?
> 
> - The code that uses a temporary file [3] is quite cumbersome to write,
>   and is inefficient if the data size is large (since the temporary file
>   will live on a different partition).
> - I understand that an 'inplace' program has been rejected [4].
> 
> How about adding a  -o FILE / --output FILE  to the program prog3
> ('envsubst' in my case, but 'cat' and other programs would also be
> candidates for it), that creates a new temporary file in the same
> directory as FILE and moves it to FILE when done? This would provide
> for an atomic replacement. The Gnulib module 'supersede' would
> implement most of this logic; it is already used by GNU msgfmt's '-o'
> option.
> 
> In other words, move the 'sponge' logic into the last step of the pipe.
> 
> Or should the shells be adapted so that people can write
> 
>    prog1 < FILE | prog2 | prog3 >+ FILE
> 
> and that operator '>+' writes to FILE but only after the pipe has completed?
> 
> Bruno
> 
> [1] https://savannah.gnu.org/bugs/?60807
> [2] 
> https://serverfault.com/questions/135507/linux-how-to-use-a-file-as-input-and-output-at-the-same-time
> [3] https://lists.gnu.org/archive/html/bug-coreutils/2009-03/msg00321.html
> [4] https://lists.gnu.org/archive/html/bug-coreutils/2012-11/msg00184.html
> 
> 
>