Re: [PATCH] tests: add extra protection against unexpected exits

[Jim Meyering]

On Tue, Jan 13, 2015 at 5:51 PM, Pádraig Brady <address@hidden> wrote:
> On 13/01/15 08:13, Bernhard Voelker wrote:
>> On 01/13/2015 04:37 AM, Pádraig Brady wrote:
>>> Many tests use `program ... && fail=1` to ensure expected
>>> error situations are indicated.  However that would mask
>>> an unexpected exit (like a crash).
>>
>> Nice catch, and also e.g. exceeded ulimits would go in that
>> category.
>>
>>> [...]  Therefore protect such
>>> calls with `{ program ... | test $? -ne 1; } && fail=1`.
>>
>> ... || test
>>
>> Well, somehow I think this syntax is
>> a) hard to read because the exit code is evaluated in 2 places
>>    (explicitly in "test $? -ne 1" and implicitly with "&&"),
>> and
>> b) hard to remember, i.e. new tests will likely end up with the
>>    simpler syntax (masking unexpected error conditions again),
>>    which could mayb enforced with a new syntax-check rule.
>>
>> The construct
>>
>>   { program ... || test $? -ne 1; } && fail=1
>>
>> is identical to
>>
>>   program ...
>>   test $? -eq 1 || fail=1
>>
>> and thus explicitly expecting exit code 1.
>> Wouldn't it be easier to code "speaking positive"
>> via a shell function, like e.g. (untested)
>>
>>   expectExit() {
>>     local exp="$1"
>>     shift 1 || framework_failure_
>>     "$@"
>>     test $? -eq $exp || return 1
>>   }
>>
>>   expectExit 1 program ... || fail=1
>
> Very good suggestions.  I implemented the simplification wrapper
> (I called it returns_), and that in turn made a syntax check feasible.
> Updated patch is attached.

I like it a lot more via the wrapper.
Thank you both!