Re: macOS (Big Sur, Apple Silicon) 'make check' fails in test-crypto-tlscredsx509

[Christian Schoenebeck]

On Dienstag, 2. Februar 2021 06:19:42 CET Roman Bolshakov wrote:
> 'make check' of libtasn1 doesn't succeed on x86_64 either.
> 
> After a session of debugging I believe there's an issue with Clang 12.
> Here's a test program (it reproduces unexpected ASN1_VALUE_NOT_VALID
> from _asn1_time_der() in libtasn1):
> 
> #include <stdio.h>
> 
> static int func2(char *foo) {
>         fprintf(stderr, "%s:%d foo: %p\n", __func__, __LINE__, foo);
>         if (foo == NULL) {
>                 fprintf(stderr, "%s:%d foo: %p\n", __func__, __LINE__, foo);
> return 1;
>         }
>         return 0;
> }
> 
> int func1(char *foo) {
>         int counter = 0;
>         if (fprintf(stderr, "IO\n") > 0)
>                 counter += 10;
>         fprintf(stderr, "%s:%d foo: %p counter %d\n", __func__, __LINE__,
> foo, counter); if(!func2(foo + counter)) {
>                 fprintf(stderr, "good\n");
>                 return 0;
>         } else {
>                 fprintf(stderr, "broken\n");
>                 return 1;
>         }
> }
> 
> int main() {
>         char *foo = NULL;
>         return func1(foo);
> }
> 
> 
> What return value would you expect from the program?
> 
> If the program is compiled with -O0/O1 it returns zero exit code.
> Here's the output:
> IO
> func1:16 foo: 0x0 counter 10
> func2:4 foo: 0xa
> good
> 
> If it is compiled with -O2 it returns 1:
> IO
> func1:16 foo: 0x0 counter 10
> func2:4 foo: 0xa
> func2:6 foo: 0x0
> broken
> 
> That happens because clang uses register behind foo from func1 (it has zero
> pointer) inside inlined func2 (it should have non zero pointer).
> 
> So, immediate workaround would be to downgrade optimization level of
> libtasn1 to -O1 in homebrew.

Hu, confirmed.

clang 12.0.0 on x86_64 Mac fails on that demo with -O2,-O3,-Os, but works with
-O0,-O1.

clang 11.0.3 in contrast works with any optimization level.

It only fails BTW if that test uses exactly a NULL pointer, any other memory 
address (e.g. just (void*)1) works:

#include <stdio.h>

#define FLOOR_VALUE ((void*)1)

static int func2(char *foo) {
        fprintf(stderr, "%s:%d foo: %p\n", __func__, __LINE__, foo);
        if (foo == FLOOR_VALUE) {
                fprintf(stderr, "%s:%d foo: %p\n", __func__, __LINE__, foo);
                return 1;
        }
        return 0;
}

int func1(char *foo) {
        int counter = 0;
        if (fprintf(stderr, "IO\n") > 0)
                counter += 1;
        fprintf(stderr, "%s:%d foo: %p counter %d\n", __func__, __LINE__, foo, 
counter);
        if(!func2(foo + counter)) {
                fprintf(stderr, "good\n");
                return 0;
        } else {
                fprintf(stderr, "broken\n");
                return 1;
        }
}

int main() {
        char *foo = FLOOR_VALUE;
        return func1(foo);
}

Maybe that's some sort of new security feature in clang 12, in the sense of 
something like this:

        VeryLargeStruct *p = NULL;
        p->farMember = value;

to segfault always reliably and exactly with address zero, instead of pure 
luck as of NULL + veryLargeSize.

> I've submitted the issue to Apple bugtracker:
> FB8986815
> 
> Best regards,
> Roman

They could argue that operating on a NULL pointer is undefined behaviour.

Best regards,
Christian Schoenebeck