Re: qemu-ppc64-linux-user: signal handler endianness in elfv2

[Vincent Fazio]

David,

Thanks for the response.

On 2/10/20 1:08 AM, David Gibson wrote:
> On Sat, Feb 08, 2020 at 12:57:40PM -0600, Vincent Fazio wrote:
> > I ran into a crash scenario in ppc64-linux-user when trying to run openssl
> > compiled with gcc & musl which emits an ELFv2 binary. The target CPU was a
> > big endian model (e6500)
> Uh.. so I'm pretty sure ABIv2 is little-endian specific.  If you want
> BE binaries, I think you need to configure your toolchain differently.
This is _generally_ the case. MUSL libc uses ELFv2 ABI for both big and 
little endian, so it's apparently assumed that gcc is set with 
--with-abi=elfv2. This is how it's currently configured with Buildroot.

https://gitlab.com/vfazio/buildroot/commit/5ab751ca44e9da6f406876ff582b3a813056b0d4

OpenSSL recently had a similar assumption patched here: 
https://github.com/openssl/openssl/commit/098404128383ded87ba390dd74ecd9e2ffa6f530

I imagine the most frequent test case is running ppc64le on an x86_64 
host where the swap never even occurs due to the endianness match.

But if the fix is to just not do the swap for ELFv2 binaries because the 
function pointer is always aligned with the host's memory endianness due 
to how functions addresses are calculated in ELFv2, i don't mind 
submitting a patch.

If QEMU decides this is an unsupported case, that's probably fine 
because it's likely not in use in large scale.

> > Example:
> >
> > vfazio@vfazio1 ~/development/buildroot/output/target :( $ gdb
> > ~/development/qemu/ppc64-linux-user/qemu-ppc64
> > (gdb) run -d guest_errors -cpu e6500 -E 
> > LD_LIBRARY_PATH="/home/vfazio/development/buildroot/output/target/lib/:/home/vfazio/development/buildroot/output/target/usr/lib/"
> > lib/libc.so bin/bash
> > Starting program: /home/vfazio/development/qemu/ppc64-linux-user/qemu-ppc64
> > -d guest_errors -cpu e6500 -E 
> > LD_LIBRARY_PATH="/home/vfazio/development/buildroot/output/target/lib/:/home/vfazio/development/buildroot/output/target/usr/lib/"
> > lib/libc.so bin/bash
> > [Thread debugging using libthread_db enabled]
> > Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
> > [New Thread 0x7ffff7ff9700 (LWP 27186)]
> > invalid/unsupported opcode: 1f - 0c - 1a - 00 (7c005698) 0000000010028e58 0
> >
> > Thread 1 "qemu-ppc64" received signal SIGSEGV, Segmentation fault.
> > 0x00000000600a9257 in ldl_he_p (ptr=0x4c2c061000000000) at
> > /home/vfazio/development/qemu/include/qemu/bswap.h:351
> > 351        __builtin_memcpy(&r, ptr, sizeof(r));
> >
> > (gdb) !readelf -h bin/bash | grep abi
> >    Flags:                             0x2, abiv2
> > (gdb) !readelf -h lib/libc.so | grep abi
> >    Flags:                             0x2, abiv2
> > <<<<<
> >
> > openssl does "probing" for PPC features (Altivec, Crypto, etc) by trying to
> > execute possibly illegal instructions and handling them... hence that
> > SIGILL. The problem is when it comes to jumping to the signal handler
> >
> > #0  0x00000000600a9257 in ldl_he_p (ptr=0x4c2c061000000000) at
> > /home/vfazio/development/qemu/include/qemu/bswap.h:351
> > #1  0x00000000600a92fe in ldl_be_p (ptr=0x4c2c061000000000) at
> > /home/vfazio/development/qemu/include/qemu/bswap.h:449
> > #2  0x00000000600c0790 in translator_ldl_swap (env=0x62923150,
> > pc=5488768711647035392, do_swap=false) at
> > /home/vfazio/development/qemu/include/exec/translator.h:201
> > #3  0x000000006011c1ab in ppc_tr_translate_insn (dcbase=0x7fffffffd250,
> > cs=0x6291ae80) at /home/vfazio/development/qemu/target/ppc/translate.c:7856
> > #4  0x000000006005ae70 in translator_loop (ops=0x60805fc0 <ppc_tr_ops>,
> > db=0x7fffffffd250, cpu=0x6291ae80, tb=0x60a5f900
> > <static_code_gen_buffer+1681600>, max_insns=512)
> >      at /home/vfazio/development/qemu/accel/tcg/translator.c:102
> > <<<<<
> >
> > The handler has what looks like an LE address but then swaps it since the
> > host is LE and the target is BE
> >
> > setup_rt_frame (sig=4, ka=0x628c5f80 <sigact_table+96>, info=0x62931658,
> > set=0x7fffffffd2f8, env=0x62923150) at
> > /home/vfazio/development/qemu/linux-user/ppc/signal.c:575
> > 575            qemu_log_mask(LOG_GUEST_ERROR, "sa_handler NIP to "
> > TARGET_FMT_lx "\n", ka->_sa_handler);
> > (gdb) p/x ka->_sa_handler
> > $11 = 0x10062c4c
> > 579            env->nip = tswapl((target_ulong) ka->_sa_handler);
> > (gdb) p/x env->nip
> > $12 = 0x4c2c061000000000
> > <<<<<
> >
> > the memcpy later faults because that's obviously not a valid address from
> > which to grab the PPC instructions that need to be translated.
> >
> > I'm not familiar with the ELFv2 ABI or PPC assembly in general, so I'm not
> > sure what to expect here. Typically ELFv2 was reserved for ppc64le, but musl
> > uses ELFv2 for all ppc64 targets. This likely wouldn't be an issue in
> > ppc64LE since x86_64 is LE as well and no swap would take place.
> >
> > Is the signal handler address tied to the endianess of the host? I noticed
> > there was no swapping in elfload.c so wasn't sure...removing the swap gets
> > me a little further in program execution but i do eventually crash (but that
> > may not be related to this specific issue).
> >
> > Thanks
--
Vincent Fazio
Embedded Software Engineer - Linux
Extreme Engineering Solutions, Inc
http://www.xes-inc.com