Re: [PATCH 13/19] accel/tcg: Fixed tsan warnings.

[Robert Foley]

On Sat, 23 May 2020 at 16:06, Emilio G. Cota <address@hidden> wrote:
>
> On Fri, May 22, 2020 at 12:07:49 -0400, Robert Foley wrote:
> > For example:
> > WARNING: ThreadSanitizer: data race (pid=35425)
> >   Write of size 4 at 0x7bbc000000ac by main thread (mutexes: write M875):
> >     #0 cpu_reset_interrupt hw/core/cpu.c:107:28 
> > (qemu-system-aarch64+0x843790)
> >     #1 arm_cpu_set_irq target/arm/cpu.c (qemu-system-aarch64+0x616265)
> >     #2 qemu_set_irq hw/core/irq.c:44:5 (qemu-system-aarch64+0x8462ca)
> >   Previous atomic read of size 4 at 0x7bbc000000ac by thread T6:
> >     #0 __tsan_atomic32_load <null> (qemu-system-aarch64+0x394c1c)
> >     #1 cpu_handle_interrupt accel/tcg/cpu-exec.c:534:9 
> > (qemu-system-aarch64+0x4b7e79)
> >     #2 cpu_exec accel/tcg/cpu-exec.c:720:17 (qemu-system-aarch64+0x4b7e79)
> > or
> > WARNING: ThreadSanitizer: data race (pid=25425)
> >   Read of size 8 at 0x7f8ad8e138d0 by thread T10:
> >     #0 tb_lookup_cmp accel/tcg/cpu-exec.c:307:13 
> > (qemu-system-aarch64+0x4ac4d2)
> >     #1 qht_do_lookup util/qht.c:502:34 (qemu-system-aarch64+0xd05264)
> >   Previous write of size 8 at 0x7f8ad8e138d0 by thread T15 (mutexes: write 
> > M728311726235541804):
> >     #0 tb_link_page accel/tcg/translate-all.c:1625:26 
> > (qemu-system-aarch64+0x4b0bf2)
> >     #1 tb_gen_code accel/tcg/translate-all.c:1865:19 
> > (qemu-system-aarch64+0x4b0bf2)
> >     #2 tb_find accel/tcg/cpu-exec.c:407:14 (qemu-system-aarch64+0x4ad77c)
>
> I see you're working through the warnings in this file, but I think it would
> be better to forget about files and focus on the data itself.
> Therefore this patch should be split in two: (1) cpu-<interrupt_request
> and (2) gen_code_buf. (1) requires a lot of changes with a proper audit;
> the per-cpu-lock series has a possible solution for that, so I will
> ignore those hunks and just comment on (2) below.

We will be dropping the changes in this file which overlap with
the per-cpu-locks patch.
>
> > Cc: Richard Henderson <address@hidden>
> > Cc: Paolo Bonzini <address@hidden>
> > Signed-off-by: Robert Foley <address@hidden>
> > ---
> >  accel/tcg/tcg-all.c       | 4 ++--
> >  accel/tcg/tcg-runtime.c   | 7 ++++++-
> >  accel/tcg/translate-all.c | 6 +++++-
> >  hw/core/cpu.c             | 2 +-
> >  4 files changed, 14 insertions(+), 5 deletions(-)
> >
> (snip)
> > diff --git a/accel/tcg/tcg-runtime.c b/accel/tcg/tcg-runtime.c
> > index 446465a09a..bd0cd77450 100644
> > --- a/accel/tcg/tcg-runtime.c
> > +++ b/accel/tcg/tcg-runtime.c
> > @@ -31,6 +31,7 @@
> >  #include "disas/disas.h"
> >  #include "exec/log.h"
> >  #include "tcg/tcg.h"
> > +#include "qemu/tsan.h"
> >
> >  /* 32-bit helpers */
> >
> > @@ -151,6 +152,7 @@ void *HELPER(lookup_tb_ptr)(CPUArchState *env)
> >      TranslationBlock *tb;
> >      target_ulong cs_base, pc;
> >      uint32_t flags;
> > +    void *tc_ptr;
> >
> >      tb = tb_lookup__cpu_state(cpu, &pc, &cs_base, &flags, curr_cflags());
> >      if (tb == NULL) {
> > @@ -161,7 +163,10 @@ void *HELPER(lookup_tb_ptr)(CPUArchState *env)
> >                             TARGET_FMT_lx "/" TARGET_FMT_lx "/%#x] %s\n",
> >                             cpu->cpu_index, tb->tc.ptr, cs_base, pc, flags,
> >                             lookup_symbol(pc));
> > -    return tb->tc.ptr;
> > +    TSAN_ANNOTATE_IGNORE_READS_BEGIN();
> > +    tc_ptr = tb->tc.ptr;
> > +    TSAN_ANNOTATE_IGNORE_READS_END();
> > +    return tc_ptr;
>
> I'm not sure these are needed. At least after applying all other patches
> in this series, I don't get a warning here.

I think we will also be dropping the ANNOTATE calls here as we re-focus
the patch series.

Thanks & Regards,
-Rob

>
> >  }
> >
> >  void HELPER(exit_atomic)(CPUArchState *env)
> > diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
> > index 3fb71a1503..6c0e61994c 100644
> > --- a/accel/tcg/translate-all.c
> > +++ b/accel/tcg/translate-all.c
> > @@ -58,6 +58,7 @@
> >  #include "exec/log.h"
> >  #include "sysemu/cpus.h"
> >  #include "sysemu/tcg.h"
> > +#include "qemu/tsan.h"
> >
> >  /* #define DEBUG_TB_INVALIDATE */
> >  /* #define DEBUG_TB_FLUSH */
> > @@ -1704,6 +1705,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
> >          max_insns = 1;
> >      }
> >
> > +    TSAN_ANNOTATE_IGNORE_WRITES_BEGIN();
>
> Same here, I don't get a warning in this hunk if I remove these,
> except for:
> ---
> WARNING: ThreadSanitizer: data race (pid=445867)
>   Atomic read of size 1 at 0x7f906e050158 by thread T7:
>     #0 __tsan_mutex_post_lock <null> (qemu-system-aarch64+0x481721)
>     #1 qemu_spin_lock /home/cota/src/qemu/include/qemu/thread.h:244:5 
> (qemu-system-aarch64+0x5578e9)
>     #2 tb_add_jump /home/cota/src/qemu/accel/tcg/cpu-exec.c:363:5 
> (qemu-system-aarch64+0x5578e9)
>     #3 tb_find /home/cota/src/qemu/accel/tcg/cpu-exec.c:423:9 
> (qemu-system-aarch64+0x5578e9)
>
>   Previous write of size 1 at 0x7f906e050158 by thread T8:
>     #0 __tsan_mutex_create <null> (qemu-system-aarch64+0x481589)
>     #1 qemu_spin_init /home/cota/src/qemu/include/qemu/thread.h:221:5 
> (qemu-system-aarch64+0x559a71)
>     #2 tb_gen_code /home/cota/src/qemu/accel/tcg/translate-all.c:1875:5 
> (qemu-system-aarch64+0x559a71)
>
>   Thread T7 'CPU 0/TCG' (tid=445875, running) created by main thread at:
>     #0 pthread_create <null> (qemu-system-aarch64+0x43915b)
>     #1 qemu_thread_create /home/cota/src/qemu/util/qemu-thread-posix.c:558:11 
> (qemu-system-aarch64+0xaf91ff)
>
>   Thread T8 'CPU 1/TCG' (tid=445876, running) created by main thread at:
>     #0 pthread_create <null> (qemu-system-aarch64+0x43915b)
>     #1 qemu_thread_create /home/cota/src/qemu/util/qemu-thread-posix.c:558:11 
> (qemu-system-aarch64+0xaf91ff)
>
> SUMMARY: ThreadSanitizer: data race 
> (/home/cota/src/qemu/build/aarch64-softmmu/qemu-system-aarch64+0x481721) in 
> __tsan_mutex_post_lock
> ---
>
> Seems like tsan is confusing itself here.
>
> Thanks,
>                 E.