Re: Ramping up Continuous Fuzzing of Virtual Devices in QEMU

[Philippe Mathieu-Daudé]

On 10/22/20 6:39 PM, Daniel P. Berrangé wrote:
> On Thu, Oct 22, 2020 at 12:24:16PM -0400, Alexander Bulekov wrote:
> > +CC Prasad
> >
> > On 201022 1219, Alexander Bulekov wrote:
> > > Hello,
> > > QEMU was accepted into Google's oss-fuzz continuous-fuzzing platform [1]
> > > earlier this year. The fuzzers currently running on oss-fuzz are based on my
> > > 2019 Google Summer of Code Project, which leveraged libfuzzer, qtest and libqos
> > > to provide a framework for writing virtual-device fuzzers. At the moment, there
> > > are a handful of fuzzers upstream and running on oss-fuzz(located in
> > > tests/qtest/fuzz/). They fuzz only a few devices and serve mostly as
> > > examples.
> > >
> > > If everything goes well, soon a generic fuzzer [2] will land upstream, which
> > > allows us to fuzz many configurations of QEMU, without any device-specific
> > > code. To date this fuzzer has led to ~50 bug reports on launchpad. Once the
> > > generic-fuzzer lands upstream, OSS-Fuzz will automatically start fuzzing a
> > > bunch [3] of fuzzer configurations, and it is likely to find bugs.  Others will
> > > also be able to send simple patches to add additional device configurations for
> > > fuzzing.
> > >
> > > The oss-fuzz process looks roughly like this:
> > >      1. oss-fuzz fuzzes QEMU
> > >      2. When oss-fuzz finds a bug, it reports it to a few [4] people that have
> > >      access to reports and reproducers.
> > >      3. If a fix is merged upstream, oss-fuzz will figure this out and mark the
> > >      bug as fixed and make the report public 30 days later.
> > >      3. After 90 days the bug(fixed or not) becomes public, so anyone can view
> > >      it here https://bugs.chromium.org/p/oss-fuzz/issues/list
> > >
> > > The oss-fuzz reports look like this:
> > > https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=23701&q=qemu&can=2
> > >
> > > This means that when oss-fuzz find new bugs, the relevant developers do not
> > > know about them unless someone with access files a separate report to the
> > > list/launchpad. So far this hasn't been a problem, since oss-fuzz has only been
> > > running some small example fuzzers. Once [2] lands upstream, we should
> > > see a significant uptick in oss-fuzz reports, and I hope that we can develop a
> > > process to ensure these bugs are properly dealt with. One option we have is to
> > > make the reports public immediately and send notifications to
> > > qemu-devel. This is the approach taken by some other projects on
> > > oss-fuzz, such as LLVM. Though its not on oss-fuzz, bugs found by
> > > syzkaller in the kernel, are also automatically sent to a public list.
> > > The question is:
> > >
> > > What approach should we take for dealing with bugs found on oss-fuzz?
>
> If we assume that a non-negligible number of fuzz bugs will be exploitable
> by a malicious guest OS to break out into the host, then I think it is
> likely undesirable to make them public immediately without at least a basic
> human triage step to catch possibly serious security issues.
>
> Still a large % are likely to be low impact / not urgent to deal with so
> we want a low overhead way to handle the fuzz output, which doesn't create
> a bottleneck on a small number of people.
>
> Overall my feeling is that we want to be able to farm out triage to the
> respective subsystem maintainers, who can then decide whether the bug
> needs notifying to the security team, or can be made public immediately.
>
> I think ideally we would be doing triage in QEMU's own bug tracker, so
> we don't need to have maintainers create accounts on a 3rd party tracker
> to see reports.
>
> Is is practical to identify a primary affected source file from the fuzz
> crash report with any level reliablility such that we could file a private
> launchpad bug, automatically CC'ing a subsystem maintainer (and the security
> team)  ?

Also what is not very clear is, who is supposed/going to fix these bugs?

I see this pattern:

a) bug found by human: human keeps asking for the bug
  1/ security issue: someone assigned to fix
  2/ else: if human keeps asking, the bug gets eventually fixed.

b) bug found by fuzzer:
  1/ security issue: someone assigned to fix
  2/ else: nothing happens because unlikely hit by user

Do we want to keep tracking b.2 bug reports? I think this is the case of
the ~50 Alexander mentioned.

Regards,

Phil.