Re: Ramping up Continuous Fuzzing of Virtual Devices in QEMU

[Alexander Bulekov]

On 201024 1110, Li Qiang wrote:
> Alexander Bulekov <alxndr@bu.edu> 于2020年10月23日周五 上午12:20写道：
> >
> > Hello,
> > QEMU was accepted into Google's oss-fuzz continuous-fuzzing platform [1]
> > earlier this year. The fuzzers currently running on oss-fuzz are based on my
> > 2019 Google Summer of Code Project, which leveraged libfuzzer, qtest and 
> > libqos
> > to provide a framework for writing virtual-device fuzzers. At the moment, 
> > there
> > are a handful of fuzzers upstream and running on oss-fuzz(located in
> > tests/qtest/fuzz/). They fuzz only a few devices and serve mostly as
> > examples.
> >
> > If everything goes well, soon a generic fuzzer [2] will land upstream, which
> > allows us to fuzz many configurations of QEMU, without any device-specific
> > code. To date this fuzzer has led to ~50 bug reports on launchpad. Once the
> > generic-fuzzer lands upstream, OSS-Fuzz will automatically start fuzzing a
> > bunch [3] of fuzzer configurations, and it is likely to find bugs.  Others 
> > will
> > also be able to send simple patches to add additional device configurations 
> > for
> > fuzzing.
> >
> > The oss-fuzz process looks roughly like this:
> >     1. oss-fuzz fuzzes QEMU
> >     2. When oss-fuzz finds a bug, it reports it to a few [4] people that 
> > have
> >     access to reports and reproducers.
> >     3. If a fix is merged upstream, oss-fuzz will figure this out and mark 
> > the
> >     bug as fixed and make the report public 30 days later.
> >     3. After 90 days the bug(fixed or not) becomes public, so anyone can 
> > view
> >     it here https://bugs.chromium.org/p/oss-fuzz/issues/list
> >
> > The oss-fuzz reports look like this:
> > https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=23701&q=qemu&can=2
> >
> > This means that when oss-fuzz find new bugs, the relevant developers do not
> > know about them unless someone with access files a separate report to the
> > list/launchpad. So far this hasn't been a problem, since oss-fuzz has only 
> > been
> > running some small example fuzzers. Once [2] lands upstream, we should
> > see a significant uptick in oss-fuzz reports, and I hope that we can 
> > develop a
> > process to ensure these bugs are properly dealt with. One option we have is 
> > to
> > make the reports public immediately and send notifications to
> > qemu-devel. This is the approach taken by some other projects on
> > oss-fuzz, such as LLVM. Though its not on oss-fuzz, bugs found by
> > syzkaller in the kernel, are also automatically sent to a public list.
> > The question is:
> >
> > What approach should we take for dealing with bugs found on oss-fuzz?
> >
> 
> Hi Alex,
> 
> I prefer to send these bugs to public list such as qemu-devel.
> 
> There are lots of low impact bugs so no need to prepare a private
> bugtracker for the little important issues.
> Also the maintainer's decision may take a long time.
> 
> For the public issues, the security engineer, maintainer and volunteer
> can both see them and point out its
> impact more quickly.
> 
> 
> 
> > [1] https://github.com/google/oss-fuzz
> > [2] https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg06331.html
> > [3] https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg06345.html
> > [4] 
> > https://github.com/google/oss-fuzz/blob/fbf916ce14952ba192e58fe8550096b868fcf62d/projects/qemu/project.yaml#L4
> 
> BTW, is there any condition to join this lists?
> I'm quite interested to fix the qemu issues.

I guess the original message is trying to figure out what role that list
of people will play. If we go with your option and just add qemu-devel
to the list, then we wouldn't need to worry about adding anyone else to
the list. 
Otherwise, we will need to figure that out - I don't think anyone
currently on that list signed up to triage a bunch of fuzzing bugs :)

As a side note, the general fuzzers were just merged.
https://git.qemu.org/?p=qemu.git;a=commit;h=e75de8354ac5c67145b2f8874d3c36022d4a94bb
oss-fuzz should start using them for fuzzing sometime tommorrow. 
-Alex

> Thanks,
> Li Qiang
> 
> >
> > For further reference, the vast majority of these bugs, were found with the
> > generic-fuzzer:
> > https://bugs.launchpad.net/~a1xndr/+bugs
> >
> > There are more that I haven't yet had time to write reports for.
> > Thank you
> > -Alex