Re: [PATCH 1/4] fuzz: add datadir for oss-fuzz compatability

[Alexander Bulekov]

On 200520 1851, Philippe Mathieu-Daudé wrote:
> On 5/12/20 5:01 AM, Alexander Bulekov wrote:

-snip-
> > +        /*
> > +         * With oss-fuzz, the executable is kept in the root of a 
> > directory (we
> > +         * cannot assume the path). All data (including bios binaries) 
> > must be
> > +         * in the same dir, or a subdir. Thus, we cannot place the pc-bios 
> > so
> > +         * that it would be in exec_dir/../pc-bios.
> > +         * As a workaround, oss-fuzz allows us to use argv[0] to get the
> > +         * location of the executable. Using this we add exec_dir/pc-bios 
> > to
> > +         * the datadirs.
> 
> I don't understand well the problem. How do you run it?

On oss-fuzz, the build and execution happens in two separate containers.

1.) In the build container, we can do whatever we want, but we must
place the executable(s) we produce at the root of a directory /out/.
e.g. one output executable is /out/qemu-system-target-i440fx-fuzz

2.) In the runner, this "build artifact" directory is mounted at
some location(we can't assume the location). This runner container
automatically identifies the executable within the root of the  "build
artifact" dir and runs it. The path to the executable could now be 
/somedir/qemu-system-target-i440fx-fuzz

In the runner container we only have control over the files in /somedir/
(which was /out/ in the builder). Thus, in addition to copying over
shared-libs to /out/ we need to copy any data (pc-bios) that the binary
relies on. The problem is that we have to point qemu towards the
location of the bios. Normally qemu checks the /usr/share/... dir. For
local builds, qemu also examines the executable path and looks in
$executable_path/../pc-bios/. On the oss-fuzz runner we dont control
/somedir/../pc-bios, so we can't rely on this. This patch allows us to
specify /somedir/pc-bios as the datadir.

Hopefully that is not too confusing.

For reference, here is a draft of the build-script that I expect to
deploy to oss-fuzz:

# build project
# e.g.
# ./autogen.sh
# ./configure
# make -j$(nproc) all

# build fuzzers
# e.g.
# $CXX $CXXFLAGS -std=c++11 -Iinclude \
#     /path/to/name_of_fuzzer.cc -o $OUT/name_of_fuzzer \
#     $LIB_FUZZING_ENGINE /path/to/library.a

# make a dir for the shared libs
mkdir -p $OUT/lib/

#Build once to copy over the list of dynamic libs
./configure --datadir="./data/" --disable-werror --cc="$CC" --cxx="$CXX" 
--extra-cflags="$CFLAGS -U __OPTIMIZE__ "
make CONFIG_FUZZ=y CFLAGS="$LIB_FUZZING_ENGINE" -j$(nproc) i386-softmmu/fuzz

# copy over the shared libs
for i in $(ldd ./i386-softmmu/qemu-fuzz-i386  | cut -f3 -d' '); do
    cp $i $OUT/lib/
done
rm ./i386-softmmu/qemu-fuzz-i386

# Build second time to build the final binary with correct rpath
./configure --datadir="./data/" --disable-werror --cc="$CC" --cxx="$CXX" 
--extra-cflags="$CFLAGS -U __OPTIMIZE__" 
--extra-ldflags="-Wl,-rpath,'\$\$ORIGIN/lib'"
make CONFIG_FUZZ=y CFLAGS="$LIB_FUZZING_ENGINE" -j$(nproc) i386-softmmu/fuzz

# copy over bios data
cp  -r ./pc-bios/ $OUT/pc-bios
for target in $(./i386-softmmu/qemu-fuzz-i386 | awk '$1 ~ /\*/  {print $2}');
do
    cp ./i386-softmmu/qemu-fuzz-i386 $OUT/qemu-fuzz-i386-target-$target
done

-Alex
> 
> > +         */
> > +        dir = g_build_filename(g_path_get_dirname(**argv), "pc-bios", 
> > NULL);
> > +        if (g_file_test(dir, G_FILE_TEST_IS_DIR)) {
> > +            qemu_add_data_dir(dir);
> > +        }
> > +        g_free(dir);
> >       } else if (*argc > 1) {  /* The target is specified as an argument */
> >           target_name = (*argv)[1];
> >           if (!strstr(target_name, "--fuzz-target=")) {
> > 
>