Re: Discrepancy between mmap call on DPDK/libvduse and rust vm-memory crate

[Jason Wang]

On Mon, Apr 15, 2024 at 3:28 PM Yongji Xie <xieyongji@bytedance.com> wrote:
>
> On Sun, Apr 14, 2024 at 5:02 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Fri, Apr 12, 2024 at 12:15:40PM +0200, Eugenio Perez Martin wrote:
> > > Hi!
> > >
> > > I'm building a bridge to expose vhost-user devices through VDUSE. The
> > > code is still immature but I'm able to forward packets using
> > > dpdk-l2fwd through VDUSE to VM. I'm now developing exposing virtiofsd,
> > > but I've hit an error I'd like to discuss.
> > >
> > > VDUSE devices can get all the memory regions the driver is using by
> > > VDUSE_IOTLB_GET_FD ioctl. It returns a file descriptor with a memory
> > > region associated that can be mapped with mmap, and an information
> > > entry about the map it contains:
> > > * Start and end addresses from the driver POV
> > > * Offset within the mmaped region of these start and end
> > > * Device permissions over that region.
> > >
> > > [start=0xc3000][last=0xe7fff][offset=0xc3000][perm=1]
> > >
> > > Now when I try to map it, it is impossible for the userspace device to
> > > call mmap with any offset different than 0.
> >
> > How exactly did you allocate memory? hugetlbfs?
> >
> > > So the "straightforward"
> > > mmap with size = entry.last-entry.start and offset = entry.offset does
> > > not work. I don't know if this is a limitation of Linux or VDUSE.
> > >
> > > Checking QEMU's
> > > subprojects/libvduse/libvduse.c:vduse_iova_add_region() I see it
> > > handles the offset by adding it up to the size, instead of using it
> > > directly as a parameter in the mmap:
> > >
> > > void *mmap_addr = mmap(0, size + offset, prot, MAP_SHARED, fd, 0);
> >
> >
> > CC Xie Yongji who wrote this code, too.
> >
>
> The mmap() with hugetlb would fail if the offset into the file is not
> aligned to the huge page size. So libvhost-user did something like
> this. But I think VDUSE doesn't have this problem.

I think what you meant is that VDUSE IOTLB doesn't have this problem.

Btw, I think we need to understand the setup. E.g is this used for
containers (bounce pages) or VM (hugetlb or other).

Thanks

> So it's fine to
> directly use the offset as a parameter in the mmap(2) here.
>
> Thanks,
> Yongji
>