[solved] Re: Debian 9 64b / QEMU 2.8.1 / FreeDOS 1.3rc2 as a guest: DHCP requests end up zero-padded

[Frantisek Rysanek]

Dear everybody,

I'm itching to top-post, but as this is a no-no, scroll all the way 
down if you're interested to see my outcome ;-)

On 27 Dec 2019 at 18:33, address@hidden, free wrote:
> Dear everybody,
> 
> this is a superficial early question,
> just in case this is a known problem,
> or to get your feedback ("upgrade the distro
> and stick to the distro kernel", etc).
> 
> I'm playing with Debian 9 64bit (so far I haven't been motivated to 
> upgrade) which happens to contain the following QEMU:
> 
> QEMU emulator version 2.8.1(Debian 1:2.8+dfsg-6+deb9u8)
> Copyright (c) 2003-2016 Fabrice Bellard and the QEMU Project 
> developers
> 
> Actually I'm using kernel 5.2.7 on the box (boxes) = "somewhat" 
> younger, compared to the original 4.9.0 from the Debian repo... ;-)
> 
> As a QEMU guest, I'm trying to boot a DOS-based "Client for microsoft 
> networks", actually the venerable Net Boot Disk 6.5 "distro" based on
> FreeDOS and some custom candy.
> https://www.netbootdisk.com/
> I've unpacked the NBD 6.5 onto an 8MB HDD image and deactivated
> the original on-the-fly unpacking into a RAMdisk = so it effectively 
> boots off a harddisk partition. I've been using it this way for ages,
> PXE-booted using PXElinux into a MEMDISK holding the whole HDD 
> image... I've just been updating the NIC drivers.
> Now I'm trying to move this into a QEMU guest, where the disk image
> is provided to the guest as an emulated IDE disk drive.
> And it basically works - the FreeDOS boots and I can 
> do things inside the OS and the FS on the emulated disk.
> 
> While trying to work around some other problem (some software, 
> including pciscan.exe by Bart Lagerweij, and the low-level NIC 
> drivers, were exciting an error in the older FreeDOS 1.0-ish kernel)
> I upgraded the guest side to the latest FreeDOS 1.3-rc2.
> Unfortunately the BAT scripts involved apparently rely on FreeDOS 
> syntax, so I cannot easily port the whole thing onto MSDOS 6.22 just 
> to give it a try...
> So I'm at FreeDOS 1.3-rc2, PCISCAN works and the network drivers 
> appear to load - they report a MAC address etc. I've tried about two 
> older Intel NIC chips and a Realtek RTL8139 (all of them as emulated 
> by QEMU).
> 
> Now finally for the problem:
> 
> The network "client" software consists of several resident layers
> that get loaded step by step. And the progress gets stuck
> when the TCP/IP stack tries to ask for DHCP = when the 
> machine just booted tries to actually use the NIC for the first time 
> to generate traffic on the network. See a screenshot attached.
> I believe the misbehaving piece of software is called TCPTSR.EXE .
> 
> I'm using QEMU's "-net nic -net bridge" to hook the guest into the 
> host's LAN. I actually have a soft-bridge instance prepared in the 
> host OS, before I start QEMU, and Windows PE 7/10 guests work just 
> fine on top of that arrangement: QEMU instantiates a tap0 and adds 
> that to the host's soft-bridge, and the Windows guest starts like a 
> cheetah.
> 
> So in that context, in the Debian-based host OS, I've tried sniffing 
> the traffic coming out of tap0 with tcpdump. And, I'm seeing 
> something interesting.
> In perfect correlation with the DOS guest's DHCP client starting up, 
> I can see a "packet of all zeroes" every now and then. As if, for 
> some reason, the DHCP Discovery packets got zero-padded.
> A short snippet is attached (packet header, as reported by tcpdump).
> 
> I've tried with or without KVM acceleration in QEMU.
> I've tried several variants of the emulated NIC hardware.
> I've tried JEMMEX (5.78=latest) with NOVDS and I've tried HimemX.EXE.
> I've tried booting with Debian's stock SeaBIOS or with a more modern 
> OVMF UEFI+CSM (got pre-compiled binaries somewhere upstream).
> None of this makes a difference, the resulting symptoms are exactly 
> the same.
> 
> The next step would be to upgrade the distro and compile a fresh QEMU 
> from source. Which would take a bit of effort...
> So in that context, I'd be grateful for any opinions if this is worth 
> the bother, if someone has seen this before etc.
> 
> Thanks for your attention :-)
> 
> Frank Rysanek
> 
> P.S.: the "Debian host" itself is actually PXE-booted in UEFI mode, 
> with / mounted RW on top of overlayfs on top of a read-only NFS 
> volume, and the soft-bridge gets inserted early during initrd boot 
> (before ipconfig asks for DHCP). It's nifty, it all works like a 
> charm, and it wasn't all that much work. I love Debian. A howto will 
> follow soon.
> 
> 
> Attachments:
>   N:\Install - only LEGAL !!!\FreeDos\FreeDOS_QEMU_DHCP_hangs.jpg
>   N:\Install - only LEGAL !!!\FreeDos\zero_padded_pkt_dump.txt

...so after more than a quarter, I got back to my "DOS in QEMU in a 
diskless-booted Linux" side project. And, I have some success to 
report.

I've bitten the bullet and massaged QEMU 4.1.1 into my diskless 
booting environment.
With the QEMU Xwindows GUI front-ends being what they are, with all 
their dependencies, I felt it too difficult to either make my own 
.DEB on the build machine (correctly detailing all the deps), or to 
copy all the binaries, libraries and config files by hand.
I ended up forking a copy of the NFS bootdir for my diskless machines 
and in the experimental fork of the rootfs, I installed GCC and all 
the -devel versions of the dependencies. After that, I had a diskless 
machine in RW mode compile QEMU for me. It went surprisingly smoothly 
- and, my slightly updated NetBootDisk (from NetBootDisk.COM) now 
booted via PXE just fine! The vanilla NetBootDisk MSTCP drivers and 
wrapper scripts around them, it all started right off the bat. The 
weird zero-padded DHCP packets are gone.

Side note, away from the original topic: Ghost surprised me by 
duplicating some keystrokes - namely the "inverted T" arrow keys and 
the numpad enter. No other keys behaved that way, and the culprit 
keys worked just fine e.g. in Volkov. The effect looked the same in 
GTK and SDL front-end. I've managed to rectify this effect by 
specifying "-device usb-kbd", which supposedly suppresses the 
emulated PS/2 keyboard controller. So with a USB-attached virtual 
keyboard, the arrow key duplications are now gone, too.

I'm a happy camper :-)

Thanks to everyone who has tried to help me, back in late December.

Frank