Re: Mac OS real USB device support issue

[Programmingkid]

> On Apr 7, 2021, at 1:28 AM, Howard Spoelstra <hsp.cat7@gmail.com> wrote:
> 
> On Wed, Apr 7, 2021 at 7:26 AM Howard Spoelstra <hsp.cat7@gmail.com> wrote:
>> 
>> On Wed, Apr 7, 2021 at 3:53 AM Programmingkid <programmingkidx@gmail.com> 
>> wrote:
>>> 
>>> 
>>> 
>>>> On Apr 6, 2021, at 7:18 PM, BALATON Zoltan <balaton@eik.bme.hu> wrote:
>>>> 
>>>> On Tue, 6 Apr 2021, Programmingkid wrote:
>>>>>> On Apr 6, 2021, at 12:53 PM, BALATON Zoltan <balaton@eik.bme.hu> wrote:
>>>>>> On Tue, 6 Apr 2021, Programmingkid wrote:
>>>>>>>> On Apr 6, 2021, at 10:01 AM, Howard Spoelstra <hsp.cat7@gmail.com> 
>>>>>>>> wrote:
>>>>>>>> On Tue, Apr 6, 2021 at 3:44 PM Programmingkid 
>>>>>>>> <programmingkidx@gmail.com> wrote:
>>>>>>>>> 
>>>>>>>>> Hi Gerd,
>>>>>>>>> 
>>>>>>>>> I was wondering if you had access to a Mac OS 10 or Mac OS 11 machine 
>>>>>>>>> to test USB support. I am on Mac OS 11.1 and cannot make USB devices 
>>>>>>>>> work with any of my guests. So far these are the guests I have tested 
>>>>>>>>> with:
>>>>>>>>> 
>>>>>>>>> - Windows 7
>>>>>>>>> - Mac OS 9.2
>>>>>>>>> - Windows 2000
>>>>>>>>> 
>>>>>>>>> I have tried using USB flash drives, USB sound cards, and an USB 
>>>>>>>>> headset. They all show up under 'info usb', but cannot be used in the 
>>>>>>>>> guest. My setup does use a USB-C hub so I'm not sure if this is a bug 
>>>>>>>>> with QEMU or an issue with the hub. Would you have any information on 
>>>>>>>>> this issue?
>>>>>>>> 
>>>>>>>> Hi John,
>>>>>>>> 
>>>>>>>> As far as the Mac OS 9.2 guest is concerned on a mac OS host, it does
>>>>>>>> not support USB 2.0. I was successful only in passing through a USB
>>>>>>>> flash drive that was forced into USB 1.1 mode by connecting it to a
>>>>>>>> real USB 1.1 hub and unloading the kext it used.
>>>>>>>> 
>>>>>>>> Best,
>>>>>>>> Howard
>>>>>>> 
>>>>>>> Hi Howard, I was actually thinking about CC'ing you for this email. 
>>>>>>> Glad you found it. Unloading kext files does not sound pleasant. Maybe 
>>>>>>> there is some better way of doing it.
>>>>>> 
>>>>>> In any case, until you make sure nothing tries to drive the device on 
>>>>>> the host, passing it to a guest likely will fail because then two 
>>>>>> drivers from two OSes would try to access it simultaneously which likely 
>>>>>> creates a mess as the device and drivers don't expect this. So you can't 
>>>>>> just pass a device through that the host has recognised and is driving 
>>>>>> without somehow getting the host to leave it alone first before you can 
>>>>>> pass it through. Unloading the driver is one way to do that (although it 
>>>>>> probably breaks all other similar devices too). Maybe there's another 
>>>>>> way to unbind a device from the host such as ejecting it first but then 
>>>>>> I'm not sure if the low level USB needed for accessing the device still 
>>>>>> works after that or it's completely forgotten. There's probably a doc 
>>>>>> somewhere that describes how it works and how can you plug a device 
>>>>>> without also getting higher level drivers to load or if there's no 
>>>>>> official ways for that then you'll need to do some configuration on the 
>>>>>> host t
>>>> o avoid it grabbing devices that you want to pass through. On Linux you 
>>>> can add an udev rule to ignore the device (maybe also adding 
>>>> TAG+="uaccess" to allow console users to use it without needing root 
>>>> access) but not sure how USB works on macOS.
>>>>>> 
>>>>>> Regards,
>>>>>> BALATON Zoltan
>>>>> 
>>>>> Being able to dissociate a real USB device from its Mac OS driver would 
>>>>> be very useful in this situation. IOKit might be one place to look for 
>>>>> such a feature. The Mach kernel documentation is another place that might 
>>>>> have what we want.
>>>> 
>>>> Those might be a good place to start. IOKit provides the drivers and also 
>>>> the io registry which is probably where you can get if a driver is bound 
>>>> to a device and which one is it. How to dissociate the driver from the 
>>>> device though I don't know.
>>> 
>>> https://developer.apple.com/library/archive/documentation/DeviceDrivers/Conceptual/IOKitFundamentals/DeviceRemoval/DeviceRemoval.html
>>> According to this article a driver has a stop() and detach() method that is 
>>> called by the IOKit to remove a device. I'm thinking QEMU can be the one 
>>> that calls these methods for a certain device.
>>> 
>>>> 
>>>>> I have one theory. What if we introduce a middleman. A pseudo-USB device 
>>>>> that the guest operating system could apply its configuration data to and 
>>>>> will also talk directly with to the real USB device.
>>>>> So this:
>>>>> 
>>>>> USB device <-> Host <-> QEMU USB middleman <-> Guest
>>>> 
>>>> Isn't this middleman the QEMU usb-host device that we already have?
>>> 
>>> It could be. I need to research this issue some more.
>>> 
>>>> 
>>>>> This could make USB 2.0 and 3.0 flash drives compatible with an older 
>>>>> operating system like Mac OS 9. The USB middleman could fully accept Mac 
>>>>> OS 9's configuration and make it think it is talking to a USB 1.1 device. 
>>>>> Parameters like data packet payload size would no longer be a problem. 
>>>>> Host driver unloading would no longer be needed (in theory).
>>>> 
>>>> However I think you're mixing up a few things here. The idea of passing 
>>>> through USB devices is to let the guest handle it with its own drivers 
>>>> like it was connected directly to the virtual machine and not to emulate a 
>>>> USB device using host resources. If you want the latter then use 
>>>> usb-storage, usb-audio or similar. All that usb-host does is just 
>>>> forwarding the packets from guest to the physical device and let it talk 
>>>> to it and drive it without help from the host. (I may be wrong about the 
>>>> details, I haven't checked actual code but at least conceptually this 
>>>> should be the case.) If you have this scenario then you can easily see 
>>>> that both the host and guest driving the same USB device will not work. 
>>>> You should not try to mount a USB drive in guest that's already mounted by 
>>>> the host or you can't send audio from both the guest and the host at the 
>>>> same time without totally confusing the device and both drivers that don't 
>>>> expect this to ever happen. So for passing through the device you have to 
>>>> make sure the host does not try to access it while it's used by the guest.
>>> 
>>> This clarifies things on my part.
>>> 
>>>> 
>>>> If your guest does not have drivers for the device that you want to pass 
>>>> through that's a different problem. With pass through the guest is 
>>>> exclusively given the task of driving the device so it should have a 
>>>> driver for it. If the device does not work with the guest if you plug it 
>>>> in a physical machine then it won't work with pass through either. But the 
>>>> problem here is probably not that but the disagreement between USB speed 
>>>> between host and guest. If the guest does not have USB 2.0 then you can't 
>>>> pass through USB 2.0 devices unless downgrading them on the host as well 
>>>> in some way. So you either connect them to an USB 1 hub to match the 
>>>> emulated USB hardware in the guest or you need to emulate an USB 2 card in 
>>>> the guest and connect passed through devices to that. Did USB 2 cards 
>>>> exist for older G3/G4 Macs?
>>> 
>>> Yes. I upgraded a friend's PowerMac G4 with such a card.
>>> 
>>>> Some PowerBooks had USB 2 ports, what hardware did those use?
>>> 
>>> I'm assuming it was the G4 PowerBooks.
>>> 
>>>> Could those be emulated in QEMU?
>>> 
>>> The PowerMac already is.
>>> 
>>>> These are separate problems though from getting the device freed from host 
>>>> drivers to avoid the problems with both guest and host accessing the 
>>>> device.
>>> 
>>> Thank you again for the help.
>>> 
>>> I think a simple algorithm would be
>>> 1) find out if a host driver is already using a real USB device.
>>> 2) If it is call that driver's stop() and detach() methods for only that 
>>> USB device (other devices should not be effected).
>>> 3) Let the guest start using the USB device.
>>> 
>>> 
>> 
>> This is what Gerd wrote about the USB 1.1 -> USB 2.0 issue:
>> 
>> "Problem is when the device is plugged into a usb2 port you can't query
>> the usb1 descriptors.  So qemu presents the wrong descriptors to the
>> guest in case host and guest use different usb speeds.  That may or may
>> not work ...
>> 
>> The other way around is less problematic, when plugging a usb2 device
>> into a usb3-capable (xhci) port I can tell the guest "this is a usb2
>> device".  But reporting "this is a usb2 device" via ohci isn't going to
>> fly for obvious reasons ..."
>> 
>> So then I forced my device it into USB 1.1 mode by attaching it to a
>> real 1.1 hub. The qemu hub will not do this for you. Mac OS and Mac OS
>> X guests up to 10.2.8 do not have USB 2.0 support, but from 10.2.8
>> upwards you can use the ehci hub. This works for a simple usb storage
>> device that uses the usbstorage kext in macOS. Indeed, as Zoltan
>> wrote, unloading a kext for a device that has its class driver loaded,
>> one looses access to other devices that match too. Reboot time ;-)
>> 
>> There used to be a solution by creating an empty kext that loaded with
>> higher priority compared to the standard kexts. I don't know whether
>> that solution could still work with Apple putting new requirements on
>> kext loading. I believe unloading a kext requires disabling SIP. Sudo
>> won't do anymore from Catalina upwards.
>> 
>> I gather Qemu is currenty relying on libusb to do the heavy lifting.
>> For Linux that seems OK together with perhaps first unbinding a
>> driver, but for macOS and Windows hosts the unbinding does not work.
>> 
>> Here is some discussion on how to create an empty kext:
>> https://github.com/libusb/libusb/issues/158#issuecomment-190582178
>> 
>> Best,
>> Howard
> 
> And this:
> https://github.com/libusb/libusb/issues/906

Have you tried the proposed changes yet for libusb?