Re: [PATCH] Long USB transfers problem

[Aleš Nesrsta]

Vladimir 'φ-coder/phcoder' Serbinenko píše v Po 10. 12. 2012 v 21:33
+0100:
> On 10.12.2012 18:49, Aleš Nesrsta wrote:
...
> So it would be that maximum transfer size would be part of structure.
> Another 2 questions are:
> 1) Is this multi-schedule transfer reliable?
I think yes - and it possibly depends on place where You will do
splitting:

a)
First possible place is in scsi.c inside grub_scsi_readXX before the
values are calculated for rd. In these functions You can calculate
smaller transfers and repeat whole SCSI command more than one time
(together with request sense).

>From the USB point of view it will looks e.g. in this way:
Function grub_scsi_readXX receives request to read 4096 bytes and
maximal safe limit of QDs/TDs is 512 bytes for one bulk transfer for
related USB device.
So, function grub_scsi_readXX creates 8 smaller SCSI read transfers
which will on USB side result in
8-times repeating whole sequence of these transactions: 
- bulk OUT CBW (SCSI command: read 1 sector)
- bulk IN DATA (512 bytes)
- bulk IN CSW
- bulk OUT CBW (SCSI command: request sense)
- bulk IN DATA (sense data)
- bulk IN CSW

This kind of splitting should be absolutely correct according to any
specification - but there are very high overheads.


b)
Second possible place is in usbms.c inside grub_usbms_read (or also on
lower level - usbtrans.c, as I did in first experimental try). If
function grub_usbms_read receives read request of 4K data, its normal
flow without splitting will be:
- bulk OUT transaction for CBW (header & CDB contain length code of 4K)
- bulk IN transaction for 4K DATA transfer
- bulk IN transaction for CSW
With splitting it will be in this way:
- bulk OUT transaction for CBW (header & CDB contain length code of 4K)
- 8-times bulk IN transaction for 512 bytes DATA transfer
- bulk IN transaction for CSW

I.e., if we will do full SCSI command with final request sense with the
same value of size as in example in a), we will have only ONE execution
of following transaction sequence:
- bulk OUT CBW (SCSI command: read 4K)
- 8-times bulk IN DATA (512 bytes)
- bulk IN CSW
- bulk OUT CBW (SCSI command: request sense)
- bulk IN DATA (sense data)
- bulk IN CSW

I expect that also this kind of splitting is correct according to any
specification - in case of BULK transactions. It should be because bulk
transaction is nothing else than chain of IN/DATAx/ACK packets which I
can do on USB host controller sequentially in any chunks - there are not
any other "control" types of packets in bulk transactions which will
really tell "hello, device, there is end of bulk transfer planned in USB
host controller" :-) . This feature is related only to BULK transactions
(Maybe it will be work also on interrupt transfer - but I think probably
not. Fortunately, interrupt and control transfers should be never so
"long" to exhaust all TDs...).
There is important only one thing - all packets in chain should be of
wMaxPacketSize length (with exception of absolutely last packet) - even
if USBMS specification allows also smaller packets than wMaxPacketSize
(some devices (more precisely most of devices) have not proper
implementation of USBMS specification and have problems if intermediate
bulk transaction is not of wMaxPacketSize length).


If You compare a) and b), You will see big difference in number of
transactions which should be done - in a) case it should be 8x6=48, in
case b) there are only 13 transactions!
That is why I prefer to do splitting in usbms.c - or in lower level if
such "long" transfers can be generated by some another USB device than
USBMS.


>  What if we fail to add QD's
> in time?
I don't understand what do You mean by this question. Could You explain
it little bit more?


> 2) Do such transfers work with other device, in particular the ones we
> may be interested in in the future?
There I am also not sure if I good understood this question. Probably
You are afraid what happens if You will add some another kind of USB
device into GRUB (e.g. USB network class device etc.) - ?

If the new kind of USB device will generate also some "long" BULK
transfer which cannot be fulfilled with limited number of USB host
controller QDs/TDs, such bulk transfer should be also split.

As I tried to explain above, I am almost sure that splitting of BULK
transfers can be done generally and also on lowest possible level, i.e.
also in usbtrans.c like in my current experimantal patch - and it could
be done also for OUT bulk transactions, if necessary.

In usbtrans.c we know at least the wMaxPacketSize and possibly we can
more or less easily know here also the maximum of QDs/TDs of the USB
driver, so we can split bulk transfer here to be sure it allocates
maximally e.g. half of QDs/TDs etc.
If we will do bulk splitting in usbtrans.c, we will be (almost) sure we
are prepared for any kind of USB device... :-)
(OK, not for any kind - some USB device needs e.g. isochronous transfer
which we have not implemented in GRUB...)

-----

BTW, whole this discussion is related to one thing - all GRUB USB
drivers (UHCI, OHCI, EHCI) have limited number of QDs/TDs because of
static preallocation of these structures.

My question is - why is it done in this way? Why we have static
allocation of QDs/TDs?
(It was programmed in this way in OHCI/UHCI before I did any change. And
I did EHCI driver later in the same way only to keep the same philosophy
as OHCI/UHCI drivers, to be "compatible" as most as possible.)

Maybe is it according to necessary alignment of QDs/TDs which is/was not
possible in GRUB on dynamically allocated memory?
Or there are some high overheads if we will often allocate lot of small
memory chunks (specially of DMA "capable" memory) ?
Or?

If we will allocate/deallocate QDs/TDs dynamically, we can allocate any
possible number of them - and we will not need to limit size of bulk
transfers... But, of course, we should rewrite a lot in this case... :-(

BR,
Ales
> 
> > As this problem is together USB implementation specific and USB mass
> > storage specific, I suggest to do splitting in usbms.c.
> > Functions grub_usbms_read and grub_usbms_write have access to related
> > grub_scsi structure, i.e. splitting in these functions can use SCSI
> > device block size of related USB mass storage device without any
> > additional modifications.
> > 
> > BR,
> > Ales
> > 
> > 
> > _______________________________________________
> > Grub-devel mailing list
> > address@hidden
> > https://lists.gnu.org/mailman/listinfo/grub-devel
> 
> 
> 
> _______________________________________________
> Grub-devel mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/grub-devel