none
Issue with USB drivers Usbser, Usbccid regarding ZLP RRS feed

  • Question

  • Hello,

    I have found some topics discussing about this point, but often/always in the DEVICE->HOST way. I am interested in talking about the opposite one (HOST->DEVICE).

    Currenly I have a composite device communicating in USB with through CDC and CCID interfaces. I wanted to use windows built-in drivers for communication. But but when the HOST transfers data with a transfer (in the sense of the USB specification) which size is a multiple of MaxPacketSizeno zero-length packet issued ! (note: same problem with Usbser, Usbccid)

    Consequently the USB controller embedded in the device doesn't know that the transfer is actually completed and is still excepting some more data. It's even painful as the controller only trigger interrupt to signal the software when the transfer is achieved.

    This makes me think that Microsoft considers that it's not up to the driver to deal with it. It seems to me that (regarding the USB 2.0 standard, see 5.8.3) it should be from drivers responsibility to generate this ZLP if needed.

    Because obviously without this ZLP, as only the HOST is aware of the amount of data "expected" we can't expect the DEVICE to know whether the transfer is completed or not as there is no such field in some packet in USB protocole to indicate the size of a USB transfer.

    What do you think about this ? Is it normal that the driver seems to rely on upper layer to deal with such concern ?

    Friday, April 26, 2019 5:04 PM

Answers

  • It is the driver's problem.  The web page you mentioned is really just allowing an application to be the driver, or more accurately to configure a driver.  Once that bit is set, the WinUSB driver will always send a ZLP when the end of a transfer falls at a packet boundary.

    usbser.sys ought to be doing that as well, but I strongly suspect that most USB-to-serial chips simply don't care about transfers.  Serial is just a bucket of bytes -- there's no structure, so there is no reason not to process each packet as soon as it arrives, regardless of its size.  There's no point to storing up transfers.  Frankly, if I were you, I would investigate whether your USB chip has a config bit that says "interrupt on packet" instead of "interrupt on transfer".  Most of them do.


    Tim Roberts | Driver MVP Emeritus | Providenza & Boekelheide, Inc.

    Tuesday, April 30, 2019 7:02 AM
  • Not sure about CCID, but in case of CDC (such as USB to serial) the bulk data is not specially structured.

    Serial data is just a sequence of bytes. When the device receives a bulk data packet, it just consumes it, or puts it on the wire.  Message-oriented protocols can use ZLPs to separate messages, or use length field in the data stream.

    Why are you concerned? Are you writing the device firmware?

    -- pa

    Saturday, April 27, 2019 11:02 PM

All replies

  • Not sure about CCID, but in case of CDC (such as USB to serial) the bulk data is not specially structured.

    Serial data is just a sequence of bytes. When the device receives a bulk data packet, it just consumes it, or puts it on the wire.  Message-oriented protocols can use ZLPs to separate messages, or use length field in the data stream.

    Why are you concerned? Are you writing the device firmware?

    -- pa

    Saturday, April 27, 2019 11:02 PM
  • Hello Pavel,

    Thank you for your response.

    Yes I/we develop the firmware. The problem here is that the USB controller embedded in the microcontroller of our device only (roughly) issues interrupts on transfer completion, that is not at transaction or packet level (note: there are some special case for SOP, ....). And is unable to detect the end of the transfer as the host doesn't issue ZLP.

    Device controller behaviour seems legit for me (regarding the USB 2.0 standard 5.8.3 paragraph)

    When a bulk IRP involves more data than can fit in one maximum-sized
    data payload, all data payloads are required to be maximum size except for the last data payload, which will
    contain the remaining data. A bulk transfer is complete when the endpoint does one of the following:
    • Has transferred exactly the amount of data expected
    • Transfers a packet with a payload size less than wMaxPacketSize or transfers a zero-length packet

    I must admit that I find the first bullet talking about the amount of data "expected" about a bit irrelevent in case of BULK(-OUT) transfers. It can make sense for CONTROL transfers because there is a "length" field in the SETUP packet that device can use to determine whether transfer is finished or not.

    Anyway I get your point regarding data structure for BULK and I understand that the nature of the data that host is issuing is a don't care from driver perspective but so as for the USB standard. Completion conditions of a BULK transfer is defined in the USB standard not in a class specific standard. It isn't conditionned by the kind of data that are conveyed.

    I found this link: https://docs.microsoft.com/en-us/uwp/api/windows.devices.usb.usbwriteoptions

    which seems to confirm that Microsoft considers that it's up to application to deal with ZLP (in this case). Maybe I am missing something but I don't think that it's normal. Besides I am surprized that nobody has faced this issue before (for BULK-OUT)

    Monday, April 29, 2019 10:50 AM
  • It is the driver's problem.  The web page you mentioned is really just allowing an application to be the driver, or more accurately to configure a driver.  Once that bit is set, the WinUSB driver will always send a ZLP when the end of a transfer falls at a packet boundary.

    usbser.sys ought to be doing that as well, but I strongly suspect that most USB-to-serial chips simply don't care about transfers.  Serial is just a bucket of bytes -- there's no structure, so there is no reason not to process each packet as soon as it arrives, regardless of its size.  There's no point to storing up transfers.  Frankly, if I were you, I would investigate whether your USB chip has a config bit that says "interrupt on packet" instead of "interrupt on transfer".  Most of them do.


    Tim Roberts | Driver MVP Emeritus | Providenza & Boekelheide, Inc.

    Tuesday, April 30, 2019 7:02 AM
  • Hi Tim,

    Thank you for responding.

    Unfortunately I haven't seen in my device datasheet any configuartion bit in the USB controller register set that whould trigger interrupt at packet level (there is such interrrupt for special case like SOP but for not bulk). I have contact manufacturer so that he'll confirm but it makes sense to me that there isn't such bit as, regarding the USB 2.0 standard 5.8.3 paragraph, ZLP should be sent if transfer size is multiple of MaxPacketSize(saving significant amount of interrupts).

    However there are still points that I don't get. I found this (my apologies I am not allowed to add hyperlink yet) :

    https://www.google.fr/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&ved=2ahUKEwjc6Pnl-_7hAhUKkxQKHVKmBVYQjRx6BAgBEAU&url=%2Furl%3Fsa%3Di%26rct%3Dj%26q%3D%26esrc%3Ds%26source%3Dimages%26cd%3D%26ved%3D%26url%3Dhttps%253A%252F%252Fwww.slideserve.com%252Fjessamine-bentley%252Fembedding-usb%26psig%3DAOvVaw3C8geusH-sm6GoiET6uLLv%26ust%3D1556959246756358&psig=AOvVaw3C8geusH-sm6GoiET6uLLv&ust=1556959246756358

    If I understand correctly :

    1. application builds an IRP including the buffer holding data to be transfered
    2. client/class driver (usbser in this case) is dealing with the IRP and builds an URB
    3. usbd deals with the URB and builds transactions (/transfers ?)
    4. hcd creates packets, SOF, ....

    So this would mean that actually usbser is indeed not responsible for the transfer management (therefore cannot on its own decide to add the ZLP). Unless there is a parameter that usbser should have been passed to usbd, indicating that the latter need to add a ZLP. I had a look in _URB_BULK_OR_INTERRUPT_TRANSFER structure and I don't see such one (for bulk-out at least).

    If such parameter does indeed not exist, and given the link I provided in my previous post, for me it looks like if ShortPacketTerminate flag is set, somewhere between application and usbd an additional IRP with the ZLP is created and sent to usbd along with the IRP holding data.

    Besides in the title of this thread I mentioned Usbccid because we have exactly the same problem (ZLP missing) than with Usbser. It tends to prove that broadly speaking class drivers don't dela with such transfer completion concern.

    Having said all this, doesn't it look like it is more a problem with usbd (or hcd) ? Which is not compliant with USB standard (or rather is counting on upper layer, meaning application or client/class driver, to be compliant) ?

    I am more embedded developer so I am probably missing a point but I don't see where if so ?

    Regards.



    Friday, May 3, 2019 10:18 AM
  • doesn't it look like it is more a problem with usbd (or hcd) ? Which is not compliant with USB standard (or rather is counting on upper layer, meaning application or client/class driver, to be compliant) ?

    IMHO, it does not. Maybe automatic addition of ZLP is not always desired. For example consider that application can send a large transfer in several URBs, and only the last one is short. 

    It is unfortunate that the device cannot interrupt after each packet, but maybe it's possible to workaround this: detect timeout after the last full packet, or something like this.

    Maybe this device is not Designed For Windows ;)

    -- pa


    • Edited by Pavel A Friday, May 3, 2019 1:19 PM
    Friday, May 3, 2019 1:15 PM
  • > application builds an IRP including the buffer holding data to be transferred

    Applications do not build IRPs.  Applications make ReadFile, WriteFile, or DeviceIOControl requests.  The kernel's I/O manager converts that to an IRP.

    > client/class driver is dealing with the IRP and builds an URB

    Correct.

    > usbd deals with the URB and builds transactions(/transfers ?)

    Well, not so much.  USBD really just passes through.  The whole URB is usually considered a "transfer".

    > hcd creates packets, SOF

    Yes.  HCD chops the transfer up into packets.

    > So this would mean that actually usbser is indeed not responsible for the transfer management

    Not so.  The problem is that URBs cannot be arbitrarily long.  Say, for example, that my device wants to have a 16 MB transfer.  An URB can't be that long, so my driver has to submit it as a series of (for example) 512kB URBs.  In this case, I don't want a ZLP sent out until the end of the 16MB.  The HCD can't decide this.  It is up to the driver to send a zero-length URB if that occurs.  And you are right, most drivers don't do that.


    Tim Roberts | Driver MVP Emeritus | Providenza & Boekelheide, Inc.

    Friday, May 3, 2019 6:26 PM