[libusb] How to access composite devices properly on Windows

[Matthias Bolte]

Hi,

I'm using a composite device. It's first function (interface 0) is
used with WinUSB, it's second function (interface 1 and 2) is a CDC
ACM serial console.

This setup works fine, but there is an issue on Windows I'd like to discuss.

libusb_get_device_list() reports three libusb_devices with the VID:PID
of my composite device, but there is only one device physically
connected. As it turns out only for the third libusb_open() call will
succeed. The other two fail with LIBUSB_ERROR_NOT_SUPPORTED.

After some digging around in the libusb code I figured out the libusb
is reporting the two functions of the composite device as if their
were devices as well. This are the two "devices" libusb_open() will
fail for with LIBUSB_ERROR_NOT_SUPPORTED.

Now I wonder If this is intentional or if this is a bug. As far as I
can tell there is no way I can tell these three libusb_devices apart
using the libusb API, except from trying to libusb_open() them and see
which one fails and which one doesn't. Also the xusb tool doesn't work
with this composite device, because libusb_open() fails with
LIBUSB_ERROR_NOT_SUPPORTED.

I can workaround this in my program, but I'd like to know if this is
intentional, or if this is a bug, of if my understanding of how
composite devices are supposed to work on Windows is wrong. Because
once I ignore the failing libusb_open() calls and found the actual
device then I can claim interface 0 and everything works as expected.

Regards,
Matthias

[Tim Roberts]

Post by Matthias Bolte
After some digging around in the libusb code I figured out the libusb
is reporting the two functions of the composite device as if their
were devices as well. This are the two "devices" libusb_open() will
fail for with LIBUSB_ERROR_NOT_SUPPORTED.
Now I wonder If this is intentional or if this is a bug.

At worst, it is an impedance mismatch between the Linux philosophy and
the Windows philosophy. In the Windows world. your hardware really is
exposed as 3 separate devices: the composite device, a WinUSB device,
and a communication device. You, as a driver writer, would write an INF
that matches exactly the device you want to target. The fact that it
happened to be part of a larger composite device is supposed to be
irrelevant. Windows wants to abstract away those details.

In the libusb world, things have been geared to think of the whole
VID/PID as one device, and then you claim the interfaces you happen to
want to drive.

Post by Matthias Bolte
As far as I
can tell there is no way I can tell these three libusb_devices apart
using the libusb API, except from trying to libusb_open() them and see
which one fails and which one doesn't.

Yes, but only one of those interfaces has a WinUSB driver, and hence
only one will talk to libusb. I think that's really your path.

[michel]

i have exactly the same type of device
a vendor specific function on intf 0 + 2x cdc/acm on itf 1,2 and 2,3
but i can't remember any problem,
I do not have the code with me here but i think why i'm doing is to first
list and match for vid/pid and then interface class and sub class code to
identify the right function.
doing so ther's no way to mix and try to open wrong interface




--
View this message in context: http://libusb.6.n5.nabble.com/libusb-How-to-access-composite-devices-properly-on-Windows-tp5714029p5714031.html
Sent from the LibUSB Dev mailing list archive at Nabble.com.

[Matthias Bolte]

This post might be inappropriate. Click to display it.

[Pete Batard]

Post by Matthias Bolte
So libusb is modeled the Linux way and should only really report one
libusb_device for my one physical device.

That's not exactly accurate.

libusb was designed with the aspiration of being a cross platform API,
yet (and I speak here as the guy who had to code much of the API for
Windows) without the faintest awareness of how non POSIX platform, such
as Windows, were actually handling their USB stacks.
So I would say that libusb simply happened to model itself against the
Linux way, for lack of better knowledge of other platforms (proof being
some non cross-platform Linux-isms like attach/detach driver for
instance), in the hope that it would be good enough.

Of course, when the Windows backend got added, we then found that we had
to try to fit a square peg into a round hole behind the scenes...

If you ever wondered why the length of the Windows backend code dwarfs
the one from every other platform, and why its enumeration seems so
convoluted, this is the reason.

Post by Matthias Bolte
I think that this is a bug in the libusb Windows backend,

Let me give you the trick here: just consider the USB composite parent
as an extra hub on Windows - that's pretty much all there is to it (and
to your port numbering issue).

And yes, that means that you can't blindly write libusb code to access
composite devices on Linux, and expect it to work unmodified on Windows.
But the other option you suggest, which is: the Windows backend should
spend an awful lot of time and effort to abstract the fact that a
composite device with 3 interfaces may use 3 completely different
Windows drivers (say: HID, WinUSB and "none", or even worse, something
proprietary that we don't have an API for and can't replace), to try to
make it transparent to end users, is just not viable.

Heck, if anything, because the required changes would be A LOT easier to
implement for non Windows platforms, the smart thing to do with regards
to access uniformization would be to make Linux, OS X and co, split
their composite devices, like Windows does, to bring everyone under the
same roof.

As far as I am concerned, this really is HOW a library, that has an
established goal of being cross platform, should actually _compromise_
on achieving that goal, instead of trying to push for the original
platform the library happened to be developed for to be "just a bit more
equal than the others". But of course, if we ask Linux people to
compromise and adopt Windows-isms, just as you are, in effect,
requesting Windows libusb users to adopt alien Linux-isms when it comes
to composite devices, they'll probably just scream bloody murder...

All in all then, I don't see your request as something that can be
considered, even more so as you seem happy with your current workaround
so any work we'd do would not benefit you anyway. You've seen how
lengthy and difficult to digest the Windows code is, which again is a
direct consequence of trying to make Windows masquerade as a Linux-like
affair, and yet you seem to be expecting that somebody (not you) should
try to spend a lot of time and effort adding even more convolution for
something that, rather than being a bug, is in fact a different OS
paradigm (since it is the privilege of whoever designs the USB stack of
their OS to choose how they want to handle composite devices).

Regards,

/Pete

[Matthias Bolte]

Post by Pete Batard

Post by Matthias Bolte
So libusb is modeled the Linux way and should only really report one
libusb_device for my one physical device.

That's not exactly accurate.
libusb was designed with the aspiration of being a cross platform API,
yet (and I speak here as the guy who had to code much of the API for
Windows) without the faintest awareness of how non POSIX platform, such
as Windows, were actually handling their USB stacks.
So I would say that libusb simply happened to model itself against the
Linux way, for lack of better knowledge of other platforms (proof being
some non cross-platform Linux-isms like attach/detach driver for
instance), in the hope that it would be good enough.
Of course, when the Windows backend got added, we then found that we had
to try to fit a square peg into a round hole behind the scenes...
If you ever wondered why the length of the Windows backend code dwarfs
the one from every other platform, and why its enumeration seems so
convoluted, this is the reason.

Oh, I'm well aware of all of this, don't worry.

My point is just that if the libusb API is modeled a certain way and
if libusb's stated goal is to have one model that works the same on
all platforms (at least that's what I assume is the goal of libusb)
then I think the current handling of composite devices (or at least of
my specific device) on Windows doesn't follow the model.

I didn't want to offend or blame anybody. I'm sorry, I've seemed to
offended you and I should have chosen my words more carefully.

Post by Pete Batard

Post by Matthias Bolte
I think that this is a bug in the libusb Windows backend,

Let me give you the trick here: just consider the USB composite parent
as an extra hub on Windows - that's pretty much all there is to it (and
to your port numbering issue).
And yes, that means that you can't blindly write libusb code to access
composite devices on Linux, and expect it to work unmodified on Windows.
But the other option you suggest, which is: the Windows backend should
spend an awful lot of time and effort to abstract the fact that a
composite device with 3 interfaces may use 3 completely different
Windows drivers (say: HID, WinUSB and "none", or even worse, something
proprietary that we don't have an API for and can't replace), to try to
make it transparent to end users, is just not viable.
Heck, if anything, because the required changes would be A LOT easier to
implement for non Windows platforms, the smart thing to do with regards
to access uniformization would be to make Linux, OS X and co, split
their composite devices, like Windows does, to bring everyone under the
same roof.
As far as I am concerned, this really is HOW a library, that has an
established goal of being cross platform, should actually _compromise_
on achieving that goal, instead of trying to push for the original
platform the library happened to be developed for to be "just a bit more
equal than the others". But of course, if we ask Linux people to
compromise and adopt Windows-isms, just as you are, in effect,
requesting Windows libusb users to adopt alien Linux-isms when it comes
to composite devices, they'll probably just scream bloody murder...
All in all then, I don't see your request as something that can be
considered, even more so as you seem happy with your current workaround
so any work we'd do would not benefit you anyway. You've seen how
lengthy and difficult to digest the Windows code is, which again is a
direct consequence of trying to make Windows masquerade as a Linux-like
affair, and yet you seem to be expecting that somebody (not you) should
try to spend a lot of time and effort adding even more convolution for
something that, rather than being a bug, is in fact a different OS
paradigm (since it is the privilege of whoever designs the USB stack of
their OS to choose how they want to handle composite devices).

Well, I didn't really request anything here, I was merely looking for
advise on "How to access composite devices properly on Windows",
because I had trouble with it and wasn't sure where the problem was
located. Am I using libusb wrong, is it a driver issue like the
Renesas one, or is there a problem with my device that only happens to
affect Windows, or is my understanding of the whole situation just
wrong, etc.

Anyway, thanks for giving me a detailed insight into your view of the
problems of libusb.

Also I'm happy to help fixing problems and improving the whole
situation. I've already submitted some smaller patches for libusb in
the past. So don't blame me for just wanting somebody else to do the
work. You're not the only person that has time constrains. But my
additional problem is that I'm more of a libusb user than a libusb
developer. I don't have the knowledge of all the details and inner
workings of libusb that's required to do a major overhaul. That's why
I've settled for the crappy short term solution and added a workaround
for my specific problem. Because that's what I can easily test,
because my patched libusb version only has to work with my specific
USB device.

[Pete Batard]

Post by Matthias Bolte
My point is just that if the libusb API is modeled a certain way and
if libusb's stated goal is to have one model that works the same on
all platforms (at least that's what I assume is the goal of libusb)
then I think the current handling of composite devices (or at least of
my specific device) on Windows doesn't follow the model.

One thing I should have mentioned, because I'm not sure if you're aware
of it, is that you can replace the Composite Parent driver on Windows
with one of the libusb-supported drivers. If you do that, then libusb
should be able to access your device in a manner that is closer to its
Linux counterpart, as it will be enumerated as single device with
multiple interfaces (within the limitations imposed by the Windows drivers).

So Windows can actually follow the "Linux" model, but it just happens
that we don't do that as default, because the OS automatically installs
a system driver for the composite parent and it's a major pain to get
users to replace it (plus if one of your interface is handled by a
system driver, such as HID or Mass Storage, you will lose default OS
access to it). But, depending on your target, I have to wonder if
replacing composite parent driver might not be what you've been after
all along. FYI, Zadig can do just that if you enable "List all devices"
and uncheck "Ignore Hubs or Composite Parents".

And to come back to what I was saying earlier, you also have to bear in
mind that, outside of whatever we decide to do in libusb, if they ever
think there's a sufficient use case for it, the Linux kernel people
could very well decide to offer an _option_ to split a composite device
into one device per interface, in a manner similar to what Windows does.
In that case the idea that there should be an "established _single_
model libusb should follow, to handle composite" would make even less
sense, and we might actually find that we backed ourselves into a corner
by declaring that one model is more valid than another.

Regards,

/Pete

[Matthias Bolte]

Post by Pete Batard

Post by Matthias Bolte
My point is just that if the libusb API is modeled a certain way and
if libusb's stated goal is to have one model that works the same on
all platforms (at least that's what I assume is the goal of libusb)
then I think the current handling of composite devices (or at least of
my specific device) on Windows doesn't follow the model.

One thing I should have mentioned, because I'm not sure if you're aware
of it, is that you can replace the Composite Parent driver on Windows
with one of the libusb-supported drivers. If you do that, then libusb
should be able to access your device in a manner that is closer to its
Linux counterpart, as it will be enumerated as single device with
multiple interfaces (within the limitations imposed by the Windows drivers).

I think I've read about this in the libusb wiki section on github or
somewhere related, but never considered it.

Post by Pete Batard
So Windows can actually follow the "Linux" model, but it just happens
that we don't do that as default, because the OS automatically installs
a system driver for the composite parent and it's a major pain to get
users to replace it (plus if one of your interface is handled by a
system driver, such as HID or Mass Storage, you will lose default OS
access to it). But, depending on your target, I have to wonder if
replacing composite parent driver might not be what you've been after
all along. FYI, Zadig can do just that if you enable "List all devices"
and uncheck "Ignore Hubs or Composite Parents".

I'm actually fine with the default way of Windows using its default
composite parent driver. My device contains a WinUSB device at MI_00
that uses WCID to tell Windows (at least 8) that it's WinUSB
compatible. It also contains a CDC ACM device on MI_01 to expose a
serial console to be used with Putty or some other terminal tool.

All of this works fine, no problem with that. My only problem with
libusb here is that I get three devices on Windows with the same
VID:PID. The composite parent, the WinUSB device and the CDC ACM
device. But I only want to access the WinUSB MI_00 device of it. To
achieve this I have to open the composite parent device and claim
interface 0, then it all works.

But as far as I can tell from the libusb API I have no way of telling
these three libusb_devices apart. I cannot know beforehand which of
the three devices is the composite parent I need to open. I can only
try to open them all and see which one succeeds. That's why I added a
workaround to the init_device function in the libusb Windows backend
to filter out the MI_00 and MI_01 device from the libusb_devices list.

I just retested this to check that I don't get the story wrong, guess
what, the problem vanished. It was there on three different Windows
PCs last week and now it's gone. That's confusing me, I changed
nothing in the setup.

Before I was getting three init_device calls (for composite, MI_00 and
MI_01) and therefore three libusb_devices with the same VID:PID and
one set_composite_interface call (for MI_00). But now I get only one
init_device call (for composite) plus the set_composite_interface (for
MI_00). That's exactly how I think it should work. My point was that I
should not get init_device calls for MI_00 and MI_01, but I did last
week.

Sorry, I think something strange is going on here. I think libusb is
basically fine, but last week I managed to trip the enumeration logic
in some way I don't understand yet and am currently unable to
reproduce. I'll have to investigate this.