Discussion:
[Libusb-devel] Windows support for libusb-1.0
Daniel Drake
16 years ago
Permalink
Before I closed the SF tracker, Josh Perry created a ticket saying that
he'd started working on windows support. I just went in to dig out his
findings that he posted. Now saved at http://www.libusb.org/ticket/1


> I began an attempt to create a libusb backend for Windows using WinUSB
> which would give libusb support for >=WinXP. I began by creating a nulled
> out backend struct for WinUSB to get my development environment (MSVC)
> compiling properly. It turns out that though some of the functionality for
> USB device access has been abstracted away, there are a lot of dependencies
> that don't exist on Windows and coding semantics that aren't supported by
> MSVC.
>
> I think I've not finally got it compiling by finding or writing wrappers
> for the dependencies and changing some of the code semantics. I haven't
> tested my changes yet so I don't know if it still buil
>
> I thought I would hit the compatibility bullet points here, to see if there
> is a better way to abstract these incompatibilities:
> * pthreads - pthreads is used for its mutex and cond variable
> functionality; Windows has an incompatible set of syncro primitives, I
> worked around this with http://sourceware.org/pthreads-win32/ but this
> project hasn't been updated since '06
> * poll - Windows doesn't have poll, it has mechanisms for similar
> functionality such as WaitForMultipleObjects etc... I stole some code from
> glib to make a fake poll() using those windows funcs.
> * sys/time.h - This isn't even a POSIX header, it's BSD, the definitions
> are pretty simple so I just patched a time.h together from some defs in
> winsock2.h and the original.
> * stdint.h - I know this is in the C99 standard, but MSVC doesn't have it,
> I grabbed a copy made by paul hsieh
> http://www.azillionmonkeys.com/qed/pstdint.h
> * variadic macros - these are used for some logging functionality. I'm not
> familiar with the syntax for variadic macros in GCC but I changed these
> definitions to work in MSVC, I'll have to test them in GCC to see if it is
> compatible.
> * typeof - This is a GCC extension not supported by MSVC, I changed the
> list-walking macros that used this to just take a last parameter of the
> type.
> * pipes - again the WinAPI has a different piping API, I worked around this
> by using io.h that defines a _pipe function and a macro from glib that
> defines a compatible POSIX pipe. This is the only point I had to use an
> ifdef in the code to include io.h and fcntl.h instead of unistd.h on MSVC.
>
> I would love to maybe abstract these dependencies away instead of creating
> such a fragile framework of stand-ins. Perhaps using alternatives that are
> cross-platform, or pulling the platform specific pieces out into different
> source files so that the implementation can be swapped out easily.
>
> I'm going to see if I can rig something up for testing so that I can tell
> if this port actually works properly (it compiles! now whats a segfault?).
> Once I've verified the functionality I can send in a patch for the changes
> I've made.
>
> I would love to hear some input on how you guys think this should proceed,
> I really want to make libUSB work for my projects on all platforms!
>
> - Josh Perry


Some brief comments from me:

pthreads: only used internally, so we don't have to actually use
pthreads on windows. We just have to use an alternative that provides
the basic locking primitives.

poll and the file descriptor model: when designing the library early on,
I took a look at glib and noted that they have somehow managed to
emulate windows event sources as numbers that look like file
descriptors, so I figure we can do the same.

MSVC: this seems adventurous. Shouldn't we start with just being able to
get it compiled and working with gcc on windows? Or is that impossible?
I have very little knowledge of windows development practicalities.

Daniel
Michael Plante
16 years ago
Permalink
Daniel Drake wrote:

> MSVC: this seems adventurous. Shouldn't we start with just being able to
> get it compiled and working with gcc on windows? Or is that impossible?
> I have very little knowledge of windows development practicalities.

I don't see it as excessively ambitious.

The link you posted indicates it compiles for Josh. I ported (and continue
to use weekly to daily) a different project in 2007 using pthreads-win32;
pthreads-win32 compiles fine in msvc6 and works. Ironically, the pthreads
stuff needed no changes to work (perhaps why it hasn't been updated
recently), but a lot of other supposedly-compatible stuff (e.g., sockets)
did need changes. I compiled libusb-win32-0.1 in msvc (except the driver,
which I use the ms ddk build environment for). MSVC provides a number of
integrated tools that make it easier to understand and navigate the source.
IMO, this more than makes up for any compatibility headaches. That said,
it's ultimately up to Josh, as my need for 1.0 is not pressing at the
moment.

I have heard of people doing Windows development with gcc, but never
investigated. If he decides to switch to gcc, I don't see any reason for it
to be difficult to also make it work in msvc, given that he seems to have
already identified the relevant compiler incompatibilities.

An interesting question down the line will be whether we can build on this
effort and make a backend that uses the existing libusb driver, which
supports certain features WinUSB skimped on.

Is there a repo up for this that I can start looking at?

Michael
Michael Plante
16 years ago
Permalink
> perhaps why it hasn't been updated recently

Hrmm, now that I look... Their CVS has been updated within the last 5
months:

http://sourceware.org/cgi-bin/cvsweb.cgi/pthreads/?cvsroot=pthreads-win32

There just hasn't been a release beyond 2.8 in 06, but 2.9 is what is in
"PTW32_VERSION" in the CVS pthread.h. The mailing list is also active, with
the maintainer posting frequently:

http://sourceware.org/ml/pthreads-win32/

In other words, I don't think it's abandoned.

Michael
Francesco Montorsi
16 years ago
Permalink
Daniel Drake <dsd <at> gentoo.org> writes:
> Before I closed the SF tracker, Josh Perry created a ticket saying that
> he'd started working on windows support.

Hi all,
first of all sorry for reviving this (relatively) old post.

I'm a developer of various open source projects and I'd like to help with the
development of the Windows port of libusb-1.x. I think it would be a great
addition to libusb even if the windows backend wouldn't/couldn't cover all
libusb features.
I already have some experience in Windows driver development (see
http://emfprinter.sourceforge.net/) and I think that using WinUSB API it
shouldn't be _so_ difficult to create the Win backend...

> I just went in to dig out his
> findings that he posted. Now saved at http://www.libusb.org/ticket/1

do you have any news about Josh Perry's work?
Or at least his email address so that we can ask him to post his un-finished
patch to the tracker?

I know that also Stephan Meyer has written a layer above the WinUSB API, but I
think it's an abstraction layer for libusb-0.1 (see http://alturl.com/jwpa
and
http://libusb-win32.svn.sourceforge.net/viewvc/libusb-win32/trunk/libusb1/)...

Maybe joining the efforts we could finally reach a unique (I don't understand
why libusb-win32 and libusb are kept separed - wouldn't be better to merge the
two SF projects?), modern, really cross-platform USB api :)

Thanks!
Francesco Montorsi
Pete Batard
16 years ago
Permalink
Hi Francesco,

Well, you're not the only that has been lamenting about lack of Windows
support for libusb 1.0x, and I have recently started working on a Windows
backend for libusb myself.

I am now at a stage where I have a fully working lsusb sample on Windows
(hub/device enumeration + readout of device descriptors), using the libusb
1.0x backend model, and after some much needed code cleanup, I'm hoping I
can start posting automake patches and new backend sources against the
current tree tomorrow...

The development chain I am using, which I think is the one that makes the
most sense for a Windows port, is MinGW, since it solves much of the issues
pointed by Josh Perry.
Once you add pthread-win32 to the equation and the poll/pipe compatibility
layer for git on Windows (look for compat/mingw.c in the git source tree), a
Windows port of libusb no longer looks that that problematic.

The only major issue I see right now is that not every USB operation is open
to user mode on Windows, and any kernel operation will require the writing
of a driver.
Because of the requirement for signed drivers on 64 bit platforms, which
makes for painful driver installation for regular users, I think it might be
a good idea to do as much as we can on the user side before we start looking
at anything that requires kernel access.
Hopefully, even with non-implemented kernel ops, a usermode only libusb can
have enough functions to be useful to some libusb based projects, and could
be a quick win. Then we can concentrate on a new win32 driver.

Of course, this is still very early stage, so if anybody has other ideas,
please feel free to chime in.

Anyway, I'll be posting more shortly.

Regards,

/Pete


2009/11/28 Francesco Montorsi <***@gmail.com>

> Daniel Drake <dsd <at> gentoo.org> writes:
> > Before I closed the SF tracker, Josh Perry created a ticket saying that
> > he'd started working on windows support.
>
> Hi all,
> first of all sorry for reviving this (relatively) old post.
>
> I'm a developer of various open source projects and I'd like to help with
> the
> development of the Windows port of libusb-1.x. I think it would be a great
> addition to libusb even if the windows backend wouldn't/couldn't cover all
> libusb features.
> I already have some experience in Windows driver development (see
> http://emfprinter.sourceforge.net/) and I think that using WinUSB API it
> shouldn't be _so_ difficult to create the Win backend...
>
> > I just went in to dig out his
> > findings that he posted. Now saved at http://www.libusb.org/ticket/1
>
> do you have any news about Josh Perry's work?
> Or at least his email address so that we can ask him to post his
> un-finished
> patch to the tracker?
>
> I know that also Stephan Meyer has written a layer above the WinUSB API,
> but I
> think it's an abstraction layer for libusb-0.1 (see http://alturl.com/jwpa
> and
>
> http://libusb-win32.svn.sourceforge.net/viewvc/libusb-win32/trunk/libusb1/).<http://libusb-win32.svn.sourceforge.net/viewvc/libusb-win32/trunk/libusb1/%29.>
> ..
>
> Maybe joining the efforts we could finally reach a unique (I don't
> understand
> why libusb-win32 and libusb are kept separed - wouldn't be better to merge
> the
> two SF projects?), modern, really cross-platform USB api :)
>
> Thanks!
> Francesco Montorsi
>
>
>
>
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus
> on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now. http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Libusb-devel mailing list
> Libusb-***@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/libusb-devel
>
Kustaa Nyholm
16 years ago
Permalink
>I am now at a stage where I have a fully working lsusb sample on Windows (hub/device enumeration + readout of device descriptors), using the libusb 10x backend model, and after some much >needed code cleanup, I'm hoping I can start posting automake patches and new backend sources against the current tree tomorrow..

Fantastic Pete,

I'm sure a lot of people appreciate this effort, thanks a million!

br Kusti
Tim Roberts
16 years ago
Permalink
Pete Batard wrote:
>
> The only major issue I see right now is that not every USB operation
> is open to user mode on Windows, and any kernel operation will require
> the writing of a driver.

Unfortunate, but true. I've become quite a fan of Microsoft's WinUSB
interface. It does the job nicely, without a lot of overhead, and it
integrates nicely into a multithreaded application, but it doesn't
handle isochronous pipes at all. (Of course, neither does the KMDF
kernel USB interface. You have to use "old-fashioned" custom URBs.)


> Because of the requirement for signed drivers on 64 bit platforms,
> which makes for painful driver installation for regular users, I think
> it might be a good idea to do as much as we can on the user side
> before we start looking at anything that requires kernel access.

Yes, this is an awkward point. As long as someone who is willing to
speak for the project has an approved code-signing certificate, they can
sign a "blessed" 64-bit driver binary and release it in binary form.
For most users, that probably works just fine. I know there are purists
who won't like being forced to use a pre-built binary. I guess those
people will be forced to override the signing requirement every time
they boot.

--
Tim Roberts, ***@probo.com
Providenza & Boekelheide, Inc.
Peter Stuge
16 years ago
Permalink
Hi,

Daniel Drake wrote:
> pthreads: only used internally, so we don't have to actually use
> pthreads on windows. We just have to use an alternative that
> provides the basic locking primitives.

While pthreads-win32 works perfectly fine (although not much updated)
I think it is much nicer to use win32 primitives in a win32 backend.

This might mean some need for internal locking abstraction.


> MSVC: this seems adventurous. Shouldn't we start with just being
> able to get it compiled and working with gcc on windows?

We should definately try to make the library easily usable also for
developers using MSVC, but I agree that since we're used to gcc it's
easier to look to MinGW for starters. It's usually easier to go this
way (gcc->MSVC) than the other way around, for a given source.


> Or is that impossible?
> I have very little knowledge of windows development practicalities.

It's very possible. Using crossdev it's almost a noop to get a
Windows compiler installed on a Linux system, with headers and all.
The tuple is i_86-mingw32, where you get to choose _ yourself, 3 is
the minimum that makes sense, to be maximally portable. I've made a
small source example that builds in MinGW if you want to look;
http://stuge.se/dlg.zip but remember to add the tuple prefix for
cross compilation. (The zip is for native MinGW installs in Windows.)


Pete Batard wrote:
> Well, you're not the only that has been lamenting about lack of
> Windows support for libusb 1.0x, and I have recently started
> working on a Windows backend for libusb myself.

Great. I was going to look into doing so too.


> I am now at a stage where I have a fully working lsusb sample on
> Windows (hub/device enumeration + readout of device descriptors),
> using the libusb 1.0x backend model, and after some much needed
> code cleanup, I'm hoping I can start posting automake patches and
> new backend sources against the current tree tomorrow...

Ok. Depending on how much code it is and how big (if any) changes
will be required it might make sense to add another repo, or branch,
for the code while it is being merged.


> The development chain I am using, which I think is the one that
> makes the most sense for a Windows port, is MinGW, since it solves
> much of the issues pointed by Josh Perry.

I think it is nicer too. And maybe most importantly it allows us to
fairly easily produce win32 binaries, which is usually very much
appreciated at least by MSVC users, where I guess buliding libraries
is a bit more complicated than building apps. I think it is a great
benefit to get Windows developers to also use the libusb-1.0 API.


> Once you add pthread-win32 to the equation

Again, I disagree with relying on pthread-win32 and would prefer to
use native Windows locking. So this particular platform might mean
that internal locking needs some work.


> and the poll/pipe compatibility layer for git on Windows (look for
> compat/mingw.c in the git source tree),

Sweet! Is the license compatible?


> The only major issue I see right now is that not every USB
> operation is open to user mode on Windows,

What exactly is possible? MSDN links would be great too. If you want,
feel free to make a Trac account and add a wiki page about it. Let me
know if you need any help with that.


> I think it might be a good idea to do as much as we can on the user
> side before we start looking at anything that requires kernel
> access.

Yes - I think that sounds good.


> Anyway, I'll be posting more shortly.

Looking forward to it!


//Peter
Pete Batard
16 years ago
Permalink
Alright, I have now posted my code.

Now, to reply to the various points raised:

o pthreads-win32
I agree that using native threads on Windows would be a lot better, and
provide better performance. However, since I haven't had that much threads
exposure, I don't have any idea about how much effort would be involved. My
guess is that it would likely require some modifications to the core files
to make threading more generic, rather than assume that Posix threads are
available.
If we find that pthread-win32 works good enough for the time being, it's
probably best to leave the threads harmonization effort for after we have a
fully functional version of libusb on Windows.

o MSVC
Pretty much the same as above. Yes, it would be nice to have an MSVC
compatible version of libusb, and it would probably have a greater reach as
a result, but let's take it one step at a time.

o git's compat/mingw source license
git is GPL, libusb is LGPL. Unless I'm mistaken, that should work, right?

o userland available USB API functions under Windows
The usermode / kernel mode USB I/O breakdown is referenced at
http://msdn.microsoft.com/en-us/library/ms793173.aspx. Now, I don't know
enough of libusb yet to know how bad this will affect us.
On the other hand, if we want to target Vista and later version of Windows
only (probably not a good idea, but it might make our lives easier), and we
don't need data streaming through isochronous endpoints (Does libusb require
it?), it looks like we might be able to get away with getting full access to
USB functions without the need for a driver. See
http://msdn.microsoft.com/en-us/library/aa476426.aspx
About that Trac account, I'll see what I can do.

Regards,

/Pete
Pete Batard
16 years ago
Permalink
Oh, one thing I forgot. I don't think a new branch/repo is necessary, as
core changes are very minimal and should leave all existing OSes unaffected
(I tested this on Linux), but I leave the choice up to you guys.

Regards
Xiaofan Chen
16 years ago
Permalink
On Mon, Nov 30, 2009 at 4:25 AM, Pete Batard <***@gmail.com> wrote:

> o userland available USB API functions under Windows
> The usermode / kernel mode USB I/O breakdown is referenced at
> http://msdn.microsoft.com/en-us/library/ms793173.aspx. Now, I don't know
> enough of libusb yet to know how bad this will affect us.


> On the other hand, if we want to target Vista and later version of Windows
> only (probably not a good idea, but it might make our lives easier), and we
> don't need data streaming through isochronous endpoints (Does libusb require
> it?), it looks like we might be able to get away with getting full access to
> USB functions without the need for a driver. See
> http://msdn.microsoft.com/en-us/library/aa476426.aspx
> About that Trac account, I'll see what I can do.

WinUSB is a good backend. In the current libusb-win32 1.0 branch, there
are three backend, one is WinUSB (XP, Vista, Windows 7, both 32bit and
64bit), the other one is HID (pretty much Windows 98SE and above), yet
the other is the libusb-win32 device driver (kernel mode driver and user
mode DLL, Windows 98SE/ME/2K/XP/Vista, XP 32bit/64bit, Vista 32bit
only, could work for Vista 64bit if a paid digital signature recognized by
Microsoft for device driver use is used). You can still have these multiple
backend.

If you have WinUSB and HID backend done, isochronous transfer
is missing but you have covered many USB device out there which
may want to use libusb.


--
Xiaofan http://mcuee.blogspot.com
Graeme Gill
16 years ago
Permalink
Xiaofan Chen wrote:

> WinUSB is a good backend. In the current libusb-win32 1.0 branch, there
> are three backend, one is WinUSB (XP, Vista, Windows 7, both 32bit and
> 64bit), the other one is HID (pretty much Windows 98SE and above), yet
> the other is the libusb-win32 device driver (kernel mode driver and user
> mode DLL, Windows 98SE/ME/2K/XP/Vista, XP 32bit/64bit, Vista 32bit
> only

Ideally libusb would support both WinUSB and the kernel driver,
with the .inf selecting the appropriate support according
to the operating system version (ie. WinUSB where it
is available, the libusb kernel driver where it is not).

Wrapping the OS dependencies in a small compatibility
layer is the go too, to get it working with VC as well as
mingw etc. I'm happy to contribute any of the compatibility
code from argyllcms/spectro/ntio.c, unixio.c etc. if that
can be of any help (mainly the simple thread class may be of
interest, or a place to start).

Graeme Gill.
Francesco
16 years ago
Permalink
Hi Xiaofan,

2009/11/30 Xiaofan Chen <***@gmail.com>:
> WinUSB is a good backend. In the current libusb-win32 1.0 branch, there
> are three backend, one is WinUSB (XP, Vista, Windows 7, both 32bit and
> 64bit),
do you mean the backend stored here:
http://libusb-win32.svn.sourceforge.net/viewvc/libusb-win32/trunk/libusb1/
?
Unfortunately there's no activity there since 2 years and AFAIK it's
not included in the libusb releases... (in fact the libusb-win32
forums contain lots of people asking for 64bit support AFAIK).

> the other one is HID (pretty much Windows 98SE and above), yet
> the other is the libusb-win32 device driver (kernel mode driver and user
> mode DLL, Windows 98SE/ME/2K/XP/Vista, XP 32bit/64bit, Vista 32bit
> only, could work for Vista 64bit if a paid digital signature recognized by
> Microsoft for device driver use is used). You can still have these multiple
> backend.
>
> If you have WinUSB and HID backend done, isochronous transfer
> is missing but you have covered many USB device out there which
> may want to use libusb.
I agree...

Francesco
Pete Batard
16 years ago
Permalink
2009/11/30 Francesco <***@gmail.com>

> do you mean the backend stored here:
> http://libusb-win32.svn.sourceforge.net/viewvc/libusb-win32/trunk/libusb1/
> ?
> Unfortunately there's no activity there since 2 years and AFAIK it's
> not included in the libusb releases... (in fact the libusb-win32
> forums contain lots of people asking for 64bit support AFAIK).
>

OK, I have to admit that I probably didn't look hard enough because I wasn't
aware of this port before I started coding.

But I guess that's because libsub-win32 is a separate project from libusb,
and the aim of this new Windows implementation should be to do whatever it
takes to conform and be integrated to mainstream libusb, as well as provide
full Windows 64 bit support (but I don't see it as much of an issue for 64
bit until we try to implement a driver part, as MinGW-w64 seems to be fairly
stable and almost 100% compatible with MinGW-w32)

I do like the separation of HID, WinUSB and driver implementations that
libusb-win32 1.0 has though, although I plan to concentrate on WinUSB for
now. I'll try to factor in some abstraction in the source when using WinUSB
so that HID and driver can be easily added later.

Still need to code some more (and have a closer look at the libusb-win32 1.0
source) before I get a better picture of how we can unfold this whole
thing...

Regards,

/Pete
Continue reading on narkive:
Loading...