Skip to content

Add Android 5-7 support#242

Closed
vianney wants to merge 7 commits into
libusb:masterfrom
vianney:android
Closed

Add Android 5-7 support#242
vianney wants to merge 7 commits into
libusb:masterfrom
vianney:android

Conversation

@vianney
Copy link
Copy Markdown
Contributor

@vianney vianney commented Dec 13, 2016

This patch series adds support for recent Android devices (tested up to 7.1.1). On such devices, SELinux policies prevent the use of sysfs and reading /dev/bus/usb. The only way to use libusb as a regular user is to get a file descriptor through the Android Platform API and pass it on. All operations must use that file descriptor.

This series adds an API function to libusb to wrap an open file descriptor as a libusb_device_handle. To ease reviewing, it is decomposed as follows:

  • Patches 1-3 contain light refactoring and small utility extensions needed later on.
  • Patch 4 adds the public API function.
  • Patch 5 implements the new API for the Linux platform.
  • Patch 6-7 ensure the libusb library is able to initialize on Android.
  • Patch 8 is a minor warning fix as a bonus.

Please note that patch 7 (disable hotplug events on Android) means that libusb cannot be used anymore standalone on a rooted Android device. Is that an issue? If so, I guess the way to go is to make it a compile-time option.

@mcuee
Copy link
Copy Markdown
Member

mcuee commented Jan 12, 2017

Please refer to the comments at #233

#242 can solve the problem. The naming of the function is confusing though. The function libusb_wrap_fd is escentially opening the usb by file descriptor. Also, it will not add the usb device the to device list in libusb, so mind that some libraries depend on libusb may need to be modified to make it working.

@vianney
Copy link
Copy Markdown
Contributor Author

vianney commented Jan 12, 2017

@mcuee @kennethtang4 Thanks for the feedback. I am willing to improve this patchset to get it merged upstream. So let's get the discussion started.

Concerning the naming, I have chosen "wrap" over "open" to make it clear that we do not open a new file descriptor (and hence, that the provided file descriptor must not be closed), but I would not mind naming it "libusb_open_fd" or anything else. Suggestions are welcome.

I also have the concern that file descriptors are backend-specific (event though most OS have such concept). Having them exposed in the user API is not very nice, but I am afraid there is no way around it. Maybe we should replace the int argument by something more platform-agnostic, but what?

About the device list: yes, this feels a bit hacky. I would appreciate some input from libusb developers on the best way to handle this. The requirement for Android is that we should not rely on anything besides the provided file descriptor. This means that in the worst case, if the link in /proc/self/fd is unreadable or non-standard, we do not know the bus number of the device (afaik, there is no ioctl for that). Maybe we could do the following:

  • If we got the complete device address: reuse existing libusb_device if it was enumerated. Otherwise create a new one, add it to the list, and remove it when the handle is closed.
  • If the bus number is unknown: create a new libusb_device using the file descriptor as session id, add it to the list, and remove it when the handle is closed.

To libusb maintainers: is there any interest in merging this kind of patches? If so, should we keep compatibility with using libusb in the standard way on a rooted Android device (i.e., with hotplug enumeration and libusb_open)?

@mcuee
Copy link
Copy Markdown
Member

mcuee commented Jan 24, 2017

@vianney It will be helpful that you can subscribe to libusb mailing list and post there to seek comments. Thanks.

@reflog
Copy link
Copy Markdown

reflog commented Apr 6, 2017

@vianney Thanks for the patch! If I'm calling libusb_wrap_fd instead of libusb_open, does it mean I shouldn't call libusb_close?

@chipweinberger
Copy link
Copy Markdown

chipweinberger commented Apr 28, 2017

Could someone explain how this Android solution should be used? What to call and in what order?

rexut added a commit to rexut/libusbpp that referenced this pull request May 18, 2017
Mostly untested. Write more in the commit message.

libusb/libusb#242

Signed-off-by: Stephan Linz <linz@li-pro.net>
rexut added a commit to rexut/libusbpp that referenced this pull request May 18, 2017
Mostly untested. Write more in the commit message.

libusb/libusb#242

Signed-off-by: Stephan Linz <linz@li-pro.net>
rexut added a commit to rexut/libusbpp that referenced this pull request May 18, 2017
Mostly untested. Write more in the commit message.

libusb/libusb#242

Signed-off-by: Stephan Linz <linz@li-pro.net>
@dirkhh
Copy link
Copy Markdown

dirkhh commented May 30, 2017

This looks like it might be what I'm looking for in order to be able to connect to USB dive computers from Android... but just like @chipweinberger I am wondering how to use these patches in order to be able to open an FTDI device from Android... Help would be appreciated.

@lenaschimmel
Copy link
Copy Markdown

I'm also currently trying to use these patches, and libusb_wrap_fd seems to work, basically. However, I'm unsure what else is supposed to work.

I'm trying to use existing code (libmaxtouch) that relies on the contents of the libusb_device_descriptor struct. It seems to me as if the common way to get that descriptor would be to use libusb_get_device_list and then libusb_get_device_descriptor and I do not know if this just fails for me and my devices, or if this is not supported at all on Android.

I tried the listdevs.c example - embedded into an Android App via the NDK - and I get 0 devices. I tried this on Android 7.0 on x86 (a 64 bit system, but with 32 bit binaries) and Android 5.0 on armeabi-v7a. In both cases a USB device was attached and my app had the needed permission and already acquired the file descriptor from the Java API. On both devices, libusb reports: found usbfs at /dev/bus/usb, but I understand that find_usbfs_path will always print this message when it actually cant' find any usbfs. So I checked for myself:

  • On the Android 7.0 system and the device of interest is actually present at /dev/bus/usb/001/004 as I can confirm using adb shell. The device is rooted.
  • On the 5.0 system, I cannot even ls /dev/bus/usb via adb shell because of unsufficient permissions. The device is not rooted.

If it was - at least on some devices - really impossible to get the libusb_device_descriptor from C, I think it should be possible to get most of that from the Java API and somehow pass it back to libusb, however that seems overly complicated.

@NickAtAccuPS
Copy link
Copy Markdown

NickAtAccuPS commented May 31, 2017

@reflog If I'm reading the comment in this commit you should call libusb_close to close the libusb object, but you would still need to manually close the OS file descriptor manually afterward.

@vianney Does the above sound right?

@lenaschimmel Not immediately useful for libusb but you might want to look into getRawDescriptors in the Android API.

@lenaschimmel
Copy link
Copy Markdown

@NickAtAccuPS On a first glimpse, that seems very useful. The android documentation says nothing about its endianness, but that should be an easy trial-end-error process. Also, it's unclear to me which descriptor(s) is/are returned. If it's just the 18 byte device descriptor, that's not entirely useful, because there seems to be no API to get the other descriptors (Configuration Descriptors, Interface Descriptors, Endpoint Descriptors, String Descriptors), but I guess this will return all the descriptors concatenated, just as if we read them from the usbfs? (Sorry, didn't have time to just try it on Android).

If so, I think we could just add a few more parameters to libusb_wrap_fd so that we get:

int LIBUSB_CALL libusb_wrap_fd (libusb_context *ctx,
				int fd,
				libusb_device_handle **dev_handle,
				uint8_t bus_number,
				uint8_t port_number,
				char* raw_descriptor,
				int desc_length);

Am I missing anything?

@vianney
Copy link
Copy Markdown
Contributor Author

vianney commented Jun 1, 2017

Guys, did you read the documentation? Let me quote the doc for libusb_wrap_fd():

Wrap an open file descriptor and obtain a device handle for the underlying
device. A handle allows you to perform I/O on the device in question.

The file descriptor must remain open until libusb_close() is called.
The file descriptor will not be closed by libusb_close().

Internally, this function creates a temporary device and makes it
available to you through libusb_get_device(). This device is destroyed
during libusb_close(). The device shall not be opened through libusb_open().

and what I wrote in the description of this pull request:

This patch series adds support for recent Android devices (tested up to 7.1.1). On such devices, SELinux policies prevent the use of sysfs and reading /dev/bus/usb. The only way to use libusb as a regular user is to get a file descriptor through the Android Platform API and pass it on. All operations must use that file descriptor.

It follows that:

  1. You have to use the Java Android API to open a file descriptor to the USB device before you can use libusb.
  2. As a regular user, you cannot use libusb's enumeration functions to get a libusb_device because you cannot read sysfs and /dev/bus/usb.
  3. After obtaining a libusb_device_handle, you can call libusb_get_device() to get a libusb_device which can be used to get all descriptors you want. No need for additional trickery with the Android API.
  4. You must of course properly clean up the libusb_device_handle by calling libusb_close() when you are done.

To maintainers: when I get some time, I will rebase this patch onto master and post to the mailing list. Unfortunately, other projects keep me quite busy right now.

@lenaschimmel
Copy link
Copy Markdown

@vianney Thank you very much! I was (mostly) aware of your points 1, 2 and 4 but I completely missed 3 (probably because the existing code I worked with assumed to have access to the descriptors before opening the device). A quick test on my device confirms that I understood it correctly this time.

@akshaytaru2007
Copy link
Copy Markdown

Hi All,
First it is not issue, I am asking for help. I am trying to connect a Hardware(IoT based) via USB to Android. USB Manager not detecting the devices and not listing the device. The Android is non-rooted, 6.1 It does support OTG.
As all of you successfully used libusb in Android, I am still struggling as there is no enough documentation or support.
I have built the .SO file using NDK and added those under jniLibs/AIBs/libusb1.0.SO

Now in my native-lib.cpp I am trying to use libusb's functions to init or get list of device. But I am failing.
I belive there must be some configuration needs to be done.
Those who have succeeded kindly help.

I have also posted Question on stackoverflow related to this: https://stackoverflow.com/q/44281689/1468354

If you need any more info. I will provide

@lenaschimmel
Copy link
Copy Markdown

lenaschimmel commented Jun 2, 2017

@akshaytaru2007 As stated in this readme, the usual libusb will not work on you unrootet device and you must use "android.hardware.usb.UsbManager to request permission to use the UsbDevice and then opening the device".

Also, you need a patched version of libusb to actually handle the device descriptor. This PR that you are commenting on contains the necessary changes, but in your StackOverflow question you did not mention that you are basing your work on this PR. So are you on the master branch or on this PR?

Regarding the list of devices, I think it is not possible to use libusb_get_device_list on Android 6.1 without root, no matter if you use the code of this PR or not. Instead, you have to use Android's Java API for USB Host. In your Java code, you can list the devices, request permission, open the device and then give the file descriptor to your native library, which can then call libusb_wrap_fd.

@ALL Maybe I can setup a sample Android project somewhere next week and post a link. This should make it easier for new users to get started.

@vianney I respect that you don't have much time right now. I just want to say that it seems that maybe rebasing might be a matter of 5 minutes work - I just had to fix two trivial merge conflicts. Anyway, I'm looking forward to the day when this goes upstream!

@akshaytaru2007
Copy link
Copy Markdown

@lenaschimmel I tried using android.hardware.usb.UsbManager to give the list of device but it is giving me empty list though the device is plugged in via USB. So I can not request permission to UsbDevice
I am using libusb-1.0.20 version and the code is not part of this PR.
I do have more info about Hardware. The Hardware has the OS Windows CE 7.0. It has ActiveSync USB device class selected.

@lenaschimmel As you guys know much about USB communication, I am asking your assistance that is there any way I can talk to this device
PS. I can not disclose the Device details as it's client's hardware

@lenaschimmel
Copy link
Copy Markdown

lenaschimmel commented Jun 6, 2017

@akshaytaru2007 I'm very new to USB communication and libusb myself, so I can't really help that much, but I'll try...

I think if you get an empty list from android.hardware.usb.UsbManager there's some problem either with your Android / Java code, or your hardware / Android OS version does not support USB host functionality like it should. Anyway, that has nothing to do with libusb and should IMHO be discussed elsewhere, for example by posting another question on StackOverflow.

Once you get the fileDescriptor from the UsbDeviceConnection, it's time to try libusb again. Also, I think you will not be successful if you use the standard version of libusb-1.0.20, you will need to compile it from this branch / PR. (Exception: if some time passes, and this PR gets merged into master by then, you can of course just grab the newest version of libusb.)

rexut added a commit to rexut/libusbpp that referenced this pull request Jun 16, 2017
Mostly untested. Write more in the commit message.

libusb/libusb#242

Signed-off-by: Stephan Linz <linz@li-pro.net>
@FDmitnick
Copy link
Copy Markdown

I modification bu your code ,now can use the libusb in android 7.0,THX ;
but ,by the test in android 4.0 and 5.0 not can ok .

depau added a commit to depau-forks/libusb-compat-0.1 that referenced this pull request Dec 28, 2017
@dickens dickens force-pushed the master branch 2 times, most recently from c816259 to 8681342 Compare February 24, 2018 10:03
@hjelmn
Copy link
Copy Markdown
Member

hjelmn commented Dec 5, 2018

Could even make the old call be a helper:

static inline int libusb_wrap_fd (libusb_context *ctx, int fd, libusb_device_handle **dev_handle) {
    return libusb_wrap_sys_device (ctx, (intptr_t) fd, dev_handle);
}

Handle devices that are initialized with NULL as sysfs_dir.  For such
devices, the usbfs functions should be used instead of their sysfs
counterparts.

Signed-off-by: Vianney le Clément de Saint-Marcq <code@quartic.eu>
Slightly refactor op_open() to extract the device handle initialization
code into a new initialize_handle() function, similar to the
initialize_device() function.

Signed-off-by: Vianney le Clément de Saint-Marcq <code@quartic.eu>
Extend linux_get_device_address() to try to read the device address from
a file descriptor as a last resort, if provided.

Additionally, linux_get_device_address() will now return an error if the
path could not be parsed.

Signed-off-by: Vianney le Clément de Saint-Marcq <code@quartic.eu>
Introduce a new API function for wrapping an existing platform-specific
device handle as a libusb_device_handle.

Signed-off-by: Vianney le Clément de Saint-Marcq <code@quartic.eu>
Add support for the libusb_wrap_sys_device() API on Linux.  Because
sysfs_dir is set to NULL, only the provided file descriptor will be
used.  This is needed on some platforms (e.g., Android) where sysfs
might not be available.

Signed-off-by: Vianney le Clément de Saint-Marcq <code@quartic.eu>
On newer Android devices (5+), SELinux policies block normal processes
from reading /dev.  The consequence is that the usbfs path cannot be
found.  Rather than failing to initialize libusb in such case, fall back
to /dev/bus/usb.  This path will actually not be used, as USB devices
should be opened through the Android API, passing the file descriptor to
libusb_wrap_sys_device.

Signed-off-by: Vianney le Clément de Saint-Marcq <code@quartic.eu>
On Android, the platform API should be used to scan for and open devices
and pass file descriptors to libusb.  Newer devices (Android 5+) even
prohibit listening for hotplug events, resulting in libusb failing to
initialize without this patch.

Note that this patch effectively renders libusb useless on older devices
that do not have USB support in the platform API (anything before
Android 5).

Signed-off-by: Vianney le Clément de Saint-Marcq <code@quartic.eu>
@vianney
Copy link
Copy Markdown
Contributor Author

vianney commented Dec 5, 2018

@hjelmn I have just rebased my branch to the latest master and renamed the API function to libusb_wrap_sys_device with an intptr_t argument as per your suggestion.

The patch set is sent to the libusb-devel mailing list. I hope it will go through, as I am not registered to the list.

@hjelmn
Copy link
Copy Markdown
Member

hjelmn commented Dec 5, 2018

@vianney It will once I approve it. I will keep an eye out for the email.

@hjelmn
Copy link
Copy Markdown
Member

hjelmn commented Dec 5, 2018

@vianney Approved it on the mailing list. Will see what (if anything) other people have to say.

@larry-xmos
Copy link
Copy Markdown

I've been trying to use this, and wonder if anyone could help me understand if:

  1. There is a way to invoke a USB connection getFileDescriptor in Java without first calling USB manager requestPermission that brings up a GUI. I am looking for a solution contained on the command line.

  2. Does my C program have to be embedded in the Java app, or is the file descriptor valid across processes so I can pass it across from Java app to C program? Again, I am looking to be able to use libusb in a standalone C program.

Grimler91 added a commit to Grimler91/ttwatch that referenced this pull request Feb 15, 2020
On android we need to get a file descriptor through the android API
and then get the device handle from libusb_wrap_sys_device, instead of
through libusb_open as on a "normal" GNU/Linux system.

With these changes it is possible to use ttwatch from termux with a
command like:

```
termux-usb -r -e "ttwatch --update-gps --usb-fd" /dev/bus/usb/001/002
```

The file descriptor is then passed as the last argument to ttwatch
from termux-usb.

Further reading: libusb/libusb#242

I should refactor this, make it compatible with both android and
GNU/Linux and open a PR against the ttwatch repo.
Wandering-Consciousness pushed a commit to vfp2/libftdi that referenced this pull request May 28, 2021
libusb_wrap_sys_device was the requested name in the pull request to the libusb project for Android support:
libusb/libusb#242
@mcuee mcuee removed the pending Pending maintainer merge label Jul 4, 2021
Seneral pushed a commit to Seneral/libusb that referenced this pull request Sep 21, 2021
On Android, the platform API should be used to scan for and open devices
and pass file descriptors to libusb.  Newer devices (Android 5+) even
prohibit listening for hotplug events, resulting in libusb failing to
initialize without this patch.

Note that this patch effectively renders libusb useless on older devices
that do not have USB support in the platform API (anything before
Android 5).

Closes libusb#242

Signed-off-by: Vianney le Clément de Saint-Marcq <code@quartic.eu>
Signed-off-by: Nathan Hjelm <hjelmn@me.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.