I was exited to try this code, but unfortunately I did not have success with it on Windows 11 for listing the properties of BBC micro:bit v2.

With unmodified Pyserial I got:

device: COM8
name: COM8
description: USB Serial Device (COM8)
hwid: USB VID:PID=0D28:0204 SER=9904360249624E45004160150000002F000000009796990B LOCATION=1-1:x.1
vid: 3368
pid: 516
serial_number: 9904360249624E45004160150000002F000000009796990B
location: 1-1:x.1
manufacturer: Microsoft
product: None
interface: None

but after applying your patch I got even less information:

device: COM8
name: COM8
description: n/a
hwid: USB VID:PID=0D28:0204 SER= LOCATION=1-1:x.1
vid: 3368
pid: 516
serial_number: 
location: 1-1:x.1
manufacturer: Microsoft
product: None
interface: None

The value of the Product field I am looking for, is "BBC micro:bit CMSIS-DAP" and usbview.exe does report it like this.

I also tested it on Windows 10 and got the same result. I'm attaching my usbview report from Windows 11, in case you'd like to investigate, why I didn't get the expected result.
USBViewAll.txt

I also tested it on Windows 10 and got the same result. I'm attaching my usbview report from Windows 11, in case you'd like to investigate, why I didn't get the expected result.
USBViewAll.txt

By the way, are you using the latest commit (c090f45667c68b7a7259a38de7a59b49b016f851)?

Since the earliest commit I forgot about the composite device. the latest commit should fix these issues.

By the way, are you using the latest commit (c090f45)?

Yes, I was using https://github.com/pyserial/pyserial/blob/c090f45667c68b7a7259a38de7a59b49b016f851/serial/tools/list_ports_windows.py and https://github.com/pyserial/pyserial/blob/c090f45667c68b7a7259a38de7a59b49b016f851/serial/win32.py for patching my Pyserial.

I'll see later, whether the patched version works with my other devices.

I now tested your patch with more devices (SAMD-s from Adafruit, Rasperry Pi Pico, several ESP32-S2 and S3 devices, some plain ESP32 devices with CP and CH UART adapters, an ESP8266 with CP2104 UART adapter, a NRF52 device) and found out that with most of these it worked nicely.

The few, that showed proper Product (and Manufacturer) in usbview.exe, but not with your patch, were:

• Lilygo/TTGO T-Micro 32 ESP32 device (expected product value: "CP2104 USB to UART Bridge Controller", actual: None)
• Lolin C3 mini ESP32-C3 (expected product value: "USB JTAG/serial debug unit", actual: None). Curiously, your patch does enhance the value of the Manufacturer field, by changing it from "Microsoft" to "Espressif".
• M5Stack AtomU and AtomLite, ESP32 devices with FT232 or FT234 serial converter (expected value "M5Stack")
• Sparkfun ESP32 Thing (FT X-Series serial converter)
• BBC micro:bit v2. Besides not showing proper Product and Manufacturer fields, your patch turned serial_number field from "99063602000528203977597D73765584000000006E052820" to "" (empty string)

Please let me know, if you need usbview report for any of these!

I now tested your patch with more devices (SAMD-s from Adafruit, Rasperry Pi Pico, several ESP32-S2 and S3 devices, some plain ESP32 devices with CP and CH UART adapters, an ESP8266 with CP2104 UART adapter, a NRF52 device) and found out that with most of these it worked nicely.

The few, that showed proper Product (and Manufacturer) in usbview.exe, but not with your patch, were:

• Lilygo/TTGO T-Micro 32 ESP32 device (expected product value: "CP2104 USB to UART Bridge Controller", actual: None)
• Lolin C3 mini ESP32-C3 (expected product value: "USB JTAG/serial debug unit", actual: None). Curiously, your patch does enhance the value of the Manufacturer field, by changing it from "Microsoft" to "Espressif".
• M5Stack AtomU and AtomLite, ESP32 devices with FT232 or FT234 serial converter (expected value "M5Stack")
• Sparkfun ESP32 Thing (FT X-Series serial converter)
• BBC micro:bit v2. Besides not showing proper Product and Manufacturer fields, your patch turned serial_number field from "99063602000528203977597D73765584000000006E052820" to "" (empty string)

Please let me know, if you need usbview report for any of these!

I read the USBView report you provided for the BBC micro:bit v2. It looks like my patch misidentifies the composite device as a noncomposite device. This will result in an error similar to the one below.

Reads the string description as a composite device:

device: COM4
name: COM4
hwid: USB VID:PID=303A:4001 SER=123456 LOCATION=1-4:1.0
vid: 303a
pid: 4001
serial_number: 123456
location: 1-4:1.0
manufacturer: Espressif Systems
product: Espressif Device

Reads the string description as a noncomposite device:

device: COM4
name: COM4
hwid: USB VID:PID=303A:4001 SER=123456 LOCATION=1-4:x.0
vid: 303a
pid: 4001
serial_number: 
location: 1-4:x.0
manufacturer: Microsoft

Could you provide me with reports from the USB Device Tree Viewer for these devices? It could provide me with more information to solve the problem. Thanks a lot.

Here come the reports from USB Device Tree Viewer:

• M5Stack Atom U.txt
• M5Stack AtomLite.txt
• Sparfun Thing 32.txt
• BBC MicroBit.txt
• Lilygo_TTGO T-Micro 32 ESP32.txt
• Lolin C3 mini ESP32-C3.txt

Thanks a lot! After your last commit, pyserial reports proper Product, Manufacturer and Serial number for my BBC micro:bit. I'll test with my other boards tomorrow.

Great job!

With all my boards I now get proper values for Manufacurer, Product and Serial number. For getting proper Interface, I'm combining your solution with #571.

Thank you!

With all my boards I now get proper values for Manufacurer, Product and Serial number. For getting proper Interface, I'm combining your solution with #571.

Thank you!

I refactored the code and added comments to enhance readability. Thank you for your testing, I think this patch is ready to be merged now.

With all my boards I now get proper values for Manufacurer, Product and Serial number. For getting proper Interface, I'm combining your solution with #571.

Thank you!

I noticed that you want to merge with #571. I've added the feature to read the interface (via CM_Get_DevNode_Property instead of SetupDiGetDeviceProperty) in the latest commit. So merging #571 may cause a conflict.

I put the old code for getting usb info into get_usb_info_from_device_property() as a fallback option. I suggest you put the change #571 at the end of the get_usb_info_from_device_property() function. This avoids conflicts caused by modifying the iterate_comports() function at the same time.

I've added the feature to read the interface

That's great! Somehow I missed it. I tested it and it works beautifully. I don't need #571 anymore.

It is now possible to get the correct interface field on most devices.

Unfortunately does not seem to work well on devices with multiple serial interfaces, like FT4232H that has 4 UARTs.

Unpatched result:

device: COM12
name: COM12
description: USB Serial Port (COM12)
hwid: USB VID:PID=0403:6011 SER=6
vid: 1027
pid: 24593
serial_number: 6
location: None
manufacturer: FTDI
product: None
interface: None

Patched result:

name: USB Serial Port (COM12)
description: Quad RS232-HS - Quad RS232-HS
hwid: USB VID:PID=0403:6011 LOCATION=1-1:1.0
vid: 1027
pid: 24593
serial_number: None
location: 1-1:1.0
manufacturer: FTDI
product: Quad RS232-HS
interface: Quad RS232-HS

Location is the same for all COM Ports, e.g.

device: COM11
name: USB Serial Port (COM11)
description: Quad RS232-HS - Quad RS232-HS
hwid: USB VID:PID=0403:6011 LOCATION=1-1:1.0
vid: 1027
pid: 24593
serial_number: None
location: 1-1:1.0
manufacturer: FTDI
product: Quad RS232-HS
interface: Quad RS232-HS

So there is no way to distinguish between the COM ports by location. For linux this seems to have been solved in PR #141

Here's an example of szHardwareID_str: FTDIBUS\VID_0403+PID_6011+6&1AD5555C&0&1&1\0000

Unfortunately does not seem to work well on devices with multiple serial interfaces, like FT4232H that has 4 UARTs.

Unpatched result:

device: COM12
name: COM12
description: USB Serial Port (COM12)
hwid: USB VID:PID=0403:6011 SER=6
vid: 1027
pid: 24593
serial_number: 6
location: None
manufacturer: FTDI
product: None
interface: None

Patched result:

name: USB Serial Port (COM12)
description: Quad RS232-HS - Quad RS232-HS
hwid: USB VID:PID=0403:6011 LOCATION=1-1:1.0
vid: 1027
pid: 24593
serial_number: None
location: 1-1:1.0
manufacturer: FTDI
product: Quad RS232-HS
interface: Quad RS232-HS

Location is the same for all COM Ports, e.g.

device: COM11
name: USB Serial Port (COM11)
description: Quad RS232-HS - Quad RS232-HS
hwid: USB VID:PID=0403:6011 LOCATION=1-1:1.0
vid: 1027
pid: 24593
serial_number: None
location: 1-1:1.0
manufacturer: FTDI
product: Quad RS232-HS
interface: Quad RS232-HS

So there is no way to distinguish between the COM ports by location. For linux this seems to have been solved in PR #141

Here's an example of szHardwareID_str: FTDIBUS\VID_0403+PID_6011+6&1AD5555C&0&1&1\0000

Since the FTDI driver hides the location information, we had to do some dirty work to get the interface number. I referenced libusbp to parse the hardware id to get the interface number. But apparently it doesn't work.

Could you provide me with reports from the USB Device Tree Viewer for your devices?

Hey @chinaheyu, thanks a lot for reply and the reference to libusbp. Sure, here the reports from USB Device Tree Viewer (just the first 2 UARTs, as the others are accordingly):

After the last change most of my devices seem to work same as before, but I get following exception with 3 of my ESP32-C3 devices:

  File "C:\python_stuff\thonny\thonny\vendored_libs\serial\tools\list_ports_windows.py", line 1182, in comports
    return list(iterate_comports())
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\python_stuff\thonny\thonny\vendored_libs\serial\tools\list_ports_windows.py", line 1156, in iterate_comports
    usb_info = port_device.get_usb_info(device_registry)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\python_stuff\thonny\thonny\vendored_libs\serial\tools\list_ports_windows.py", line 698, in get_usb_info
    language_id = hub_io.suggest_language_id(usb_hub_port)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\python_stuff\thonny\thonny\vendored_libs\serial\tools\list_ports_windows.py", line 870, in suggest_language_id
    return available_languages[0]
           ~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

Here is the UsbTreeview report for one of them: SeeedXiaoESP32C3.txt

The weird thing is that I get this exception only sometimes, usually the first time I call comports() after plugging in the device -- the next time I call comports() it usually works without problems (but not always).

@chinaheyu, after latest changes I don't get these errors anymore, but I now discovered that with any of these problematic ESP32-C3 devices plugged in, I now occasionally get long delays before comports() returns.

It looks like there are two methods, which may cause the delay:

• USBHubDeviceIOControl.request_supported_languages
• USBHubDeviceIOControl.request_usb_string_description

The first call of either method gets stuck in DeviceIoControl for 24-30 seconds. The next calls return quickly. For example, if I skip calling DeviceIoControl in request_supported_languages, the first request_usb_string_description becomes slow, etc.

The problem is present also in your older commits, e.g. aivarannamaa@bdab39f#diff-f9966444f1fafc17f351672a463bdcff7833ad6c14a68d9432329a458d1608b0

The next call to comports() is quick, unless in the meanwhile I've connected to or disconnected from a serial port in another process.

Unfortunately I can't create a simple script for reproducing this. The earlier I call comports() in my app, the less likely the delay becomes.

Can you recommend some tweaks I could try to learn more about this problem?

Adding to my last comment -- the DeviceIoControl call in request_usb_connection_info never becomes slow, even though it gets called before request_supported_languages and request_usb_string_description.

@aivarannamaa Thank you very much for your feedback.

This is a very peculiar issue that I haven't encountered before. Overall, the problem should be related to the device (ESP32-32 C3) as it seems not to respond with its string description promptly. Subsequent requests are fast because DeviceIoControl caches the result.

After receiving your feedback, I purchased a Seeed Xiao ESP32 C3 board, which should arrive today. I will do my best to resolve this peculiar issue.

Additionally, I used the same API as in USBView, so did you encounter this issue when using USBView?

@waterfallwhitebread Thank you for providing the screenshot. The latest patch should be able to correctly retrieve the location field of the FT4232H chip.

@aivarannamaa You are right, if another process opens and closes the serial port, it blocks DeviceIOControl. This phenomenon only occurs on ESP32 C3 devices.

I also found an issue on StackOverflow that may be relevant: https://stackoverflow.com/questions/6749681/freeze-on-serialport-open-deviceiocontrol-getcommstate-with-usbser-sys

I can't fix this, but we can cache the usb info to avoid calling DeviceIOControl every time. The latest commit should be helpful for your problem.

@chinaheyu, thanks! Curiously, now I can't reproduce this problem anymore.

Any chance it gets merged into pyserial ?
This PR solved a problem I had on windows to differentiate a Nordic PPK2 from a Nordic Dongle as they appeared the same with vanilla pyserial.

Thanks for this @chinaheyu !

Hi! Is there anything that Adafruit can do to move inclusion of this patch forward? It would be very helpful to us.

I have a report that this method doesn't work with Windows 7 (keirf/greaseweazle#472). Is that unexpected?

It should theoretically support Windows 7 (though I haven't tested it), as Cfgmgr32 API has been available since Windows 2000. I plan to test it on Windows 7 next to see if I can reproduce the issue. Thank you @keirf .

I have a report that this method doesn't work with Windows 7 (keirf/greaseweazle#472). Is that unexpected?

Sorry, I can't reproduce the problem. The patch works fine in Windows 7. Can you provide more detailed python version and system information?

The system image used above: Windows 7 Ultimate with Service Pack 1 (7601.24214) (x64)

Thanks for testing! My user's report is on Microsoft Windows [Version 6.1.7601] and running 32-bit Python 3.8.

I will get Hyper-V running and try a Win7 VM myself.

At the end of the day though, Win7 is long EOL, and soon Python 3.8 will be too. And that's the last Python version to support Win7. So the problem will solve itself soon enough. :)

Had some difficulty installing python 3.8, so used python 3.6, but the result should be the same.

32-bit Windows 7 and 32-bit Python

Okay, I went to the hassle of installing Win7 in a VM to look at this, and what a pain that was! But I have reproduced the issue. I think it must depend on the device or its driver. Specifically, my "Greaseweazle" device enumerates as a generic CDC ACM device, and on Windows 7/8 requires use of Zadig to associate the driver usbser.sys (Zadig calls this "USB Serial (CDC)").

Curiously, the default pyserial list_port_windows.py finds the correct product string for this device on Windows 7 ("Greaseweazle"), which it fails to do on Windows 10/11. And on Win7 your list_port_windows.py fails to list it at all. Perhaps it gets filtered out somewhere?

By the way, as a baseline I tested a PL2303 as regular proprietary USB serial device. This worked fine on Windows 7 with your implementation.

I also confirmed this issue on Windows 8.1. This is perhaps more related to Zadig, or pre-Win10 usbser.sys.

@keirf Bug maybe fixed. Could you try the latest commit (91177bd)?

@keirf Bug maybe fixed. Could you try the latest commit (91177bd)?

It works great! Thanks for the very quick fix. I have already ported it into Greaseweazle tools and I will have the original reporter also confirm the fix.

Thanks again!

Is there any update on this? This functionality (corrected device naming on windows) is actually addressed in a couple of the pending PRs, but does not seem to ever get completed. It would be great if one of these could reach completion as this functionality would be very helpful for a number of different of implementations.

I have a report of failure with empty port name (keirf/greaseweazle#549) with the following fix: keirf/greaseweazle#550

Could you please consider this fix, or equivalent, for this upstream PR?

I have a report of failure with empty port name (keirf/greaseweazle#549) with the following fix: keirf/greaseweazle#550

Could you please consider this fix, or equivalent, for this upstream PR?

It seems that some serial devices are not assigned a COMx-style port name. I'm not sure what's causing this, but simply skipping these devices can prevent the program from crashing. In theory, even without a port name, we can still open these serial devices via their interface path, which might serve as an alternative when the port name is None (possibly a better solution than just skipping them). That said, keirf/greaseweazle#550 is correct and necessary. Many thanks for the fix.

Hello,
Is there any progress on the issue?
The branch solves my issue with CP210x on Windows 11 and product field, so interested in any plan here.

Thanks.

Hello,

This also solved my issue on Windows 11 and interface field, with a USB composite (2 CDC ACM) device.

Thank you.