I was exited to try this code, but unfortunately I did not have success with it on Windows 11 for listing the properties of BBC micro:bit v2.

With unmodified Pyserial I got:

device: COM8
name: COM8
description: USB Serial Device (COM8)
hwid: USB VID:PID=0D28:0204 SER=9904360249624E45004160150000002F000000009796990B LOCATION=1-1:x.1
vid: 3368
pid: 516
serial_number: 9904360249624E45004160150000002F000000009796990B
location: 1-1:x.1
manufacturer: Microsoft
product: None
interface: None

but after applying your patch I got even less information:

device: COM8
name: COM8
description: n/a
hwid: USB VID:PID=0D28:0204 SER= LOCATION=1-1:x.1
vid: 3368
pid: 516
serial_number: 
location: 1-1:x.1
manufacturer: Microsoft
product: None
interface: None

The value of the Product field I am looking for, is "BBC micro:bit CMSIS-DAP" and usbview.exe does report it like this.

I also tested it on Windows 10 and got the same result. I'm attaching my usbview report from Windows 11, in case you'd like to investigate, why I didn't get the expected result.
USBViewAll.txt

I also tested it on Windows 10 and got the same result. I'm attaching my usbview report from Windows 11, in case you'd like to investigate, why I didn't get the expected result.
USBViewAll.txt

By the way, are you using the latest commit (c090f45667c68b7a7259a38de7a59b49b016f851)?

Since the earliest commit I forgot about the composite device. the latest commit should fix these issues.

By the way, are you using the latest commit (c090f45)?

Yes, I was using https://github.com/pyserial/pyserial/blob/c090f45667c68b7a7259a38de7a59b49b016f851/serial/tools/list_ports_windows.py and https://github.com/pyserial/pyserial/blob/c090f45667c68b7a7259a38de7a59b49b016f851/serial/win32.py for patching my Pyserial.

I'll see later, whether the patched version works with my other devices.

I now tested your patch with more devices (SAMD-s from Adafruit, Rasperry Pi Pico, several ESP32-S2 and S3 devices, some plain ESP32 devices with CP and CH UART adapters, an ESP8266 with CP2104 UART adapter, a NRF52 device) and found out that with most of these it worked nicely.

The few, that showed proper Product (and Manufacturer) in usbview.exe, but not with your patch, were:

• Lilygo/TTGO T-Micro 32 ESP32 device (expected product value: "CP2104 USB to UART Bridge Controller", actual: None)
• Lolin C3 mini ESP32-C3 (expected product value: "USB JTAG/serial debug unit", actual: None). Curiously, your patch does enhance the value of the Manufacturer field, by changing it from "Microsoft" to "Espressif".
• M5Stack AtomU and AtomLite, ESP32 devices with FT232 or FT234 serial converter (expected value "M5Stack")
• Sparkfun ESP32 Thing (FT X-Series serial converter)
• BBC micro:bit v2. Besides not showing proper Product and Manufacturer fields, your patch turned serial_number field from "99063602000528203977597D73765584000000006E052820" to "" (empty string)

Please let me know, if you need usbview report for any of these!

I now tested your patch with more devices (SAMD-s from Adafruit, Rasperry Pi Pico, several ESP32-S2 and S3 devices, some plain ESP32 devices with CP and CH UART adapters, an ESP8266 with CP2104 UART adapter, a NRF52 device) and found out that with most of these it worked nicely.

The few, that showed proper Product (and Manufacturer) in usbview.exe, but not with your patch, were:

• Lilygo/TTGO T-Micro 32 ESP32 device (expected product value: "CP2104 USB to UART Bridge Controller", actual: None)
• Lolin C3 mini ESP32-C3 (expected product value: "USB JTAG/serial debug unit", actual: None). Curiously, your patch does enhance the value of the Manufacturer field, by changing it from "Microsoft" to "Espressif".
• M5Stack AtomU and AtomLite, ESP32 devices with FT232 or FT234 serial converter (expected value "M5Stack")
• Sparkfun ESP32 Thing (FT X-Series serial converter)
• BBC micro:bit v2. Besides not showing proper Product and Manufacturer fields, your patch turned serial_number field from "99063602000528203977597D73765584000000006E052820" to "" (empty string)

Please let me know, if you need usbview report for any of these!

I read the USBView report you provided for the BBC micro:bit v2. It looks like my patch misidentifies the composite device as a noncomposite device. This will result in an error similar to the one below.

Reads the string description as a composite device:

device: COM4
name: COM4
hwid: USB VID:PID=303A:4001 SER=123456 LOCATION=1-4:1.0
vid: 303a
pid: 4001
serial_number: 123456
location: 1-4:1.0
manufacturer: Espressif Systems
product: Espressif Device

Reads the string description as a noncomposite device:

device: COM4
name: COM4
hwid: USB VID:PID=303A:4001 SER=123456 LOCATION=1-4:x.0
vid: 303a
pid: 4001
serial_number: 
location: 1-4:x.0
manufacturer: Microsoft

Could you provide me with reports from the USB Device Tree Viewer for these devices? It could provide me with more information to solve the problem. Thanks a lot.

Here come the reports from USB Device Tree Viewer:

• M5Stack Atom U.txt
• M5Stack AtomLite.txt
• Sparfun Thing 32.txt
• BBC MicroBit.txt
• Lilygo_TTGO T-Micro 32 ESP32.txt
• Lolin C3 mini ESP32-C3.txt

Thanks a lot! After your last commit, pyserial reports proper Product, Manufacturer and Serial number for my BBC micro:bit. I'll test with my other boards tomorrow.

Great job!

With all my boards I now get proper values for Manufacurer, Product and Serial number. For getting proper Interface, I'm combining your solution with #571.

Thank you!

With all my boards I now get proper values for Manufacurer, Product and Serial number. For getting proper Interface, I'm combining your solution with #571.

Thank you!

I refactored the code and added comments to enhance readability. Thank you for your testing, I think this patch is ready to be merged now.

With all my boards I now get proper values for Manufacurer, Product and Serial number. For getting proper Interface, I'm combining your solution with #571.

Thank you!

I noticed that you want to merge with #571. I've added the feature to read the interface (via CM_Get_DevNode_Property instead of SetupDiGetDeviceProperty) in the latest commit. So merging #571 may cause a conflict.

I put the old code for getting usb info into get_usb_info_from_device_property() as a fallback option. I suggest you put the change #571 at the end of the get_usb_info_from_device_property() function. This avoids conflicts caused by modifying the iterate_comports() function at the same time.

I've added the feature to read the interface

That's great! Somehow I missed it. I tested it and it works beautifully. I don't need #571 anymore.

It is now possible to get the correct interface field on most devices.

Unfortunately does not seem to work well on devices with multiple serial interfaces, like FT4232H that has 4 UARTs.

Unpatched result:

device: COM12
name: COM12
description: USB Serial Port (COM12)
hwid: USB VID:PID=0403:6011 SER=6
vid: 1027
pid: 24593
serial_number: 6
location: None
manufacturer: FTDI
product: None
interface: None

Patched result:

name: USB Serial Port (COM12)
description: Quad RS232-HS - Quad RS232-HS
hwid: USB VID:PID=0403:6011 LOCATION=1-1:1.0
vid: 1027
pid: 24593
serial_number: None
location: 1-1:1.0
manufacturer: FTDI
product: Quad RS232-HS
interface: Quad RS232-HS

Location is the same for all COM Ports, e.g.

device: COM11
name: USB Serial Port (COM11)
description: Quad RS232-HS - Quad RS232-HS
hwid: USB VID:PID=0403:6011 LOCATION=1-1:1.0
vid: 1027
pid: 24593
serial_number: None
location: 1-1:1.0
manufacturer: FTDI
product: Quad RS232-HS
interface: Quad RS232-HS

So there is no way to distinguish between the COM ports by location. For linux this seems to have been solved in PR #141

Here's an example of szHardwareID_str: FTDIBUS\VID_0403+PID_6011+6&1AD5555C&0&1&1\0000

Unfortunately does not seem to work well on devices with multiple serial interfaces, like FT4232H that has 4 UARTs.

Unpatched result:

device: COM12
name: COM12
description: USB Serial Port (COM12)
hwid: USB VID:PID=0403:6011 SER=6
vid: 1027
pid: 24593
serial_number: 6
location: None
manufacturer: FTDI
product: None
interface: None

Patched result:

name: USB Serial Port (COM12)
description: Quad RS232-HS - Quad RS232-HS
hwid: USB VID:PID=0403:6011 LOCATION=1-1:1.0
vid: 1027
pid: 24593
serial_number: None
location: 1-1:1.0
manufacturer: FTDI
product: Quad RS232-HS
interface: Quad RS232-HS

Location is the same for all COM Ports, e.g.

device: COM11
name: USB Serial Port (COM11)
description: Quad RS232-HS - Quad RS232-HS
hwid: USB VID:PID=0403:6011 LOCATION=1-1:1.0
vid: 1027
pid: 24593
serial_number: None
location: 1-1:1.0
manufacturer: FTDI
product: Quad RS232-HS
interface: Quad RS232-HS

So there is no way to distinguish between the COM ports by location. For linux this seems to have been solved in PR #141

Here's an example of szHardwareID_str: FTDIBUS\VID_0403+PID_6011+6&1AD5555C&0&1&1\0000

Since the FTDI driver hides the location information, we had to do some dirty work to get the interface number. I referenced libusbp to parse the hardware id to get the interface number. But apparently it doesn't work.

Could you provide me with reports from the USB Device Tree Viewer for your devices?

Hey @chinaheyu, thanks a lot for reply and the reference to libusbp. Sure, here the reports from USB Device Tree Viewer (just the first 2 UARTs, as the others are accordingly):

After the last change most of my devices seem to work same as before, but I get following exception with 3 of my ESP32-C3 devices:

  File "C:\python_stuff\thonny\thonny\vendored_libs\serial\tools\list_ports_windows.py", line 1182, in comports
    return list(iterate_comports())
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\python_stuff\thonny\thonny\vendored_libs\serial\tools\list_ports_windows.py", line 1156, in iterate_comports
    usb_info = port_device.get_usb_info(device_registry)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\python_stuff\thonny\thonny\vendored_libs\serial\tools\list_ports_windows.py", line 698, in get_usb_info
    language_id = hub_io.suggest_language_id(usb_hub_port)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\python_stuff\thonny\thonny\vendored_libs\serial\tools\list_ports_windows.py", line 870, in suggest_language_id
    return available_languages[0]
           ~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

Here is the UsbTreeview report for one of them: SeeedXiaoESP32C3.txt

The weird thing is that I get this exception only sometimes, usually the first time I call comports() after plugging in the device -- the next time I call comports() it usually works without problems (but not always).

@chinaheyu, after latest changes I don't get these errors anymore, but I now discovered that with any of these problematic ESP32-C3 devices plugged in, I now occasionally get long delays before comports() returns.

It looks like there are two methods, which may cause the delay:

• USBHubDeviceIOControl.request_supported_languages
• USBHubDeviceIOControl.request_usb_string_description

The first call of either method gets stuck in DeviceIoControl for 24-30 seconds. The next calls return quickly. For example, if I skip calling DeviceIoControl in request_supported_languages, the first request_usb_string_description becomes slow, etc.

The problem is present also in your older commits, e.g. aivarannamaa@bdab39f#diff-f9966444f1fafc17f351672a463bdcff7833ad6c14a68d9432329a458d1608b0

The next call to comports() is quick, unless in the meanwhile I've connected to or disconnected from a serial port in another process.

Unfortunately I can't create a simple script for reproducing this. The earlier I call comports() in my app, the less likely the delay becomes.

Can you recommend some tweaks I could try to learn more about this problem?

Adding to my last comment -- the DeviceIoControl call in request_usb_connection_info never becomes slow, even though it gets called before request_supported_languages and request_usb_string_description.

@aivarannamaa Thank you very much for your feedback.

This is a very peculiar issue that I haven't encountered before. Overall, the problem should be related to the device (ESP32-32 C3) as it seems not to respond with its string description promptly. Subsequent requests are fast because DeviceIoControl caches the result.

After receiving your feedback, I purchased a Seeed Xiao ESP32 C3 board, which should arrive today. I will do my best to resolve this peculiar issue.

Additionally, I used the same API as in USBView, so did you encounter this issue when using USBView?

@waterfallwhitebread Thank you for providing the screenshot. The latest patch should be able to correctly retrieve the location field of the FT4232H chip.

@keirf Bug maybe fixed. Could you try the latest commit (91177bd)?

@keirf Bug maybe fixed. Could you try the latest commit (91177bd)?

It works great! Thanks for the very quick fix. I have already ported it into Greaseweazle tools and I will have the original reporter also confirm the fix.

Thanks again!

Is there any update on this? This functionality (corrected device naming on windows) is actually addressed in a couple of the pending PRs, but does not seem to ever get completed. It would be great if one of these could reach completion as this functionality would be very helpful for a number of different of implementations.

I have a report of failure with empty port name (keirf/greaseweazle#549) with the following fix: keirf/greaseweazle#550

Could you please consider this fix, or equivalent, for this upstream PR?

I have a report of failure with empty port name (keirf/greaseweazle#549) with the following fix: keirf/greaseweazle#550

Could you please consider this fix, or equivalent, for this upstream PR?

It seems that some serial devices are not assigned a COMx-style port name. I'm not sure what's causing this, but simply skipping these devices can prevent the program from crashing. In theory, even without a port name, we can still open these serial devices via their interface path, which might serve as an alternative when the port name is None (possibly a better solution than just skipping them). That said, keirf/greaseweazle#550 is correct and necessary. Many thanks for the fix.

Hello,
Is there any progress on the issue?
The branch solves my issue with CP210x on Windows 11 and product field, so interested in any plan here.

Thanks.

Hello,

This also solved my issue on Windows 11 and interface field, with a USB composite (2 CDC ACM) device.

Thank you.

Hi,

There is an issue in serial number.

Without patch:

VID:PID: E0D0:0002
Product: None
Serial: 0X829463B1-0X2A27980E
Interface: None
location: None
device: COM16
USB VID:PID=E0D0:0002 SER=0X829463B1-0X2A27980E

Linux

VID:PID: E0D0:0002
Product: mydevice
Serial: 0x829463b1-0x2a27980e
Interface: control (CIC)
location: 1-1:1.0
device: /dev/ttyACM0
USB VID:PID=E0D0:0002 SER=0x829463b1-0x2a27980e LOCATION=1-1:1.0

With patch:

VID:PID: E0D0:0002
Product: mydevice
Serial: 0x829463b1-0x2a27980e杩oꮐ翸臠咋Ơ䒀孒翹Ĝ孒翹齰翸����
Interface: control (CIC)
location: 2-6.2.3:1.0
device: COM16
USB VID:PID=E0D0:0002 SER=0x829463b1-0x2a27980e杩oꮐ翸臠咋Ơ䒀孒翹Ĝ孒翹齰翸���� LOCATION=2-6.2.3:1.0

Notice the serial number junk.

Thanks,
Alon

Hey, @alonbl,

During decoding, we did not use decoding APIs like WideCharToMultiByte. We assumed the device returned the correct length and then converted it to a Python string using ctypes.wstring_at. The code looks something like this.

# Convert wstring to python str.
# Very few devices will have an extra null at the end of string, strip it.
return ctypes.wstring_at(description.contents.bString, description.contents.bLength // 2 - 1).rstrip('\0')

Therefore, we are curious whether the device returned an incorrect bLength.

Could you download usbtreeview and show me the serial number in the string description section? It should look something like this:

             ------ String Descriptor 3 ------
bLength                  : 0x24 (36 bytes)
bDescriptorType          : 0x03 (String Descriptor)
Language 0x0409          : "20120501030900000"
Data (HexDump)           : 24 03 32 00 30 00 31 00 32 00 30 00 35 00 30 00   $.2.0.1.2.0.5.0.
                           31 00 30 00 33 00 30 00 39 00 30 00 30 00 30 00   1.0.3.0.9.0.0.0.
                           30 00 30 00                                       0.0.

Thanks.

Hi @lukehugh,

This is interesting!

I get the following:

             ------ String Descriptor 2 ------
bLength                  : 0x1A (26 bytes)
bDescriptorType          : 0x03 (String Descriptor)
Language 0x0409          : "xxxxxxxxxxxx"
             ------ String Descriptor 3 ------
bLength                  : 0x82 (130 bytes)
bDescriptorType          : 0x03 (String Descriptor)
Language 0x0409          : "829463b1-2a27980e°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°"  *!*ERROR  contains 47 NULL characters  *!*CAUTION  Windows uses only up to 63 characters from iSerial for building Device IDs of DISK devices

Update: I found the hex dump... and now I can see that the host indeed does not receive correct size, I will investigate, thank you for your help!

             ------ String Descriptor 3 ------
bLength                  : 0x82 (130 bytes)
bDescriptorType          : 0x03 (String Descriptor)
Language 0x0409          : "829463b1-2a27980e°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°"  *!*ERROR  contains 47 NULL characters  *!*CAUTION  Windows uses only up to 63 characters from iSerial for building Device IDs of DISK devices
Data (HexDump)           : 82 03 38 00 32 00 39 00 34 00 36 00 33 00 62 00   ..8.2.9.4.6.3.b.
                           31 00 2D 00 32 00 61 00 32 00 37 00 39 00 38 00   1.-.2.a.2.7.9.8.
                           30 00 65 00 00 00 00 00 00 00 00 00 00 00 00 00   0.e.............
                           00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
                           00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
                           00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
                           00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
                           00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
                           00 00                                             ..

Thanks,
Alon

Hi @lukehugh ,

When using Linux and libusb, I get:

iSerialNumber index: 3
Raw serial string descriptor bytes:
24 03 38 00 32 00 39 00 34 00 36 00 33 00 62 00 31 00 2d 00 32 00 61 00 32 00 37 00 39 00 38 00 30 00 65 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

I do not understand why in windows we get 0x82 which is the buffer size and in Linux we get 0x24 which is the string size. Maybe there usbview is not showing the correct values?

Thanks,

@alonbl Although I can't do anything about what your device returns on Windows. Theoretically, if the string contains extra NULL characters, we should strip them (and indeed, the code does this). Perhaps your Windows PC uses some special code page that doesn't interpret NULL as '\0', so I explicitly used UTF-16LE encoding in the latest commit (00961ac). This might help you.

Hi @lukehugh,
I found that windows overwrites the size byte as if it was sent from device.
In both Linux and MacOS this is not happening.
I modified the device to adjust the capacity as well, so it is working with this implementation as well.
There should be no extra NULL in USB descriptor strings as far as I know.
I checked the last commit and I can confirm it is working.
What are the chances this get merge? It has been two years since you submitted.
Thanks,
Alon

hello,
there will be joy on my side once this is eventually merged and released (when?), so the serial returns finally something sensible.
if you go to wait for new year, I'll be back at work and can quickly test it with our internal USB device, if it behaves nicelly too on Windows and Debian at the same time.
thanks.

Jaroslav

Regarding whether this PR will be merged, I’d like to give a unified response here.

This patch originated from one of my personal side projects. At the time, I was struggling to reliably obtain the correct product string (since registering a USB VID is expensive, we wanted to differentiate devices via the product string instead). Later, I discovered USBView and learned how to retrieve proper USB descriptors using DeviceIoControl. To achieve a correct product string, we replaced the outdated setupapi with cfgmgr32, and added a thin wrapper around the Device Node and DeviceIoControl. That resulted in my first working version. While the implementation was admittedly minimal, it was sufficient to solve my problem.

Considering that other users in the pyserial community might have similar needs—and hoping this approach could benefit from broader validation—we decided to submit this PR. With the help of several enthusiastic contributors, it gradually evolved. That said, I am not a pyserial maintainer, so I am not in a position to decide whether this PR should ultimately be merged. Unfortunately, around that time pyserial also appeared to be unmaintained. As a result, I chose to make the patch as self-contained as possible, so users could simply copy the file into their own projects as a workaround. In fact, the code in list_ports_windows.py only depends on list_ports_common.py, and even that dependency could be removed if necessary.

Today, I’m happy to see that pyserial is active again and gradually receiving new commits. However, I remain uncertain about the future of this PR. From my perspective, the original problem has already been solved; for the community, integrating this patch into other projects is relatively straightforward; but for pyserial itself, merging this PR could introduce breaking changes. In any case, I would very much appreciate hearing from the maintainers. @dpgeorge, @kurtmckee.

Thanks everyone for the attention and discussion.

Hi @lukehugh, thanks for the ping. I'm also not a pyserial maintainer.

I've been holding off on reviewing this PR for only one reason: I'm waiting for #838 to merge. #838 introduces a command to generate a "test asset", which is a JSON file that aims to contain all of the available information for a given device (or devices, if needed) on a given system. That's useful both for debugging detection issues (like what @alonbl reported with trailing null characters) as well as for mocking system calls to confirm that pyserial is handling the devices properly.

#838 introduces such files for sysfs-based operating systems (like Linux), but I've intended to expand the command to support Windows once the PR merges, so that I could ask @alonbl and yourself to please generate the test assets for the devices you're working with. With those test asset files in-hand, as well as files from my own devices on Windows, I've intended to build out the Windows side of the test suite.

The Windows code in pyserial is completely untested, and your PR is a significant contribution that inherits that complete lack of testing. I've therefore been waiting to dive into a review of the PR until I can build out some level of device mocking and testing to better understand the before-and-after behavior of your PR.

I hope this explanation is helpful for understanding why I haven't reviewed the PR yet.

Hello @kurtmckee ,

There is always a question of CI and hardware tests....

Do we gain value from simulating hardware for synthetic tests or we invest large amount of resources and eventually we do not test the real-world scenarios.

In case of sysfs, it is quite simple as the sysfs abstraction is plain files, so you could commit the relevant sysfs as-is, and only have a SYSFS_ROOT environment variable (for example) to scan the custom directory even without monkey patching the code... So you can have one or two tests for sysfs to make sure we read it correctly... this is not all-on-one-multi-platform agenda, just a simple unit test for sysfs.

In case of windows, it is more difficult, as Windows APIs are much more complex. However, these are static, and decent manual tests of the low-level parts should be sufficient.

For sure, having CI or not should not block us from reviewing code...

Regards,
Alon

For sure, having CI or not should not block us from reviewing code...

You are correct. I look forward to seeing your review of the code in this PR!