The AIA specification says Only naturally aligned 32-bit simple reads and writes are supported within an interrupt file’s memory region., which seems to be okay because the PCI/PCIe spec defines MSIs as "DWORD memory write" (DWORD here is defined as on x86 PC -- 32-bit); however, either PCI and PCIe has DWORD as the smallest transfer size, sub-DWORD writes are described with extra byte enable, and it's not clear that MSI write should have a byte enable of 0b1111 (i.e. all bytes enabled).
SpacemiT K3 is a SoC implementing AIA, and its IMSIC follows the rule above by ignoring 16-bit writes to seteipnum_le register (no interrupts are triggered for such writes), and when attaching a PCIe card with VID 0x1002 and PID 0x6779 (AMD/ATI CAICOS) to it, the system never receives any interrupts from this card.
The following experiment redirecting MSI to on-chip SRAM shows that the MSI write request from this card is 16-bit:
(ssh)root@k3picoitx [ ~ ] # busybox devmem 0xc0800000 # check the value of the fake doorbell at SRAM
0x12345678
(ssh)root@k3picoitx [ ~ ] # busybox devmem 0xe0540000 w 0x000c # write the payload to the original MSI address to trigger interrupt handler for clearing pending interrupts on the card
(ssh)root@k3picoitx [ ~ ] # busybox devmem 0xc0800000 # check what's overriden by the MSI
0x1234000C
In contrary, a working card's behavior is like below (plugged at the same slot and got allocated the same MSI by the system):
(ssh)root@k3picoitx [ ~ ] # busybox devmem 0xc0800000 # check the value of the fake doorbell at SRAM
0x12345678
(ssh)root@k3picoitx [ ~ ] # setpci -s 0001:01:00.0 CAP_MSI+4.L=0xc0800000 # redirect the MSI to the fake doorbell. The MSI is working previously so there's no need to clear pending interrupts.
(ssh)root@k3picoitx [ ~ ] # busybox devmem 0xc0800000 # check what's overriden by the MSI
0x0000000C
This shows that the problematic card do issue 16-bit MSI write instead of 32-bit ones. As the problematic card is from a famous and important vendor, maybe AIA should do the incompatible change to support such card.
I suggest the rule to be changed to Only aligned 32-bit simple reads and writes and 16-bit writes aligned to 32-bit address are supported within an interrupt file’s memory region. to support such kind of card.
The AIA specification says
Only naturally aligned 32-bit simple reads and writes are supported within an interrupt file’s memory region., which seems to be okay because the PCI/PCIe spec defines MSIs as "DWORD memory write" (DWORD here is defined as on x86 PC -- 32-bit); however, either PCI and PCIe has DWORD as the smallest transfer size, sub-DWORD writes are described with extra byte enable, and it's not clear that MSI write should have a byte enable of 0b1111 (i.e. all bytes enabled).SpacemiT K3 is a SoC implementing AIA, and its IMSIC follows the rule above by ignoring 16-bit writes to
seteipnum_leregister (no interrupts are triggered for such writes), and when attaching a PCIe card with VID 0x1002 and PID 0x6779 (AMD/ATI CAICOS) to it, the system never receives any interrupts from this card.The following experiment redirecting MSI to on-chip SRAM shows that the MSI write request from this card is 16-bit:
In contrary, a working card's behavior is like below (plugged at the same slot and got allocated the same MSI by the system):
This shows that the problematic card do issue 16-bit MSI write instead of 32-bit ones. As the problematic card is from a famous and important vendor, maybe AIA should do the incompatible change to support such card.
I suggest the rule to be changed to
Only aligned 32-bit simple reads and writes and 16-bit writes aligned to 32-bit address are supported within an interrupt file’s memory region.to support such kind of card.