PCIe Enumeration Failure with OFS 2024.1 Design on Linux 5.15.92-DFL and 6.1.78-DFL Kernels

[Divya-25-design]

• We generated the OFS 2024.1 design for the AGIB027R31B FPGA and compiled it using Quartus 2024.1. The PCIe endpoint was evaluated on a Precision 3460 system with a Gen4 x16 PCIe interface running Ubuntu 22.04.3 LTS.

• We are currently observing that PCIe endpoints generated from OFS 2024.1 are not detected when using Linux kernels 5.15.92-DFL and 6.1.78-DFL. In contrast, the same OFS design enumerates successfully on the Linux 6.8 generic kernel.

• For reference, the PCIe example design enumerates correctly on both Linux 5.15-DFL and Linux 6.8 generic kernels, while the OFS generated design is not detected on any Linux DFL. Logs from all test cases are attached for reference.

• System Configuration:

1. OFS Design Version: 2024.1
2. Quartus Version: 2024.1
3. PC: Precision 3460 gen4 x16 PCIe
4. OS: Ubuntu 22.04.3 LTS
5. FPGA Platform: AGIB027R31B1E1V
6. Tested Linux Kernels: 5.15.92-DFL, 6.1.78-DFL, 6.8.0-90 generic

The test results for the 5.15.92dfl and 6.1.78dfl dmesg -w given below
[ 61.457345] dfl-pci 0000:01:00.0: enabling device (0000 -> 0002)
[ 61.457357] dfl-pci 0000:01:00.0: PCIE AER unavailable -5.
[ 61.457904] dfl-pci 0000:01:00.1: enabling device (0000 -> 0002)
[ 61.457910] dfl-pci 0000:01:00.1: PCIE AER unavailable -5.
[ 61.458061] dfl-pci 0000:01:00.1: Unknown feature type 0xf id 0xfff
[ 61.458144] dfl-pci 0000:01:00.1: enumeration failure -19.
[ 61.458192] dfl-pci 0000:01:00.2: enabling device (0000 -> 0002)
[ 61.458196] dfl-pci 0000:01:00.2: PCIE AER unavailable -5.
[ 61.458343] dfl-pci 0000:01:00.2: Unknown feature type 0xf id 0xfff
[ 61.458415] dfl-pci 0000:01:00.2: enumeration failure -19.
[ 61.458452] dfl-pci 0000:01:00.3: enabling device (0000 -> 0002)
[ 61.458456] dfl-pci 0000:01:00.3: PCIE AER unavailable -5.
[ 61.458604] dfl-pci 0000:01:00.3: Unknown feature type 0xf id 0xfff
[ 61.458678] dfl-pci 0000:01:00.3: enumeration failure -19.
[ 61.458716] dfl-pci 0000:01:00.4: enabling device (0000 -> 0002)
[ 61.458720] dfl-pci 0000:01:00.4: PCIE AER unavailable -5.
[ 61.458864] dfl-pci 0000:01:00.4: Unknown feature type 0xf id 0xfff
[ 61.458935] dfl-pci 0000:01:00.4: enumeration failure -19.
[ 63.426459] mei_me 0000:00:16.0: enabling device (0000 -> 0002)

But the kernel 6.8.0-90 generic able to list PCIe device, shown below.
iwave@iwave:~$ lspci
00:00.0 Host bridge: Intel Corporation Device a703 (rev 01)
00:01.0 PCI bridge: Intel Corporation Device a70d (rev 01)
00:02.0 VGA compatible controller: Intel Corporation Device a780 (rev 04)
00:04.0 Signal processing controller: Intel Corporation Device a71d (rev 01)
00:08.0 System peripheral: Intel Corporation Device a74f (rev 01)
00:14.0 USB controller: Intel Corporation Device 7ae0 (rev 11)
00:14.2 RAM memory: Intel Corporation Device 7aa7 (rev 11)
00:15.0 Serial bus controller: Intel Corporation Device 7acc (rev 11)
00:16.0 Communication controller: Intel Corporation Device 7ae8 (rev 11)
00:17.0 SATA controller: Intel Corporation Device 7ae2 (rev 11)
00:1f.0 ISA bridge: Intel Corporation Device 7a88 (rev 11)
00:1f.3 Audio device: Intel Corporation Device 7ad0 (rev 11)
00:1f.4 SMBus: Intel Corporation Device 7aa3 (rev 11)
00:1f.5 Serial bus controller: Intel Corporation Device 7aa4 (rev 11)
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (17) I219-LM (rev 11)
01:00.0 Processing accelerators: Intel Corporation Device bcce (rev 01)
01:00.1 Processing accelerators: Intel Corporation Device bcce (rev 01)
01:00.2 Processing accelerators: Intel Corporation Device bcce (rev 01)
01:00.3 Processing accelerators: Intel Corporation Device bcce (rev 01)
01:00.4 Processing accelerators: Intel Corporation Device bcce (rev 01)

Since the OFS design requires DFL for testing, we are seeking guidance on how to debug or resolve the enumeration failures observed on Linux DFL kernels 5.15.92-DFL and 6.1.78-DFL.