Resolving Linux PCI Interrupt Conflicts

Summary:

A guide to resolving PCI interrupt problems when using the Linux operating system, focusing on network device drivers.

Background

The PCI bus is designed so that multiple devices may physically share a single interrupt line. Most current Linux device drivers for PCI devices support this feature, but there are exceptions and several potential problem you may encounter.

Symptoms

The usual symptom of an interrupt problem with an Ethernet device driver is that 'ifconfig' reports an EAGAIN error when the interface is started. This error is typically reported with the message "Try again", which means "try again when the interrupt conflict is cleared".

When you encounter this error message you can run the 'ifconfig' or 'dmesg' programs and compare the requested IRQ with the contents of /proc/interrupts to determine the device conflicts.

The solutions

If the card is reporting IRQ0 or IRQ255, that indicates the card has not been assigned an interrupt. There are two likely fixes: either the BIOS does not have enough IRQ lines available for PCI devices (e.g. all are assigned to "legacy" ISA devices instead of "PnP"), or the BIOS has a "PnP OS" setting that must be disabled. Yes, this is confusing: the "PnP OS" setting is bad (it really means "Windows OS?"), but the PnP IRQ assignment is good.

If the card is reporting a valid IRQ, but that IRQ is being used by another device you have an interrupt conflict. The easiest and generally best solution is to put the conflicting device on another IRQ line. This can only be done through the PCI BIOS setup. Unlike ISA cards, PCI cards have no way of setting their own IRQ. That is done at boot time by the PCI BIOS, and the BIOS reports its selection

If the conflicting device is a SCSI driver, updating the kernel to 2.0.33 or later will often allow you to use the newer SCSI drivers that are willing to share an IRQ line. (See below for information on how to do with without upgrading to 2.0.33.)

Technical background.

Unlike ISA and EISA cards, PCI cards do not select their own IRQ line. Instead they can only select one of four "interrupt pins", which are mapped by the motherboard chipset to IRQ lines. I'll stress this again -- only the motherboard BIOS knows how to change the IRQ mapping for the card. The IRQ mapping is changed in the PCI BIOS setup, however the exact method used is BIOS-specific.

Common Problems

My other PCI board always uses the same IRQ as an Adaptec 2940
The preferred solution is to put each device on its own IRQ line. But some motherboard BIOSes, notably the Intel PR440FX, don't permit this. To use both at the same time requires a slight modification of the 2940 driver: remove the SA_INTERRUPT flag to the request_irq() call. This is required whenever the 2940 driver must share an IRQ with any other device driver.

-     if (request_irq(config->irq, aic7xxx_isr, SA_INTERRUPT | SA_SHIRQ,
+     if (request_irq(config->irq, aic7xxx_isr, SA_SHIRQ, "aic7xxx", NULL))

Using the Intel PR440FX Motherboard

An alternate solution when using the PR440FX "Providence" motherboard is to force assignment of separate interrupt lines. A way to do this is detailed in Configuring the PR440FX motherboard written by Larry M. Augustin at VA Research.

Larry also reports that

Upgrading to the latest PR440FX BIOS (05 revision) makes it much easier to force PCI devices to get their own interrupts.
The 2.0.27 kernel at ftp.varesearch.com:/pub/kernel/linux2.0.27-gcc2.7.2 contains an updated Adaptec driver that doesn't seem to suffer the kernel stack overflows of the older driver and works fine when sharing an interrupt with other device drivers.

PCI detection order

PCI cards are detected in PCI device order. There is no fixed rule, but use the following guidelines:

The primary PCI bus is Bus #0. (Most machines have only one PCI bus, so this is easy.)
PCI device numbers are usually assigned in slot order.
The left-most PCI slot facing the back panel is often the lowest numbered.
On-motherboard devices are assigned the highest device numbers on Bus #0. (This is so that a plugged-in adapter card overrides a built-in one.)
Bridged cards, e.g. the 4 port AHA6944TX, are assigned Bus#1 and up, and thus these devices are last detected.

In-depth Technical Flames

You might as well learn something pointlessly technical while you are here...

Why SA_INTERRUPT in the SCSI drivers is a Bad Thing

The SA_INTERRUPT flag indicates "fast interrupts", where interrupt handlers run with all other interrupts disabled in return for the promise that they will limit their work to trivial, strictly-limited actions. But the SA_INTERRUPT requirement cannot be met when the interrupt line is shared: an arbitrary collection of trivial tasks is no longer trivial.

SA_INTERRUPT was designed for no-FIFO (and thus high-priority) serial ports where the only work that needed to be done was removing a single byte from the UART, writing it to a queue, and setting the "bottom half" flag. All other work was done later, with interrupts enabled, in the kernel "bottom half".

While the Ethernet drivers (and most other drivers) will work with the SA_INTERRUPT flag, it could potentially have a very negative impact on all other interrupt-driven kernel service. That includes just about everything that the system does, including timer ticks.

I believe that very few complex devices can be correctly run by a device driver that uses SA_INTERRUPT. The BusLogic driver, in particular, does an extensive and potentially unlimited amount of work. It should *not* be using SA_INTERRUPT. My Ethernet drivers do much less work at interrupt time, and have strict work limits, and even I don't feel that they should be run with SA_INTERRUPT.

Other other

Clarifications, suggestions, changes and updates to this page are always welcome.

Author: Donald Becker, becker@cesdis.gsfc.nasa.gov.