PCI helper library overview

The PCI helper library contains functions to discover and configure PCI devices in order to use them from an INtime application.

Before a PCI device may be accessed from an INtime for Windows application the device should be assigned to an INtime node using the INtime Device Manager utility, accessible from the INtime Configuration applet. If a device is not assigned to the INtime node then accessing that device's register may cause Windows to crash, and it will not be possible to create an interrupt handler for the device. It is possible to determine whether the device is accessible by inspecting the bUnusable field of the PCIDEV stucture. If the field is non-zero then the device should not be accessed from an INtime application.

The basic functions available from the library allow a device to be searched for by its PCI Vendor and PCI Device IDs, or its PCI Class ID, returning a populated PCIDEV structure. See PciFindDevice and PciFindClass. If the absolute PCI address of a device is known it is also possible to populate the structure using PciReadHeader. Configuration registers may be read and written using PciGetConfigRegister and PciSetConfigRegister.

A second group of functions performs management tasks, such as enabling and disabling features, searching for capabilities and returning configuration information. Enabling a device may involve turning on and off access to its register sets (PciEnableDevice/PciDisableDevice), enabling bus mastering capability (PciEnableDevice), and so on.

A third group returns textual information about a device, such as its Vendor and Device names, and Class name (PciVendorName, PciDeviceName, PciClassName).

PCI device discovery

When the system boots, the firmware or operating system enumerates the PCI bus and assigns memory and I/O addresses to each device according to their requirements, then assigns interrupt numbers if required. Because of this dynamic configuration process, software must search for the device it wants to program and discover the parameters it has been assigned. INtime software provides a library of functions to simplify the process of discovering and enabling a PCI device.

PCI devices are identified by two numbers: Vendor ID and Device ID. You need to know these numbers for your device before you start. The software uses these numbers to scan the PCI bus for the target device.

Each PCI device may be assigned up to six areas where their registers are mapped, and these areas may be in memory or I/O space, according to the device's requirements. You also need to know which of the six areas (BARs) are used by the device and what each is used for.

The fundamental call of the PCI library is PciFindDevice. It takes a pointer to a structure of type PCIDEV which contains parameters used to find the device. Information about the device is returned in the same structure after the call.

Prior to making the call you need to initialize these fields:

wVendorId

contains the vendor ID of the device you are searching for

wDeviceId

contains the device ID of the device you are searching for.

wDeviceIndex

contains the instance of the device you want to find.

A system may contain several devices of the same type. To locate a particular device, you must specify the instance in the wDeviceIndex field, where 0 (zero) represents the first instance, 1 the second, and so on.

Once you have initialized the fields, call PciFindDevice:

PCIDEV dev;
dev.wVendorId = VENDOR_ID;
dev.wDeviceId = DEVICE_ID;
dev.wDeviceIndex = 0;
if (!PciFindDevice(&dev)) {
  printf("Unable to find PCI device\n");
  return;
}

Device registers are not accessible until enabled. In addition, many PCI devices conform to the ACPI power management spec, and can be left in a powered-down state. To ensure that the device is properly initialized to the D0 state, call PciEnableDevice before any other programming:

PciEnableDevice(&dev);

If your device has bus mastering capability (if the device is capable of reading and writing to memory), turn the capability on like this:

PciSetMaster(&dev);

Access configuration space registers

The PCI configuration space may be read from or written to using the PciGetConfigRegister and PciSetConfigRegister calls. The extended PCI Express header space for a PCIe device may be accessed using the PcieGetConfigRegister and PcieSetConfigRegister calls.

// Read the first capability record location
pciCaps = PciGetConfigRegister(&dev, PCI_REG_CAPABILITY_LIST, T_BYTE);

Programming the device registers

Now the PCIDEV structure contains all the values you need to set up your software to program the device. First, the IoSpace array contains the values you need to access the device registers. For example, let us suppose that the device registers are mapped to BAR0 as memory registers, and they are also mapped to BAR1 as I/O registers (this is quite a common occurrence giving the programmer the choice which space to program the device in). You can choose to program in memory space or I/O space.

To access the registers in memory space, we need to map the address of the space to our process so we can access the physical device. The fields in IoSpace[0] tell us the physical memory address and the size of the space — perfect for calling MapRtPhysicalMemory:

DWORD *regs;

regs = MapRtPhysicalMemory(dev.IoSpace[0].start, dev.IoSpace[0].size);

Now the pointer regs can be used to access the device registers. Note that the regs variable can be a pointer to any type to suit your application and the hardware register structure.

Alternatively, using I/O space:

WORD iobase;

iobase = (WORD)dev.IoSpace[1].start;

Now iobase may be used as the base address for the I/O registers. I/O register may be accessed via the INtime I/O functions (inbyte, outbyte, etc.)

PCI device interrupts

Let us first consider how a PCI device generates an interrupt. If it has the capability to generate an interrupt using one of the INT signals on the PCI card (referred to as a "legacy interrupt" in the rest of this article) this will be indicated in the PCI device header, and the header will also contain the interrupt number ("IRQ") for the device. Your device may also have the capability of generating a Message Signaled Interrupt ("MSI") in which case the interrupt is signaled directly to the CPU and bypasses the interrupt controller. This capability is optional for PCI devices and mandatory for PCI Express devices which generate an interrupt.

In order to handle interrupts from the device there are a couple of things to consider. First, the device must be assigned to the INtime kernel. This prevents Windows from also attempting to access the device at the same time as INtime is trying to do so. Secondly, we need to realize that PCI legacy interrupts and level-triggered interrupts can share an interrupt signal with other PCI devices. This can be a problem if a Windows device shares an interrupt signal with an INtime device. If both devices were to generate an interrupt at the same time, both INtime and Windows have to allow interrupt handlers to run in order to clear the interrupt signal for reuse. This would destroy any determinism we have achieved by assigning the device to INtime in the first place, so it is not allowed.

To solve both problems, configure the device in the INtime Control Panel by opening the INtime Device Manager. This gives a view of the devices divided into two zones, Windows devices and INtime devices. At first, all devices are assigned to Windows. To reassign a device to INtime, identify the device in the right-hand pane, right click on it and select "Pass to INtime (with legacy IRQ)". The description will change and the device will be added to a list of devices to pass to INtime. If there is a conflict of interrupt signals then you will be warned. If two devices share an interrupt signal but both have been transferred to INtime, then the warning disappears because this is an allowed configuration.

Note that you may also pass a device to the INtime kernel without an interrupt. Do this in the case where your device does not have an interrupt, or you are not going to use the interrupt capability of your device, or your device is capable of generating a Message Signaled Interrupt ("MSI"). MSIs are not subject to the problem of shared interrupts.

Once you have saved your configuration, you can restart the INtime kernel and now the devices will be ready to generate interrupts to the INtime kernel.

PCI legacy interrupt programming

To program your application to handle legacy interrupts from your device, first we have to look up how the device is configured. The PciGetInterruptLevel call will obtain that information, which you can then use to set up an interrupt handler for your device. Note that PCI interrupt signals are designed to be shared with other PCI devices, so you should generally assume that they will be, so use the SHARED_LEVEL parameter when setting up the interrupt handler:

WORD wHwLevel;
WORD wLevel;

SetRtProcessMaxPriority(NULL_RTHANDLE, 0);
wHwLevel = PciGetInterruptLevel(&pci);
if (wHwLevel == 0xffff) {
  printf("Device does not generate legacy interrupts\n");
  return;
}

wHwLevel |= SHARED_LEVEL;
wLevel = SetRtInterruptHandlerEx(wHwLevel, 16, MyIntHandler, pParam);

Note that the value returned by the call to SetRtInterruptHandler is a modified version of the wHwLevel parameter passed into the call. This is because the input parameter, wHwLevel, contains only enough information to identify the interrupt line (IRQ), whereas the return value identifies both the IRQ and the specific handler that we just installed. The return value will be used in subsequent interrupt calls.

Finally, let us look at the interrupt handler. Since this is called directly from the kernel it needs some extra decoration to ensure it works properly.

__INTERRUPT void MyIntHandler(
    WORD wCSRA,
    WORD wLevel,
    PVOID pParam)
{
    // TODO: any local variables are declared here.
    __SHARED_INTERRUPT_PROLOG();

    // WARNING:
    // inside the handler you should NEVER use the address of
    // a local variable, nor should you call a function (directly
    // or indirectly) that uses the address of a local variable

        // TODO: process the interrupt in handler context

    SignalRtInterruptThread(wLevel);

    __SHARED_INTERRUPT_RETURN();
}

You can use the INtime Wizards in the Visual Studio to generate sample interrupt code.

PCI MSI programming

MSI programming is slightly different. We can use the generic value of MSI_LEVEL for the interrupt level parameter, but we need to pass information about the PCI device to the kernel so it can program the device directly. This information is passed in a structure of type MSI_PARAM:

MSI_PARAM msi;

SetRtProcessMaxPriority(NULL_RTHANDLE, 0);
msi.PciAddress = MKPCIADDR(&pci);
msi.ReservedZero = 0;
msi.Param = param;    // user parameter to pass to the handler
wLevel = SetRtInterruptHandlerEx(wLevel, 16, MyMsiHandler, &msi);

The macro MKPCIADDR converts the PCI address information in the PCIDEV structure into an encoded value used by the kernel.

The interrupt handler is similar to the PCI handler, but the entire MSI_PARAM structure is passed to the handler so the user context parameter has to be extracted before use.

__INTERRUPT void MyMsiHandler(
    WORD wCSRA,
    WORD wLevel,
    PVOID pParam)
{
    MSI_PARAM *msi = (MSI_PARAM *)pParam;
    PVOID pUserParam = pParam->Param;       // extract user parameter
    // TODO: any local variables are declared here.
    __SHARED_INTERRUPT_PROLOG();
    // WARNING:
    // inside the handler you should NEVER use the address of
    // a local variable, nor should you call a function (directly
    // or indirectly) that uses the address of a local variable
    // TODO: process the interrupt in handler context
    SignalRtInterruptThread(wLevel);
    __SHARED_INTERRUPT_RETURN();
}

PCI library calls

Requirements