This is an old revision of the document!
GPU Passthrough
GPU passthrough refers to configuration where a graphics card is reserved for use by a virtual machine. Making use of VT-d, IOMMU, and related technologies, the graphics card can be used at native performance.
This page describes my specific system configurations.
Hardware
Hardware Configuration:
Asus P8B WS
Intel E3-1275 (Sandy Bridge)
32GB Kingston ECC Unbuffered
EVGA NVIDIA GTX 570
[Testing: NVIDIA 8800 GT]
[Purchase: Gigabyte 750 Ti GV-N75TOC-2GI]
Asus Xonar DX
iommu_groups.sh
Shows pci devices grouped by IOMMU group
#!/bin/sh
BASE="/sys/kernel/iommu_groups"
for i in $(find $BASE -maxdepth 1 -mindepth 1 -type d); do
GROUP=$(basename $i)
echo "### GROUP $GROUP ###"
for j in $(find $i/devices -type l); do
DEV=$(basename $j)
echo -n " "
lspci -s $DEV
done
done
iommu_groups.sh output
This output corresponds to the following pci card configuration:
Slot 1 (16x): Nvidia GeForce 570
Slot 2 (16x): empty
Slot 3 (1x): empty
Slot 4 (16x): Asus Xonar DX
Slot 5 (PCI): Silicon Image SATA Controller
Slot 6 (16x): empty
iommu_groups.sh output
### GROUP 0 ###
00:00.0 Host bridge: Intel Corporation Xeon E3-1200 Processor Family DRAM Controller (rev 09)
### GROUP 1 ###
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09)
01:00.0 VGA compatible controller: NVIDIA Corporation GF110 [GeForce GTX 570] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GF110 High Definition Audio Controller (rev a1)
### GROUP 2 ###
00:02.0 Display controller: Intel Corporation Xeon E3-1200 Processor Family Integrated Graphics Controller (rev 09)
### GROUP 3 ###
00:06.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09)
02:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa)
03:04.0 Multimedia audio controller: C-Media Electronics Inc CMI8788 [Oxygen HD Audio]
### GROUP 4 ###
00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 (rev 04)
### GROUP 5 ###
00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05)
### GROUP 6 ###
00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller (rev 05)
### GROUP 7 ###
00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5)
### GROUP 8 ###
00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 5 (rev b5)
### GROUP 9 ###
00:1c.5 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 6 (rev b5)
### GROUP 10 ###
00:1c.6 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 7 (rev b5)
### GROUP 11 ###
00:1c.7 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 8 (rev b5)
### GROUP 12 ###
00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05)
### GROUP 13 ###
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a5)
09:00.0 SATA controller: Silicon Image, Inc. SiI 3512 [SATALink/SATARaid] Serial ATA Controller (rev 01)
09:03.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller (rev c0)
### GROUP 14 ###
00:1f.0 ISA bridge: Intel Corporation C206 Chipset Family LPC Controller (rev 05)
00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family SATA AHCI Controller (rev 05)
00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller (rev 05)
### GROUP 15 ###
06:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
### GROUP 16 ###
07:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
### GROUP 17 ###
08:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
Kernel
Because the C206 chipset is affected by the Intel root port isolation quirk (1, 2), both graphics cards are isolated in their own IOMMU group. The pcie_acs_override patch is not needed. (To check if the root port isolation quirk exists, grep dmesg for “Intel PCH root port ACS workaround enabled”.)
If we wanted to use the integrated graphics of the E3-1275, then we would need to either:
Use the i914.enable_hd_vgaarb patch (and lose DRI).
Use a guest GPU that supports UEFI and use the OVMF boot firmware to start the virtual machine. (OVMF can initialize the GPU using legacy-free UEFI, keeping DRI and bypassing VGA and the need for arbitration.)
References
