饶磊_OS2ATC-NVMe VFIO live migration-leirao@intel.pdf

1、1 1NVMe VFIO Live Migration for IPU/DPU DevicesRao LeiCloud Software Development Engineer,IntelAgenda Background and Motivations VFIO Live Migration Introduction NVMe VFIO Live migration design&implementation NVMe Controller Internal Data Live Migration Dirty Page Tracking Changes to NVMe Specificat

2、ion Status&PlanBackground and Motivations Intel ASIC IPU View Compute complex 16 ARM Neoverse N1 cores3GHz PCIe 4.0X16 3 channel LPDDR4X channels,Up to 48GB DRAM 2 ARM Cortex A53 cores for management Tightly coupled to HW accelerators Hardware accelerators Networking:2X100GbE or 1X200 GbE,flexible p

3、acket processor,RDMA,Traffic Shaper and Qos,OVS,virtio-net Storage:NVMe controller(SRNVMe controller(SR-IOV),NVMe over Fabric,IOV),NVMe over Fabric,AES-XTS encryption,virtio-blk Compute:Live migration engineLive migration engine,address translation engine,DMA engine,inline and lookaside crypto with

4、compressionBackground and Motivations PCIe SR-IOV Pros Software simplicity IOMMU-based DMA isolation Cons Fixed resource allocation Lack of composability Not support live migrationNot support live migrationVMApplicationPFPFBackendBackendResourcesResourcesQQQ QQQ QQQ VF1VF1VFnVFnDeviceDeviceIOMMUIOMM

5、U DMA(BDF)DMA(BDF)BDFBDFBDFBDFBDFBDFVFIO Live Migration-Introduction VFIO Pass-thru device live migration PCIe config space migration Simulated by Kernel and Qemu Device internal data migrationDevice internal data migration Additional DMA CMD(Piggyback on existing DMA descriptor queue)Additional DMA

6、 CMD(Piggyback on existing DMA descriptor queue)Additional Migration Registers in VF MMIO space Dirty page trackingDirty page tracking Leverage IOMMU dirty bit tracking(Available from Intel SPR platform)Use the onUse the on-device dirty page tracking engine for the legacy platformdevice dirty page t