面向同驻虚拟机的高效共享内存文件系统

The vast file transfers among virtual machines or between virtual machines and physical machines in cloud environment bring large overhead for the system. Shared file system for the virtual machines on a same physical machine, i.e., co-resident virtual machines, and the underlying physical machine is an effective approach for improving the performance of file transfers. Emerging Non-Volatile Memory (NVM) is a promising high-performance storage for shared files for the merits of NVM, such as near-DRAM speed, high density, byte-addressability, and persistency. However, existing shared file system designs rely on either network devices or virtual I/Os, which cannot fully exploit the benefits of NVM. This paper proposes a new design of shared in-memory file system that supports efficient file access on the shared files of co-resident virtual machines and the underlying physical machine. First, the proposed design sets up a dedicated NVM zone on the physical machine as the sharing storage for the shared in-memory file system. The files of the shared in-memory file system are organized in a "shared page table" that is in the same form of the page table of the processes. A file on the shared NVM can be exposed to the contiguous virtual address spaces of multiple co-resident virtual machines and the physical machine by embedding the shared page table to the page table of the processes of the co-resident virtual machines or the physical machine in open operation. Those processes whether running in virtual machines or the underlying physical machine can access the shared file data with high throughput via the continuous virtual addresses. The traversal of the file index, i.e., the shared page table, for locating the shared file data can fully take advantage of the hardware MMU in processor. Different from the approaches of sharing file using network and the virtual I/O, the proposed design highly reduces the overhead of I/O software stacks and redundant data copies by directly accessing files using virtual address space and hardware MMU. Second, the Virtual File System (VFS) caches of the operating systems of co-resident virtual machines may cause inconsistency of shared files for each VFS cache maintains a copy of the sharing file metadata. We solve the inconsistency problem by proactively synchronizing the metadata of the sharing files in each read/write operation. Third, to ensure the concurrency of file accesses, we present a new synchronous mechanism for shared files. The variable of the shared semaphore is stored on the shared NVM of the co-resident virtual machines and the underlying physical machine. The atomicity of the updating operation of the semaphore variable is maintained by the atomic instructions supplied by the primitives of CPUs. Based on the proposed design, we implement an efficient shared in-memory file system StargateFS in the KVM platform in the Linux kernel. We conduct extensive experiments to evaluate the performance of StargateFS. The experimental results show that the average throughput of StargateFS achieves 64 times, 172 times and 191 times higher than VirtFS, the state-of-the-art shared file system, Samba, and NFS, respectively. Besides, the performance of the shared semaphore on StargateFS is 2.3 times that of the semaphore on Nahanni, the state-of-the-art shared memory mechanism of co-resident virtual machines.