The Virtual Block Store system

Project ID
FG-12
Project Categories
Computer Science
Completed
Abstract
The development of cloud infrastructures has stimulated interest in virtualized block storage systems, exemplified by Amazon Elastic Block Store (EBS), Eucalyptus EBS implementation, and the Virtual Block Store (VBS) system. Compared with other solutions, VBS is designed for flexibility, and can be extended to support various Virtual Machine Managers and Cloud platforms. However, due to its single-volume-server architecture, VBS has the problem of single point of failure and low scalability. This project involves our latest improvements to VBS for solving these problems, including a new distributed architecture based on the Lustre file system, new workflows, better reliability and scalability, and read-only volume sharing. We call this improved implementation VBS-Lustre. We have done some small-scale preliminary tests, and the results show that VBS-Lustre can provide both better throughput and higher scalability in multiple attachment scenarios than VBS. VBS-Lustre could potentially be applied to solve some challenges for current cluster file systems, such as metadata management and small file access.
Use of FutureSystems
We would like to use FutureGrid resources to launch a relatively large scale test of the Virtual Block Store (VBS) system developed by the Community Grids Lab of Indiana University. VBS is a block-level storage system that can provide virtual volumes for the virtual machines (VM) in clouds. Users can create volumes in VBS and attach them to their VMs, and then use the volumes as if they are local disks installed on their VMs. We built VBS on a distributed storage architecture based on Lustre to achieve high performance, scalability, and reliability. Since VBS is designed to work with cloud infrastructures, scalability is extremely important for its usage. In this test, we would like to set up a testbed with tens of server nodes and test the performance of VBS in case of tens or even hundreds of clients.
Scale of Use
30 to 60 bare metal machines for about two weeks.