HPC meets Clouds

Project Details

Project Lead
Li Chunyan 
Project Manager
Li Chunyan 
Project Members
lee qingchun, gu genqiang, Xiaolong Wen, Xutao Yang, han qiang, pei xiao  
Supporting Experts
Tak-Lon Wu  
Institution
YunNan University, High Performance Computing Center  
Discipline
Computer Science (401) 

Abstract

As Cloud computing emerges as a dominant paradigm in distributed systems, it is important to fully understand the underlying technologies that make Clouds possible. One technology, and perhaps the most important, is virtualization. Recently virtualization, through the use of hypervisors, has become widely used and well understood by many. However, there are a large spread of different hypervisors, each with their own advantages and disadvantages. I hope to provide an in-depth analysis of some of today’s commonly accepted virtualization technologies from feature comparison to performance analysis, focusing on the applicability to High Performance Computing environments using FutureGrid resources.

Intellectual Merit

Through our work we will extensively study the necessary requirements to deploy HPC applications in a virtualized environment complementing exiting research on FutureGrid, taking into account key metrics (e.g. throughput, communication latency, etc.) and more detailed analysis of MPI programs. This will help us understand where Clouds stand in relation to Grids/Supercomputers thus allowing us to focus our efforts on improving the current Cloud infrastructure to perform on-par with conventional supercomputing facilities. Our motivation is to develop a new standard for HPC Clouds where the large scalability, high availability and fault-tolerance would provide scientists with a unique advantage when running simulations. [1] Younge, A. J., R. Henschel, J. Brown, G. von Laszewski, J. Qiu, and G. C. Fox, "Analysis of Virtualization Technologies for High Performance Computing Environments", The 4th International Conference on Cloud Computing (IEEE CLOUD 2011), Washington, DC, IEEE, 07/2011

Broader Impacts

The current Cloud infrastructure suffers from the overhead of utilizing a virtualized environment. This has lessened the impact of Cloud computing on the high performance computing community and the scientific community, at large. Through our project we will investigate and implement solutions to decrease this overhead and establish Clouds as a viable addition to the Grid and HPC cyber-infrastructures. This process would provide scientists with an extra alternative whose benefits include on-demand provisioning (a problem in conventional Grids/HPC), enhanced fault-tolerance and dynamic scale up/scale down or elasticity.

Scale of Use

We will need a few VMs to setup the environment and prepare the actual experiments. It may take some weeks. Then, we will run a set of run at different scales using different systems and different configurations. Analysis will be performed between experiments in order to evaluate only key scenarios of interest. This process may take some months but the use of the resources will not be continued.

Results

Hello,FutureGrid:
Firstly,We are very grateful to you that you provide a very good test-bed for us.
Secondly,We have do some experiment s including testing hpc and some cloud IaaS.Meanwhile,we also have analysed performance about that.We just start to study several days because that we join the futuregrid family is late.
In the end,I report our using resources,including HPC,openstake,nimbus,opennebula,hadoop and so on.
Besides,We are already preparing for a publication.