Sky Computing

Project Details

Project Lead
Pierre Riteau 
Project Manager
Pierre Riteau 
Project Members
Mauricio Tsugawa  
Institution
University of Chicago, Computation Institute  
Discipline
Computer Science (401) 

Abstract

Scientific problems are often large and distributed by nature as they combine processing of data produced at various sources. In the context of cloud computing this leads to a question of what problems would arise if we were to use resources obtain from not just one IaaS cloud, but a federation of multiple geographically distributed infrastructure clouds. Such multi-cloud federations have been called “sky computing” (see [1]) and involve challenges of standardization, security, configuration, and networking.

Intellectual Merit

Provide later

Broader Impacts

Provide later

Scale of Use

Provide later

Results

Problem: Scientific problems are often large and distributed by nature as they combine processing of data produced at various sources. In the context of cloud computing this leads to a question of what problems would arise if we were to use resources obtain from not just one IaaS cloud, but a federation of multiple geographically distributed infrastructure clouds. Such multi-cloud federations have been called “sky computing” (see [1]) and involve challenges of standardization, security, configuration, and networking.

Project: Pierre Riteau from the University of Rennes 1 proposed one solution in this space by creating a virtual cluster combining resources obtained from six geographically distributed Nimbus clouds: three hosted on Grid’5000 and three hosted on FutureGrid. Experimenting with distribution and scale he succeeded in creating a geographically distributed virtual cluster of over 1000 cores. His solution overcame firewall and incompatible network policy problems by using the ViNe overlay to create a virtual network and secure creation of a virtual cluster by using the Nimbus Context Broker. Further, in order to overcme the image distribution problem which becomes a significant obstacle to fast deployment at this scale Pierre developed a QCOW system which uses copy-on-write techniques to speed up image distribution.

Widely distributed compatible cloud resources needed for this experiment would have been impossible to obtain with the existence of resources such as Grid’5000 and FutureGrid and their close collaboration. In addition to experimenting with research problems at unprecedented scale, this project was also a proof-of-concept and a trail blazer for a close collaboration between Grid’5000 and FutureGrid. Because of its integrative nature, this project was demonstrated at OGF 29 in June 2010.

Figure: The sky computing experiment built a virtual cluster distributed over six Nimbus clouds using the ViNe virtual network overlay and the Nimbus Context Broker.

References:

  • Sky Computing on FutureGrid and Grid’5000, Pierre Riteau, Mauricio Tsugawa, Andrea Matsunaga, José Fortes, Tim Freeman, David LaBissoniere, Kate Keahey. TeraGrid 2010, Pittsburgh, PA. August 2010 http://www.nimbusproject.org/files/riteau_tg10.ppt