SAGA

Project Details

Project Lead
Shantenu Jha 
Project Manager
Yaakoub El Khamra 
Project Members
Ole Weidner, Hartmut Kaiser, Pradeep Kumar Mantha, Andre Luckow, Sharath Maddineni, Pradeep Kumar Mantha, Andre Merzky, Melissa Romanus, Ashley Zebrowski, Mark Santcroos, Blaise Bourdin, Frank Löffler, Ole Weidner, Vishal Shah, Anjanibhargavi Ragothaman, Jeffrey Rabinowitz, Dinesh Ganapathi, Sanket Wagle, Matteo Turilli, Christian Straube, Antons Treikalis, Michael Wilde, Antons Treikalis, Nikhil Shenoy, Daniel S. Katz, Ioannis Paraskevakos, Sean Seyler, Ian Kenney, Oliver Beckstein  
Supporting Experts
 
Institution
Louisiana State University, Center for Computation & Technology  
Discipline
Computer Science (401) 

Abstract

The Simple API for Grid Applications (SAGA) is an OGF standard (http://www.ogf.org), and defines a high level, application oriented API for developing first principle distributed applications. Our SAGA implementation (in C++ and Python, see http://saga.cct.lsu.edu/) is able to interface to a variety of middleware backends. We also develop application frameworks based on SAGA, such as Master-Worker, MapReduce, AllPairs, BigJobs, etc.\n\n\n\nFor all those components, we intent to use futuregrid and the different software environments available on FG for extensive portability and interoperability testing, but also for scale-up and scale-out experiments. The proposed activities will allow to harden the SAGA components described above.

Intellectual Merit

The Simple API for Grid Applications (SAGA) is an OGF standard (http://www.ogf.org), and defines a high level, application oriented API for developing first principle distributed applications. Our SAGA implementation (in C++ and Python, see http://saga.cct.lsu.edu/) is able to interface to a variety of middleware backends. We also develop application frameworks based on SAGA, such as Master-Worker, MapReduce, AllPairs, BigJobs, etc.\n\n\n\nFor all those components, we intent to use futuregrid and the different software environments available on FG for extensive portability and interoperability testing, but also for scale-up and scale-out experiments. The proposed activities will allow to harden the SAGA components described above.

Broader Impacts

The Simple API for Grid Applications (SAGA) is an OGF standard (http://www.ogf.org), and defines a high level, application oriented API for developing first principle distributed applications. Our SAGA implementation (in C++ and Python, see http://saga.cct.lsu.edu/) is able to interface to a variety of middleware backends. We also develop application frameworks based on SAGA, such as Master-Worker, MapReduce, AllPairs, BigJobs, etc.\n\n\n\nFor all those components, we intent to use futuregrid and the different software environments available on FG for extensive portability and interoperability testing, but also for scale-up and scale-out experiments. The proposed activities will allow to harden the SAGA components described above.

Scale of Use

In general we have very low scale requirements, but would like to be able to test scale up and scale-out now and then, for very short periods of time (hours to a few days).

Results

Interoperable and Standards-based Distributed Cyberinfrastructure and Applications

Abstract:
Advances in many areas of science and scientific computing are predicated on rapid progress in fundamental
computer science and cyberinfrastructure, as well as their successful uptake by computational scientists. The
scope, scale and variety of distributed computing infrastructures (DCIs) currently available to Scientists and
CS researchers is both an opportunity, and a challenge. DCI present an opportunity, as they can support the
needs of a vast range and number of science requirements and usage-modes. The design and implementation
of DCI itself present a formidable intellectual challenge, not least because of the challenges in providing
interoperable tools and applications given the heterogeneity and diversity of DCIs.
 
Interoperability - standards based as well as otherwise, is an important necessary (though not sufficient
condition) system and application feature for the effective use of DCI and its scalability. This project report
presents a selection of results from Project No. 42 (SAGA) which makes use of FutureGrid to develop
the software components, runtime frameworks and to test and verify their usage, as well as initial e orts in
incorporating these strands into Graduate curriculum. Specially, we discuss our work on P* - a model for
pilot abstractions, and related implementations which demonstrate (amongst others) interoperability between
different pilot-job frameworks. In addition to the practical benefits interoperable and extensible pilot-
job framework, P* provides a fundamental shift in the state-of-the-art of tool development: for the first time
that we are aware, thanks to P* there now exists a theoretical and conceptual basis upon which to build the
tools and runtime systems. We also discuss standards based approaches to software interoperability, and the
related development challenges { including SAGA as a standards based generic access layer to DCIs. Finally,
we conclude by establishing how these strands have been brought together in a Graduate Classroom.

The full version of the report is available here.