FRIEDA: Flexible Robust Intelligent Elastic Data Management

Project Details

Project Lead
Lavanya Ramakrishnan 
Project Manager
Lavanya Ramakrishnan 
Project Members
Val Hendrix, Pradeep Kumar Mantha, Eugen Feller, Zhao Zhang, Tonglin Li, Mike Hance, Sowmya Balasubramanian  
Supporting Experts
Gregor von Laszewski  
Institution
Lawrence Berkeley National Lab, Advanced Computing for Science  
Discipline
Computer Science (401) 

Abstract

Scientific applications are increasingly using cloud resources for their data analysis workflows. We use the cloud loosely to signify transient environments. However, managing data effectively and efficiently over these cloud resources is challenging due to the myriad storage choices with different performance, cost trade-offs, complex application choices and complexity associated with elasticity, failure rates in these environments. The different data access patterns for data-intensive scientific applications require a more flexible and robust data management solution than the ones currently in existence.  FRIEDA is a Flexible Robust Intelligent Elastic Data Management framework that employs a range of data management strategies approaches in elastic environments.

Specifically, we are investigating

  • Semi-automated storage choices and data management strategies for data analysis science workflows
  • Management of the life cycle of scientific applications in hybrid environments using HPC and cloud resources.

Intellectual Merit

The proposed research will significantly impact current understanding on storage and data management in CS. The approaches will be useful for data-intensive applications in both cloud and future HPC environments

Broader Impacts

The project is expected to impact scientific discoveries through use of cloud resources.

Scale of Use

Many of the things we need to test will need to be done at scale since a lot of the data management issues are at scale. We will start with a few nodes used for development, testing and will then need to be able to scale up for tests. We will take as much as you can give us over the sites.