FRIEDA: Flexible Robust Intelligent Elastic Data Management

Project ID
FG-298
Project Categories
Computer Science
Project Keywords
Completed
Abstract
Scientific applications are increasingly using cloud resources for their data analysis workflows. We use the cloud loosely to signify transient environments. However, managing data effectively and efficiently over these cloud resources is challenging due to the myriad storage choices with different performance, cost trade-offs, complex application choices and complexity associated with elasticity, failure rates in these environments. The different data access patterns for data-intensive scientific applications require a more flexible and robust data management solution than the ones currently in existence.  FRIEDA is a Flexible Robust Intelligent Elastic Data Management framework that employs a range of data management strategies approaches in elastic environments.

Specifically, we are investigating

  • Semi-automated storage choices and data management strategies for data analysis science workflows
  • Management of the life cycle of scientific applications in hybrid environments using HPC and cloud resources.
Use of FutureSystems
FutureGrid will be used to investigate the following issues -
* trade-offs between different storage options in cloud environments
* elastic data management at scale in cloud environments
Scale of Use
Many of the things we need to test will need to be done at scale since a lot of the data management issues are at scale. We will start with a few nodes used for development, testing and will then need to be able to scale up for tests. We will take as much as you can give us over the sites.