MPI Java Performance Evaluation
In the last few years, Java gain popularity in processing “big data” mostly with Apache big data stack – a collection of open source frameworks dealing with abundant data, which includes several popular systems such as Hadoop, Hadoop Distributed File System (HDFS), and Spark. Efforts have been made to introduce Java to High Performance Computing (HPC) as well in the past, but were not embraced by the community due to performance concerns. However, with continuous improvements in Java performance an increasing interest has been placed on Java message passing support in HPC. We support this idea and show its feasibility in solving real world data analytics problems. This includes performance evaluation of two MPI Java frameworks - OpenMPI and FastMPJ - for real life machine learning problems.
Use of FutureSystems
FutureGrid will serve as the test bed where we can evaluate the performance of our applications.
Scale of Use
Around 20 nodes for few weeks