Publish/Subscribe Messaging as a Basis for TeraGrid Information Services

Project Details

Project Lead
Warren Smith 
Project Manager
Warren Smith 
University of Texas at Austin, Texas Advanced Computing Center  
Computer Science (401) 


"This project will use FutureGrid to evaluate the possibility of using a messaging service as the basis for TeraGrid information services. The goal is to determine whether a publish/subscribe messaging approach is more effective than the approach currently used in TeraGrid to gather information from distributed TeraGrid resources and provide this information to TeraGrid users and services.The project will initially implement a proof of concept where the messaging service is installed in a virtual machine on FutureGrid and information from TeraGrid resources is published to that messaging service. The project will subsequently evaluate the latency, throughput, and scalability of different messaging service implementations and configurations. Messaging services that support the AMQP protocol will be evaluated. "

Intellectual Merit

This work will produce quantitative information about the effectiveness of using a messaging system as the base layer of the information service for a large-scale cyberinfrastructure. This work will be published via technical reports and/or workshop, conference, or journal publications.

Broader Impacts

Improvements to the TeraGrid information services will help TeraGrid users, science gateways, and tool developers. This will result in more science being performed on TeraGrid.

Scale of Use

"The initial phase of this project will only require a single VM to run the messaging service. This VM will be instantiated for 4-6 hours at a time while I perform experiments that have TeraGrid resources publishing information to it. I expect this phase to last for approximately 2 months and to instantiate this VM roughly 2 days a week.During performance evaluation, I expect to use 9-17 VMs for an hour or two at a time. One virtual machine will be running the messaging service and 8-16 virtual machines will be emulating TeraGrid resources publishing information and TeraGrid users/tools consuming information. I expect this phase to last for approximately 2 months and to instantiate VMs for 2-3 hours at a time 2 or 3 days a week."


W. Smith. An Information Architecture Based on Publish/Subscribe Messaging. In Proceedings of the 2011 TeraGrid Conference. July, 2011. (Extended Abstract). Slides.

As outlined in the publication above, this project has used FutureGrid to prototype candidate implmementations for a new information service for TeraGrid/XSEDE and to gather performance information about the implementations. XSEDE has not yet begun serious discussions about how to implement a new information service, but I expect that FutureGrid will be used to gather data during that discussion and to perform testing on any new information service components before they go in to production on XSEDE.