Karnak Prediction Service

Project Details

Project Lead
Warren Smith 
Project Manager
Warren Smith 
Institution
University of Texas at Austin, Texas Advanced Computing Center  
Discipline
Computer Science (401) 
Subdiscipline
11.01 Computer and Information Sciences, General 

Abstract

Karnak is a prediction service that was originally deployed on TeraGrid (http://karnak.teragrid.org) and continues to be available in XSEDE. The service currently provides predictions of how long jobs will wait in scheduler batch queues before they begin to execute. This project will use FutureGrid to evaluate alternative implementations of the service, optimize the accuracy of the machine learning techniques used by the service, and investigate alternative prediction techniques.

Intellectual Merit

This work includes an evaluation of different prediction techniques when used to predict queue wait times (and perhaps other properties). This is novel work and should result in one or more publications.

Broader Impacts

The Karnak service is available to XSEDE and is being used by XSEDE users. Any improvements to the service (accuracy, reliability, response time) will benefit scientists using XSEDE. The XSEDE Karnak service is available through the XSEDE portal and can also be accessed directly by users and science gateways.

Scale of Use

While evaluating implementations, I expect to run a single virtual machine at a time and will run that virtual machine during business hours a few days a week for approximately 3 months. To optimize prediction techniques, I expect to submit HPC jobs that use approximately 48 node hours every week for approximately 6 months.