GPU-Accelerated Gaussian Processes
We are developing an efficient and flexible Gaussian processes (GPs) library based on MXNet. This library will be open-source and benifits the machine learning as well as robotics community at large.
Gaussian processes are arguably the most important methods for inference on functions. GPs are widely adopted in supervised learning, unsupervised learning, and reinforcement learning. In spatial modeling problems, GPs have become the de-facto standard due to its well-calibrated uncertainty estimate, modeling flexibility, and robustness to overfitting.
Unfortunately, the cubic time complexity and quadratic space complexity of standard GPs hinder the application of these models for many domains. In particular, robotic applications require online learning and low memory consuming. Recently, some efforts toward scalable GPs enable the application of GPs on data set with millions of data points. These works take advantage of GPU acceleration and auto-differentiation to enable scalable inference and flexible model constructions. For example, GPflow is based on TensorFlow and GPyTorch relies on Pytorch.
We noticed that there is no dedicated GPs library based on MXNet. MXNet is designed for both efficiency and flexibility, and supports Nvidia Jetson devices such as TX2 or Nano. Hence, this MXNet-based GPs library will enable the application of GPs to autonomous systems with large-scale data set. Beyond GPU acceleration and auto-differentiation, GPs will be used as building blocks in this library, which makes it easier to construct hierarchical GPs, deep GPs, recurrent GPs, and hybrid models combining deep learning components. To make these complex models efficient and tractable, stochastic variational inference will be used as the major inference engine under the hood. Besides, existing GPs libraries are designed for data analysis, focusing on batch data at large scale. Also, the convergence diagnosis is normally done by a human expert. However, autonomous robots perceive data streamingly and need to learn from the newly acquired data without human-supervision. To this end, another feature of this library is the emphasis on online learning and automatic convergence diagnosis. We believe that this library will bring to the machine learning and robotics community an efficient and flexible GP library specially designed for autonomous systems