Information-Based Calibration of Uncertainty Quantification in Product-of-Experts Gaussian Process Models

Abstract

Background: In Gaussian process (GP) regression, the use of a single global GP (GP-glo) incurs cubic computational cost, which limits scalability to large datasets. Product-of-experts GP models (GP-pro), which combine a collection of local GP models that collaboratively capture global correlations, provide a prominent approach to alleviating this computational burden. However, GP-pro models often produce overestimated posterior variances due to training local experts on disjoint subsets of the data.

Objectives: This study aims to analyse the overestimation of posterior variances in GP-pro models and to address this issue using an information-based method.

Methods: We propose GP-pro-c, a product-of-experts Gaussian process model that incorporates an information-based method to calibrate the overestimated posterior variances. The method uses the monotonicity and submodularity properties of information gain in GP to define a calibration ratio that appropriately reduces the posterior variance of individual local GP models.

Results: The performance of GP-pro-c was evaluated using three metrics: negative log-likelihood (NLL), root mean squared error (RMSE), and expected normalised calibration error (ENCE) of the uncertainty estimates. Empirical experiments on four synthetic functions and six regression datasets show that the proposed calibration method enables the GP-pro-c model to achieve average reductions of 2.3% and 12.0% in NLL and ENCE, respectively, compared to the uncalibrated GP-pro model.

Conclusions: The results demonstrate that GP-pro-c effectively mitigates the overestimation of posterior variances in product-of-experts Gaussian process models while maintaining predictive accuracy and reducing computational complexity. These findings suggest that the proposed information-based calibration method is a promising approach for improving uncertainty estimation in scalable GP models. We expect the GP-pro-c to serve as a useful surrogate model in Bayesian optimisation settings involving high-dimensional and large-scale data.