Excited to share our ICLR 2025 oral "Residual Deep Gaussian Processes on Manifolds"! With @vabor112.bsky.social & @arkrause.bsky.social, we introduce manifold-to-manifold GPs that can be composed together, generalising deep GPs to manifolds. Applications include wind prediction & Bayes opt! 1/n - ThreadSky

Excited to share our ICLR 2025 oral "Residual Deep Gaussian Processes on Manifolds"!

With @vabor112.bsky.social & @arkrause.bsky.social, we introduce manifold-to-manifold GPs that can be composed together, generalising deep GPs to manifolds. Applications include wind prediction & Bayes opt! 1/n

Comments

wyrwalkacper.bsky.social•17 days ago

When Euclidean GPs struggle to model irregular functions, stacking them into a deep GPs can help. This works because points and vectors in Euclidean space can be identified, allowing a vector-valued GP’s output to serve as another’s input. But can we do this on manifolds? 2/n

wyrwalkacper.bsky.social•17 days ago

Not quite. On general manifolds, points and tangent vectors cannot be identified. We can, however, translate points in the direction of vectors using the exponential map. Thus, we define a manifold-to-manifold GP as a composition of a Gaussian vector field with this map. 3/n

wyrwalkacper.bsky.social•17 days ago

We can build deep GPs by stacking these layers. Each layer learns a translation of inputs, allowing incremental updates of hidden representations – just like the ResNet! In fact, on the Euclidean manifold, we recover the ResNet-inspired deep GP of Salimbeni & @deisenroth.bsky.social. 4/n

wyrwalkacper.bsky.social•17 days ago

With manifold GPs at every layer, we can leverage manifold-specific methods like intrinsic Gaussian vector fields and interdomain inducing variables to improve performance. 5/n

wyrwalkacper.bsky.social•17 days ago

Our model can also serve as a plug-and-play replacement for shallow manifold GPs in geometry-aware Bayesian optimisation. This can be especially useful for complex target functions, as we demonstrate experimentally. 6/n

wyrwalkacper.bsky.social•17 days ago

We test our model on the ERA5 dataset – interpolating wind on the globe from a set of points on a satellite trajectory. Our model outperforms baselines, yielding accurate and interpretable uncertainty estimates. An example predictive mean and variance is shown below. 7/n

Comments

Posting Rules

Reply