We scale to deep neural networks using Variational Inference (VI). To obtain a sufficiently tight bound for model selection, we avoid mean-field and instead use efficient matrix normal posteriors and closed-form updates of prior and output variance (see tricks in App. D). 🧵5/16 - ThreadSky

tychovdo.bsky.social • 84 days ago

We scale to deep neural networks using Variational Inference (VI). To obtain a sufficiently tight bound for model selection, we avoid mean-field and instead use efficient matrix normal posteriors and closed-form updates of prior and output variance (see tricks in App. D). 🧵5/16

Comments

tychovdo.bsky.social•84 days ago

As far as we know, this is the first case in which Bayesian model selection with VI is successfully scaled to deep neural networks, an achievement in its own right, whereas most works so far have relied on Laplace approximations. 🧵6/16

tychovdo.bsky.social•84 days ago

Noether's razor jointly learns conservation laws and the symmetrised Hamiltonian on train data, without requiring validation data. By relying on differentiable model selection, we do not introduce additional regularizers that require tuning.🧵7/16

tychovdo.bsky.social•84 days ago

Our method discovers the correct symmetries from data. Learned Hamiltonians that obey the right symmetry generalise better as they remain more accurate in larger areas of the phase space, depicted here for a correctly learned SO(2) on a simple harmonic oscillator. 🧵8/16

tychovdo.bsky.social•84 days ago

We verify the correct symmetries and group dimensionality are learned by inspecting parallelism, singular vectors, and transformations associated with learned generators. For instance, we correctly learn the n² dimensional unitary Lie group U(n) on N-harmonic oscillators.🧵9/16

tychovdo.bsky.social•84 days ago

By learning the correct symmetries, the jointly learned Hamiltonians are more accurate, directly improving trajectory predictions at test time. We show this for n-harmonic oscillators, but also more complex N-body problems (see table below). 🧵10/16

tychovdo.bsky.social•84 days ago

On more complex N-body problems, our method correctly discovers the correct 7 linear generators which correspond to the correct linear symmetries of rotation around center of mass, rotation around the origin, translations, and momentum-dependent translations. 🧵11/16

Comments

Posting Rules

Reply