I usually happily avoid statistics in my work, but of course optimisation (in terms of extremising action functionals) plays an important role in theoretical physics. Here the extrema are given as solutions to PDE’s, which in many (even linear) cases also lack closed form solutions.