Presentation on YouTube

Global unconstrained optimization of a non-convex function f in Euclidean space is a classical problem that is attracting considerable new attention in view of applications to the training of deep neural networks, especially for highly nonlinear, possibly non-smooth loss functions f in very high space dimension. We show a connection between this problem and a "critical" eikonal-type equation arising also in ergodic control. A solution v of the critical Hamilton-Jacobi equation is built by a small discount approximation as well as the long time limit of an associated finite horizon optimal control problem. Then v is represented as the value function of a control problem with target, whose optimal trajectories are driven by a differential inclusion describing the gradient descent of v. Such trajectories are proved to converge to the set of minima of f, and in some cases they are shown to reach such points in finite time.

Joint work with Hicham Kouhkouh (arXiv:2202.02561).