Presentation on YouTube.

In this presentation, we focus on the minimization of a smooth, non-convexe function. This framework covers many applications, including deep neural network training, which high computational cost has motivated the developpement of low precision hardware. Performing computation with low precisions, i.e. , with numbers represented with a small number of bits, enables to save computational effort at the expense of more computational errors. We introduce, in the determinist case, the Quadratic Regularization (R2) algorithm, a gradient descent algorithm with adaptive step size, and how it can be extended into a Multi-Precision (MPR2) version of itself. MPR2 adapts dynamically the precision level for the computations to spare as much computational effort as possible while controlling computational error to ensure convergence to a minimum. We highlight some of the pitfalls related to the use of variable precision and how MPR2 avoids them. The performance of MPR2 is illustrated over a bank of problems.