Retour aux activités
Séminaire informel de théorie des systèmes (ISS)

Fully Adaptive Regret-Guaranteed Algorithm for Control of Linear Quadratic Systems

iCalendar

20 sept. 2024   10h00 — 11h00

Jafar Abbaszadeh Chekan University of Illinois at Urbana-Champaign, États-Unis

Jafar Abbaszadeh Chekan

Séminaire hybride à l'Université McGill ou Zoom.

The first algorithm for the Linear Quadratic (LQ) control problem with an unknown system model, featuring a regret of O(√T ), was introduced by Abbasi-Yadkori and Szepesvári (2011). Recognizing the computational complexity of this algorithm, subsequent efforts have been dedicated to proposing algorithms that are computationally tractable while preserving this order of regret. Although successful, the existing works in the literature lack a fully adaptive exploration- exploitation trade-off adjustment and require a user-defined value, which can lead to overall regret bound growth with some factors. In this work, noticing this gap, we propose the first fully adaptive algorithm that controls the number of policy updates (i.e., tunes the exploration-exploitation trade-off) and optimizes the upper-bound of regret adaptively. Our proposed algorithm builds on the SDP-based approach of Cohen et al. (2019) and relaxes its need for a horizon-dependent warm-up phase by appropriately tuning the regularization parameter and adding an adaptive input perturbation. We further show that through careful exploration-exploitation trade-off adjustment there is no need to commit to the widely-used notion of strong sequential stability, which is restrictive and can introduce complexities in initialization.


Bio: Jafar Abbaszadeh Chekan is a Ph.D. candidate in the Department of Aerospace Engineering at the University of Illinois at Urbana-Champaign (UIUC), where he is currently working on safety-aware learning-based control systems toward his doctoral degree. He is also affiliated with the Decision & Control Group at the Coordinated Science Lab (CSL). Prior joining UIUC, he received his master's degree in Engineering Mechanics from Virginia Tech in 2018. His research interest generally lies in controls, learning, game theory and their intersections.

Peter E. Caines responsable
Aditya Mahajan responsable
Shuang Gao responsable
Borna Sayedana responsable
Alex Dunyak responsable

Lieu

Salle MC 437
CIM
Pavillon McConnell
Université McGill
3480, rue University
Montréal QC H3A 0E9
Canada

Organisme associé

Centre for intelligent machines (CIM)