Behzad Farzanegan, a doctoral candidate in electrical engineering, will defend their dissertation titled “Lifelong Safe Optimal Adaptive Tracking Control of a Class of Nonlinear Discrete-time Systems.” Their advisor, Dr. Jagannathan Sarangapani, is a curators distinguished professor in the electrical and computer engineering department. The dissertation abstract is provided below.

State constraints and uncertainties are inherent in many autonomous and robotic systems, posing significant challenges for ensuring safety, adaptability, and optimal control performance. The optimal adaptive tracking control of nonlinear discrete-time (DT) systems, particularly in safety-critical applications, necessitates reinforcement learning-based solutions that can continuously learn and adapt while maintaining safety. Therefore, in this research, a suite of novel lifelong safe optimal adaptive tracking control (LSOATC) techniques is developed for nonlinear DT systems with uncertain dynamics, leveraging deep reinforcement learning (DRL) and actor-critic neural network architectures.

First, the optimal tracking control of partially uncertain nonlinear DT systems is addressed using a zero-sum game (ZSG) formulation, wherein an augmented system is designed to incorporate the tracking error and its integral value. A barrier function (BF) is incorporated into the cost function to enforce state constraints, ensuring that system trajectories remain within a safe set while optimizing performance. The actor-critic framework is employed to approximate the optimal control policy and worst-case disturbance, while a lifelong learning scheme is introduced to mitigate catastrophic forgetting.

Next, the optimal adaptive tracking problem is extended to multi-task safe optimal adaptive tracking (MSOAT) for nonlinear DT systems in strict-feedback form. A Hamilton-Jacobi-Bellman (HJB)-based reinforcement learning framework is developed, integrating a time-varying control barrier function (CBF) for real-time safety enforcement. To address catastrophic forgetting in multi-task learning, an online Elastic Weight Consolidation (EWC)-based regularization term is introduced in the critic and actor update laws, enhancing stability and adaptability across multiple tasks.

Subsequently, an explainable deep reinforcement learning-based safety-aware optimal adaptive tracking (SOAT) framework is proposed for nonlinear DT affine systems subject to state constraints. The higher-order control barrier function (HOCBF) and Karush-Kuhn-Tucker (KKT) conditions are incorporated into the optimal policy derivation to ensure safe exploration during both online learning and steady-state operation. Furthermore, the Shapley Additive Explanations (SHAP) method is utilized to provide interpretability, identifying key features that influence control decisions and enabling efficient neural network architecture design.

To enhance robustness, a safe lifelong learning (SLL)-based trajectory tracking controller is developed for autonomous surface vessels (ASV) using deep multilayer neural networks (MNN). The proposed controller employs an MNN-based observer to estimate uncertain system dynamics and a singular value decomposition (SVD)-based tuning mechanism to mitigate vanishing gradient issues. The lifelong learning-based approach improves adaptability in varying system dynamics, ensuring sustained optimal performance.

Finally, a resilient DRL-based control strategy is introduced to counteract sensor, actuator and reward adversarial attacks in autonomous systems. An MNN-based observer is designed to detect attack residuals, enabling real-time anomaly detection in networked communication. Safety is enforced using a CBF-constrained quadratic programming (QP) formulation, while adaptive reward clipping and Gaussian-based forgetting factors mitigate the impact of adversarial reward perturbations.

The proposed methodologies are rigorously validated through extensive simulations on autonomous surface vessels (ASV), autonomous underwater vehicles (AUV), shipboard power systems (SPS), and rear-wheel-drive autonomous (RWDA) vehicles. The results demonstrate significant improvements in tracking accuracy, safety enforcement, and robustness to adversarial perturbations compared to conventional actor-critic-based controllers. This research advances the field of lifelong safe optimal adaptive tracking control, providing a foundation for the real-world deployment of reinforcement learning-based controlle