We study closed-loop stability and suboptimality for MPC and infinite-horizon optimal control solved using a surrogate model that differs from the real plant. We employ a unified framework based on quadratic costs to analyze both finite- and infinite-horizon problems, encompassing discounted and undiscounted scenarios alike. Plant-model mismatch bounds proportional to states and controls are assumed, under which the origin remains an equilibrium. Under continuity of the model and cost-controllability, exponential stability of the closed loop can be guaranteed. Furthermore, we give a suboptimality bound for the closed-loop cost recovering the optimal cost of the surrogate. The results reveal a tradeoff between horizon length, discounting and plant-model mismatch. The robustness guarantees are uniform over the horizon length, meaning that larger horizons do not require successively smaller plant-model mismatch.

We consider the adaptive control problem for discrete-time, nonlinear stochastic systems with linearly parameterised uncertainty. Assuming access to a parameterised family of controllers that can stabilise the system in a bounded set within an informative region of the state space when the parameter is well-chosen, we propose a certainty equivalence learning-based adaptive control strategy, and subsequently derive stability bounds on the closed-loop system that hold for some probabilities. We then show that if the entire state space is informative, and the family of controllers is globally stabilising with appropriately chosen parameters, high probability stability guarantees can be derived.

We study deterministic, discrete linear time-invariant systems with infinite-horizon discounted quadratic cost. It is well-known that standard stabilizability and detectability properties are not enough in general to conclude stability properties for the system in closed-loop with the optimal controller when the discount factor is small. In this context, we first review some of the stability conditions based on the optimal value function found in the learning and control literature and highlight their conservatism. We then propose novel (necessary and) sufficient conditions, still based on the optimal value function, under which stability of the origin for the optimal closed-loop system is guaranteed. Afterwards, we focus on the scenario where the optimal feedback law is not stabilizing because of the discount factor and the goal is to design an alternative stabilizing near-optimal static state-feedback law. We present both linear matrix inequality-based conditions and a variant of policy iteration to construct such stabilizing near-optimal controllers. The methods are illustrated via numerical examples.

We introduce TROOP, a tree-based Riccati optimistic online planner, that is designed to generate near-optimal control laws for discrete-time switched linear systems with switched quadratic costs. The key challenge that we address is balancing computational resources against control performance, which is important as constructing near-optimal inputs often requires substantial amount of computations. TROOP addresses this trade-off by adopting an online best-first search strategy inspired by A⋆, allowing for efficient estimates of the optimal value function. The control laws obtained guarantee both near-optimality and stability properties for the closed-loop system. These properties depend on the planning depth, which determines how far into the future the algorithm explores and is closely related to the amount of computations. TROOP thus strikes a balance between computational efficiency and control performance, which is illustrated by numerical simulations on an example.

We analyze the stability of general nonlinear discrete-time stochastic systems controlled by optimal inputs that minimize an infinite-horizon discounted cost. Under a novel stochastic formulation of cost-controllability and detectability assumptions inspired by the related literature on deterministic systems, we prove that uniform semi-global practical recurrence holds for the closed-loop system, where the adjustable parameter is the discount factor. Under additional continuity assumptions, we further prove that this property is robust.

In this article, we analyze the stability properties of stochastic linear systems in closed loop with an optimal policy that minimizes a discounted quadratic cost in expectation. In particular, the linear system is perturbed by both additive and multiplicative stochastic disturbances. We provide conditions under which mean-square boundedness, mean-square stability, and recurrence properties hold for the closed-loop system. We distinguish two cases, when these properties are verified for any value of the discount factor sufficiently close to 1, or when they hold for a fixed value of the discount factor in which case tighter conditions are derived, as illustrated in an example. The analysis exploits properties of the optimal value function, as well as a detectability property of the system with respect to the stage cost, to construct a Lyapunov function for the stochastic linear quadratic regulator problem.

We consider the problem of least squares parameter estimation from single-trajectory data for discrete-time, unstable, closed-loop nonlinear stochastic systems, with linearly parameterised uncertainty. Assuming a region of the state space produces informative data, and the system is sub-exponentially unstable, we establish non-asymptotic guarantees on the estimation error at times where the state trajectory evolves in this region. If the whole state space is informative, high probability guarantees on the error hold for all times. Examples are provided where our results are useful for analysis, but existing results are not.

We consider deterministic nonlinear discrete-time systems whose inputs are generated by policy iteration (PI) for undiscounted cost functions. We first assume that PI is recursively feasible, in the sense that the optimization problems solved at each iteration admit a solution. In this case, we provide novel conditions to establish recursive robust stability properties for a general attractor, meaning that the policies generated at each iteration ensure a robust <inline-formula><tex-math notation="LaTeX">$\mathcal {KL}$</tex-math></inline-formula>-stability property with respect to a general state measure. We then derive novel explicit bounds on the mismatch between the (suboptimal) value function returned by PI at each iteration and the optimal one. However, we show by a counterexample that PI may fail to be recursively feasible, disallowing the mentioned stability and near-optimality guarantees. We therefore also present a modification of PI so that recursive feasibility is guaranteed a priori under mild conditions. This modified algorithm, called PI<inline-formula><tex-math notation="LaTeX">$^{+}$</tex-math></inline-formula>, is shown to preserve the recursive robust stability when the attractor is compact. In addition, PI<inline-formula><tex-math notation="LaTeX">$^{+}$</tex-math></inline-formula> enjoys the same near-optimality properties as its PI counterpart under the same assumptions.

This paper presents a novel event-triggered control (ETC) design framework based on measured $\mathcal{L}_{p}$ norms. We consider a class of systems with finite $\mathcal{L}_{p}$ gain from the network-induced error to a chosen output. The $\mathcal{L}_{p}$ norms of the network-induced error and the chosen output since the last sampling time are used to formulate a class of triggering rules. Based on a small-gain condition, we derive an explicit expression for the $\mathcal{L}_{p}$ gain of the resulting closed-loop systems and present a time-regularization, which can be used to guarantee a lower bound on the inter-sampling times. The proposed framework is based on a different stability- and triggering concept compared to ETC approaches from the literature, and thus may yield new types of dynamical properties for the closed-loop system. However, for specific output choices it can lead to similar triggering rules as “standard” static and dynamic ETC approaches based on input-to-state stability and yields therefore a novel interpretation for some of the existing triggering rules. We illustrate the proposed framework with a numerical example from the literature.

Given a discounted cost, we study deterministic discrete-time systems whose inputs are generated by policy iteration (PI). We provide novel near-optimality and stability properties, while allowing for non-stabilizing initial policies. That is, we first give novel bounds on the mismatch between the value function generated by PI and the optimal value function, which are less conservative in general than those encountered in the dynamic programming literature for the considered class of systems. Then, we show that the systems in closed-loop with policies generated by PI are stabilizing under mild conditions, after a finite (and known) number of iterations.