Motivated by (approximate) dynamic programming and model predictive control problems, we analyse the stability of deterministic nonlinear discrete-time systems whose inputs minimize a discounted finite-horizon cost. We assume that the system satisfies stabilizability and detectability properties with respect to the stage cost. Then, a Lyapunov function for the closed-loop system is constructed and a uniform semiglobal stability property is ensured, where the adjustable parameters are both the discount factor and the horizon length, which corresponds to the number of iterations for dynamic programming algorithms like value iteration. Stronger stability properties such as global exponential stability are also provided by strengthening the initial assumptions. We give bounds on the discount factor and the horizon length under which stability holds and we provide conditions under which these are less conservative than the bounds of the literature for discounted infinite-horizon cost and undiscounted finite-horizon costs, respectively. In addition, we provide new relationships between the optimal value functions of the discounted, undiscounted, infinite-horizon and finite-horizon costs respectively, which are very different from those available in the approximate dynamic programming literature. These relationships rely on assumptions that are more likely to be satisfied in a control context. Finally, we investigate stability when only a near-optimal sequence of inputs for the discounted finite-horizon cost is available, covering approximate value iteration as a particular case.

For certain industrial control applications an explicit function capturing the non-trivial trade-off between competing objectives in closed loop performance is not available. In such scenarios it is common practice to use the human innate ability to implicitly learn such a relationship and manually tune the corresponding controller to achieve the desirable closed loop performance. This approach has its deficiencies because of individual variations due to experience levels and preferences in the absence of an explicit calibration metric. Moreover, as the complexity of the underlying system and/or the controller increase, in the effort to achieve better performance, so does the tuning time and the associated tuning cost. To reduce the overall tuning cost, a tuning framework is proposed herein, whereby a supervised machine learning is used to extract the human-learned cost function and an optimisation algorithm that can efficiently deal with a large number of variables, is used for optimising the extracted cost function. Given the interest in the implementation across many industrial domains and the associated high degree of freedom present in the corresponding tuning process, a Model Predictive Controller applied to air path control in a diesel engine is tuned for the purpose of demonstrating the potential of the framework.

Constant-force contact-mode atomic force microscopy (AFM) relies on a feedback control system to regulate the tip–sample interaction during imaging. Due to limitations in actuators and control, the bandwidth of the regulation system is typically small. Therefore, the scan rate is usually limited in order to guarantee a desirable image quality for a constant-rate scan. By adapting the scan rate online, further performance improvement is possible, and the conditions to this improvement have been explored qualitatively in a previous study for a wide class of possible scan patterns. In this article, a quantitative assessment of the previously proposed adaptive scan scheme is investigated through experiments that explore the impact of various degrees of freedom in the algorithm. Further modifications to the existing scheme are proposed and shown to improve the closed-loop performance. The flexibility of the proposed approach is further demonstrated by applying the algorithm to tapping-mode AFM.

Periodic event-triggered control (PETC) is an appealing paradigm for the implementation of controllers on platforms with limited communication resources, a typical example being networked control systems. In PETC, transmissions over the communication channel are triggered by an event generator, which depends solely on the available plant and controller data and is only evaluated at given sampling instants to enable its digital implementation. In this paper, we consider the general scenario, where the controller communicates with the plant via multiple decoupled networks. Each network may contain multiple nodes, in which case a dedicated protocol is used to schedule transmissions among these nodes. The transmission instants over the networks are asynchronous and generated by local event generators. At given sampling instants, the local event generator evaluates a rule, which only involves the measurements and the control inputs available locally, to decide whether a transmission is needed over the considered network. Following the emulation approach, we show how to design local triggering generators to ensure input-to-state stability and $\mathcal {L}_p$ stability for the overall system based on a continuous-time output-feedback controller that robustly stabilizes the network-free system. The method is applied to a class of Lipschitz nonlinear systems, for which we formulate the design conditions as linear matrix inequalities. The effectiveness of the scheme is illustrated via simulations of a nonlinear example.

Value iteration is a method to generate optimal control inputs for generic nonlinear systems and cost functions. Its implementation typically leads to approximation errors, which may have a major impact on the closed-loop system performance. We talk in this case of approximate value iteration (AVI). In this paper, we investigate the stability of systems for which the inputs are obtained by AVI. We consider deterministic discrete-time nonlinear plants and a class of general, possibly discounted, costs. We model the closed-loop system as a family of systems parameterized by tunable parameters, which are used for the approximation of the value function at different iterations, the discount factor and the iteration step at which we stop running the algorithm. It is shown, under natural stabilizability and detectability properties as well as mild conditions on the approximation errors, that the family of closed-loop systems exhibit local practical stability properties. The analysis is based on the construction of a Lyapunov function given by the sum of the approximate value function and the Lyapunov-like function that characterizes the detectability of the system. By strengthening our conditions, asymptotic and exponential stability properties are guaranteed.

We study a class of distributed optimization problems of minimizing the sum of potentially non-differentiable convex objective functions (without requiring strong convexity). A novel approach to the analysis of asynchronous distributed optimization is developed. An iterative algorithm based on dual decomposition and block coordinate ascent is implemented in an edge based manner. We extend available results in the literature by allowing multiple and potentially overlapping blocks to be updated at the same time with non-uniform probabilities assigned to different blocks. Sublinear convergence with probability one is proved for the algorithm under the aforementioned weak assumptions. A numerical example is provided to illustrate the effectiveness of the algorithm.

We propose a novel triggering policy to implement state-feedback controllers for nonlinear systems via packet-based communication networks. The idea is to generate transmissions between the plant and the controller only when a state-dependent rule has been satisfied for a given amount of time. We refer to this new paradigm as event-holding control, in which a clock variable is thus only running when a state-dependent criterion is verified. This is different from time-regularized event-triggered control, where the clock variable keeps running after each transmission instant until it is reset to zero at the moment a state-based condition is verified. We approach the problem of designing an event-holding controller via emulation. We first synthesize a state-feedback law, which stabilizes the closed-loop system in the absence of the communication network. We then design the event-holding triggering mechanism under a set of general assumptions. The results are applied to two case studies consisting of linear systems and a class of nonlinear systems controlled by backstepping. We also provide a numerical backstepping control example, which demonstrates that the event-holding behaviour can reduce the number of transmissions.

Most emulation-based results in networked control systems rely on a bound on the maximal allowable transmission interval (MATI) under which stability is preserved. However, having only such a MATI condition can lead to conservative results, as large values of transmission intervals may only occur sporadically, while the typical transmission interval is much smaller. In this paper, we therefore propose, in addition to the existence of a MATI, to also impose a bound on the average allowable transmission interval, expressed in terms of a reverse average dwell-time (RADT) condition on the transmission intervals. We provide joint conditions on the RADT and the MATI such that stability of the NCS can still be guaranteed, which can, in addition, lead to significant higher values of the MATI itself. The strengths of these new results are illustrated on a numerical example, showing a 484% improvement of the MATI, while still guaranteeing stability.

Most extremum-seeking control approaches focus solely on the problem of finding the extremum of some unknown, steady-state performance map. However, many industrial applications also have to deal with constraints on operating conditions due to, e.g., actuator limitations, limitations on design or tunable system parameters, or constraints on measurable signals. These constraints, which can be unknown a-priori, may conflict with the otherwise optimal operational condition, and should be taken into account in performance optimization. In this work, we propose a sampled-data extremum-seeking approach for optimization of constrained dynamical systems using barrier function methods, where both the objective function and the constraint function are available through measurement only. We show that, under the assumption that initialization does not violate constraints, the interconnection between a constrained dynamical system and optimization algorithms that employ barrier function methods is stable, the constraints are satisfied, and optimization is achieved. We illustrate the results by means of a numerical example.

The electrical properties of neural tissue are important in a range of different applications in biomedical engineering and basic science. These properties are characterized by the electrical admittivity of the tissue, which is the inverse of the specific tissue impedance. Objective. Here we derived analytical expressions for the admittivity of various models of neural tissue from the underlying electrical and morphological properties of the constituent cells. Approach. Three models are considered: parallel bundles of fibers, fibers contained in stacked laminae and fibers crossing each other randomly in all three-dimensional directions. Main results. An important and novel aspect that emerges from considering the underlying cellular composition of the tissue is that the resulting admittivity has both spatial and temporal frequency dependence, a property not shared with conventional conductivity-based descriptions. The frequency dependence of the admittivity results in non-trivial spatiotemporal filtering of electrical signals in the tissue models. These effects are illustrated by considering the example of pulsatile stimulation with a point source electrode. It is shown how changing temporal parameters of a current pulse, such as pulse duration, alters the spatial profile of the extracellular potential. In a second example, it is shown how the degree of electrical anisotropy can change as a function of the distance from the electrode, despite the underlying structurally homogeneity of the tissue. These effects are discussed in terms of different current pathways through the intra- and extra-cellular spaces, and how these relate to near- and far-field limits for the admittivity (which reduce to descriptions in terms of a simple conductivity). Significance. The results highlight the complexity of the electrical properties of neural tissue and provide mathematical methods to model this complexity.