by Guy Y. Cornejo Maceda, François Lusseyran (LIMSI, CNRS, Université Paris-Saclay) and Bernd R. Noack (Harbin Institute of Technology)
Machine learning control is a model-free method based on artificial intelligence techniques to build optimal control laws exploiting non-linear dynamics in an unsupervised way. It is a game changer in discovering new dynamics for experiments and real-life applications.
The idea of controlling fluid flows is not a new one, but it is currently gaining momentum. The intense activity in this area is a consequence of both the technical developments of sensors and actuators and the advances in automatic learning methods. These methods hold potential to address many engineering problems in fields such as transport, industry and energy production. By manipulating the flow, performance can be improved and energy consumption reduced.
There are three main categories of flow manipulation: aerodynamic shape optimisation, passive control and active control. While aerodynamic shaping aims to optimise the shape of the system, passive control adds features to further increase performance, e.g., turbulators on wings and spoilers on sport cars, which are often accompanied by drawbacks such as drag increase. Active control on the other hand, uses actuators, for example Coanda blowers, small jets, plasma actuators and rotating blades, to introduce energy to the flow. Such control requires a positive balance between the energy injected and the energy recuperated but sensors make it possible for these techniques to be adaptable to the flow conditions; this is referred to as closed-loop control.
By adding feedback, closed-loop control tackles inherent challenges for control such as robustness to various conditions, rejection of noise due to the environment and model uncertainty reduction. Closed-loop control can offer enormous benefits: lift increase for drag reduction, gust mitigation, mixing enhancement and noise reduction, to name a few. However, predicting the proper control command is a difficult problem, relying on knowledge of the flow that is usually inaccessible in real time due to the time-delayed response, frequency crosstalk and the inherent high-dimensionality of the flow. Moreover, only partial information about the dynamics is accessible with non-intrusive sensors. Finally, there is currently no precise way to determine the number, location and kind of sensors and actuators required in a closed-loop system; this decision is guided by engineering wisdom.
Linear control theory may have some answers but it calls upon restrictive linear hypotheses that discard non-linear interaction between modes, which is often essential for fluid control. Indeed, examples of wake stabilisation with high-frequency forcing and low-frequency forcing show that frequency crosstalk is a key enabler for control. Thus, it is essential to consider a model-free approach that considers dynamics in its entirety, both linear and non-linear aspects, such as in turbulence [1]. Building an adequate control that makes use of non-linear interactions between modes, exploits time-delays and takes advantage of the physical properties of turbulence is not an easy task and relies on recent advances in machine learning and artificial intelligence. The main approach to solving this near-inextricable control problem is to formulate it as a function optimisation problem where the function to optimise is the control law itself, according to a cost function to minimise. The control law is a mathematical expression function of flow sensors and time-dependent functions such as periodic functions, that commands the actuators. The cost function is a measure of the performance of the control law regarding the objective, e.g., drag power, lift fluctuations, which are defined so as to determine a minimum for the optimal solution. The resulting optimisation problem is another non-convex optimisation problem, which is difficult, but much more accessible to machine learning/artificial intelligence techniques.
Machine learning control (MLC, [2]) is precisely a machine learning solver that solves such hard non-convex optimisation problems and thus builds a controller in a model-free approach. It builds on the pioneering work of Dracopoulous [3] on genetic programming to build a map between the actuation command and the sensors. The learning process mimics the Darwinian principle of survival of the fittest to build fitter control laws by testing successive control laws in an unsupervised way.
Indeed a fast evaluation loop is repeated several times to test candidate solutions, and MLC combines them to produce new ones (slow learning loop), based on the performance of past control laws (Figure 1). The strength of MLC relies on its ability to build non-linear control laws reproducing known control methods including model-based (ERA/OKID), open-loop strategies (multi-frequency forcing), closed-loop strategies (phasor control, ARMAX) and also linear and non-linear combinations of them. It can virtually build any function/control law in the control law space.
Figure 1: Machine learning control learning process. A fast evaluation loop is used to test the candidate’s control laws and a slow learning loop builds new control laws from previous ones, based on their performance.
The aim of this project is to accelerate the learning process of MLC through a smart exploration of the control law space. For this purpose, our MLC code [L1] is benchmarked on a cluster of three equidistant cylinders immersed in an incoming flow called the fluidic pinball [L2]. Thanks to the independent rotation of the cylinders, the fluidic pinball can reproduce up to six wake suppression mechanisms comprising frequency crosstalk mechanisms. MLC achieved a 46.0% net drag reduction of the fluidic pinball by successfully combining two strategies from literature: boat-tailing and phasor control without imposing this prior knowledge in the automated optimization. The unforced flow is shown in Figure 2 and the MLC solution in Figure 3. Current advances show an improvement of the learning rate by a factor of three and with future work, we aim to achieve a factor 10 improvement. Such an acceleration will benefit a multitude of experiments by enabling multi-parameter testing for experiments and simulations with the potential to reveal hidden control mechanisms unreachable with model-based approaches. This approach is a change in the classical paradigm where control laws are derived from the analysis of the system. The machine learning paradigm instead starts from the optimal solution and understanding the mechanisms at play comes afterwards.
Figure 2: Vorticity field for the unforced natural flow at Reynolds number 100. A grid of 15 sensors downstream with four delays (t, t-T/4, t-/2, t-3T/4), for a total of 60 sensor signals, have been chosen to reduce the net drag power with MLC, with t representing the natural period of vortex shedding. Simulations have been carried out with a DNS solver provided by Marek Morzynski of Poznan University.
Figure 3: Vorticity field for the best control law found with MLC; a combination of boat-tailing (inward symmetric rotation of the two back cylinders) and phasor control for the front cylinder. This solution reduces the net drag power by 46% and uses only one of the 15 sensors, in this sense, MLC acts as a sensor optimiser.
Links:
[L1] https://www.cornejomaceda.com/
[L2] http://berndnoack.com/
References:
[1] Y. Zhou et al.: “Artificial intelligence control of a turbulent jet”, J. Fluid Mech. 2020 in print.
[2] T. Duriez , S.L. Brunton, B. R. Noack: “Machine Learning Control Taming Nonlinear Dynamics and Turbulence”, Springer, 2017.
[3] D. Dracopoulos: “Evolutionary Learning Algorithms for Neural Adaptive Control”, Springer 1997.
Please contact:
Bernd R. Noack,Harbin Institute of Technology (Shenzhen), China
