Using machine learning on problem solving vs. optimal control
With the emergence of artificial intelligence, people have begun freeing their hands by making machines work for them. Machine learning was first proposed around the 1950s by Donald Hebb based on a model of brain cell interaction and created for the purpose of problem solving.
Machine learning is a subfield of artificial intelligence defined as the capability of a machine to learn human behaviors. In the1960s, the second method of general mathematical theory of optimal control was proposed specifically for problem solving. The methods of machine learning and optimal control theory have similarities and differences, creating unique advantages for solving problems.
Because of its strong learning and decision-making abilities, machine learning is becoming more and more popular in the information age.
Through the late 1970s, it was considered a part of AI’s evolution, until it branched off to evolve on its own. “Now, machine learning has become a very important response tool for cloud computing and eCommerce, and is being used in a variety of cutting edge technologies.” (Keith, 2021).
Optimal control theory was proposed by L.S. Pontryagin in 1956, the key point of this theory being the Pontryagin maximum principle.
Richard Bellman’s equation shows up everywhere in the Reinforcement Learning literature, being one of the central elements of many Reinforcement Learning algorithms.
In summary, we can say that the Bellman equation decomposes the value function into two parts, the immediate reward plus the discounted future values. Mathematically we can define the Bellman Expectation Equation as follows: Let’s call this Equation 1. The above equation tells us that the value of a particular state is determined by the immediate reward plus the value of successor states when we are following a certain policy.
The mathematical theory of optimal control has addressed practical demands, especially in space flight dynamics and automatic control theories. Through evolution, the optimal control theory also contains the mathematical basis of the theory of controlled motions. By now, the mathematical theory of optimal control is closely linked with analytical mechanics. (A.B. Kurzhanskii)
Building on this history, the workings of machine learning and optimal control can be explained.
For machine learning, the main goal is training machines to make classifications or predictions based on a huge input of data by statistical or algorithms.
Machines can emulate human behavior when processing these inputs and provide the best way to solve problems. In general, machine learning can mimic humans in analyzing data and classifying it as a specific system or category.. If information is added, machines can automatically classify this new input. The calculation speed is fast and the results are high in accuracy.
Control theory can compare specifically with reinforcement learning, a subgroup within machine learning.
Reinforcement learning solves the same tasks as the control theory but using other methods. The ‘classic’ model-free reinforcement learning operates with the data itself, without making assumptions about the dynamics. However, ‘modern’ model-based reinforcement learning is in fact quite close to the classical control theory, extending its arsenal of computational tools.
The main goal of optimal control is to formulate and solve problems to identify the best solution for a dynamical process. The problems that use optimal control theory are usually based on a dynamic model and need a constrained approach to find the best solution based on their input parameters (usually a maximum or minimum point).
Now, optimal control is a mature theory that has been used in many mathematical or engineering fields, and the models used to conduct the optimal control have increased.
In conclusion, machine learning and optimal control are all useful tools to find the ‘best’ solution by using some models or theories. Also, machine learning and
optimal control are closely linked, since optimal control can also be applied in machine learning to find results.
Lastly, the Deep Learning Godfather Yann LeCun told Rebellion Research,
“There are two ways to optimize your predictions, there are two ways to approach this problem, the first one is reinforcement learning (machine learning), the other one is optimal control. In many situations the right thing to do is more akin to optimal control than to reinforcement learning (machine learning). It is much more efficient in many ways.”
Written by Sirui Hao & Edited by Lauren Kauppila
Reference:
IBM Cloud Education. (n.d.). What is machine learning? IBM. Retrieved March 25, 2022, from https://www.ibm.com/cloud/learn/machine-learning
Becerra, V. M. (n.d.). Optimal control. Scholarpedia. Retrieved March 25, 2022, from http://www.scholarpedia.org/article/Optimal_control
Optimal control, mathematical theory of. Encyclopedia of Mathematics. URL: http://encyclopediaofmath.org/index.php?title=Optimal_control,_mathematical_theor y_of&oldid=48051 https://www.dataversity.net/a-brief-history-of-machine-learning/ Foote, K. D. (2022, January 20). A brief history of machine learning. DATAVERSITY. Retrieved March 25, 2022, from https://www.dataversity.net/a-brief-history-of machine-learning/