The explosion of the use of machine learning (ML) algorithms on unstructured high dimensional data has brought renewed attention to learning from finite and sparse data for decision/intervention purposes. In the traditional control setting, we would like to understand ‘what’ can be learned from an unstructured system in ‘finite’ data-in particular, can we relate what we learn to a lower dimensional model approximation. Moreover, can this help us understand the question of adaptation, particularly for unstable or mission critical systems.
In the context of intervention of many parallel agents (examples include drug administration, agricultural interventions, marketing), where only sparse and noisy data is available under limited intervention. In this context, we would like to come up with a statistically robust way of estimating the impact of a given intervention for a given agent. This falls under the question of causal inference where it overlaps with various methodologies including instrumental variables and synthetic control in econometrics and matrix completion methods in engineering.
Finally, an important question from control and AI perspectives, is a problem known as batch learning. In this context, there are many input/output trajectories in dynamical systems that are available. However, these are collected under unknown strategies (involving feedback). Reinforcement learning requires the presence of a simulator that can produce the output for any possible strategy. Can one derive a simulator from the set of batch data?
Recent work has also renewed interest in Hidden Markov Models (HMMs) and Markov Decision Processes (MDP) as the prototypical description of a stochastic process with latent variables. This research focuses on developing a foundational theory for model reduction of HMMs and MDPs and on connecting model reduction to statistical learning theory. Ultimately, this will have a large impact on AI with particular benefits to reinforcement learning. Of course, learning from Batch Data extends to this setting as well.