Deep learning from a control vantage point

Key points

• It reviews several classes of models including feedforward and residual neural networks, transformers, large-language models, and selective state space models while explaining how control theory plays an important role in understanding their training, properties, or architecture design.1
• The article explains how residual neural networks can be interpreted as the discretization of a continuous-time control system thereby enabling the use of control theoretic techniques for its analysis.1
• Another point of contact between control theory and deep learning is the connection between backpropagation, the algorithm used to train neural networks, and optimal control.1
• Several open problems and directions for future research are discussed, including selective state-space models, a recent class of models inspired by the models used in control.1

Sources (1)

• [1]Deep learning from a control vantage point
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 11:11 PM

  This article offers an overview of recent developments in deep learning from a control vantage point.

  It reviews several classes of models including feedforward and residual neural networks, transformers, large-language models, and selective state space models while explaining how control theory plays an important role in understanding their training, properties, or architecture design.

Extractive summary: sentences quoted from the sources.