Content:

The aim of this course is to lay the relevant mathematical backgrounds in the area of numerical methods used in Artificial Intelligence (AI). Special focus in the course will be in training the practical skills, by implementing and applying these numerical methods in the guided practical projects dealing with real life learning problems.

The topics include:

• introduction to AI numerics, stochastic vs. deterministic numerics of learning algorithms (stochastic vs. deterministic gradient descent and backpropagation, complexity and convergence estimates);
• numerics in biological, neurophysiological and other mathematical models of living neural networks (involving relevant concepts from numerics of ODEs and SDEs), numerics of Spiking Neural Networks (SNN);
• numerics of Euclidean ML/AI beyond first order numerical methods (Newton, interior point methods and beyond);
• constrained vs. unconstrained AI numerics (learning based on unconstrained convex optimization vs. LP and QP-based learning methods);
• numerics of Markovian and Bayesian learning, efficient algorithms for agent-based learning methods, numerics of decision-support systems (random forests and integer programming);
• CPU vs. GPU numerics of AI in Julia and MATLAB;
• numerics of entropic learning methods: Scalable Probabilistic Approximation and beyond.

Contact hours:

4 SWS / 56 h Vorlesung
2 SWS / 28 h Übung

Prerequisites (Contents):

A profound knowledge and skills in the following areas of mathematics are required:

(a) analysis (derivatives, gradients, Jacobians, Hesse matrix, chain rule, Leibnitz notation, relevant concepts from multivariate calculus and optimization with and without constraints),

(b) linear algebra (vectors, matrices, eigen and singular-value decomposition, calculation rules with matrices and vectors, linear solvers).

(c) probability theory and statistics (basics of probability theory, properties of expectation and (co-)variance, central limit theorems, (log-)likelihood and basic parameter estimation with maximum log-likelihood methods).

Optional requirements (nice to have):

It is desirable (but not mandatory) that students participating in this module have basic knowledge and experience in programming with one out of the following three languages: MATLAB, Python, Julia. We recommend MATLAB. Helpful (but not mandatory) is also the previous participation in the module [MAT-63-10-M-7] "Mathematical Methods in AI".

Frequency of occurence:

The lecture is irregularly given.

Link to KIS: [KIS]

OLAT

Contents

The course introduces the relevant mathematical concepts and methods in the field of artificial intelligence (AI) and conveys the practical skills to apply these methods in guided practical projects dealing with real life data and issues.The recurrent theme is in establishing a joint stochastic/statistic perspective based on optimization paradigm and complexity estimates - for various mathematical methods and algorithms deployed in machine learning (ML) and AI.

The topics covered include:

• Introduction, biological, neurophysiological and mathematical foundations of living neural networks, mathematical implications of differences between living and common synthetic classes of artificial neural networks;
• Euclidean, non-Euclidean and information-theoretical measures in ML/AI;
• transformation of measures, embedding theorems (Whitney and Takens);
• deterministic vs. stochastic approximation of functions and integrals;
• deterministic vs. stochastic unconstrained optimization methods in AI;
• constrained optimization methods in AI, soft vs. hard constraints, necessary concepts from linear and quadratic programming;
• mathematical foundations of Markovian and Bayesian learning;
• dimension reduction and feature selection in ML/AI as an optimization problem for various measures;
• ill-posedness and “small data learning challenge”, information-theoretic vs. more common (L1/L2) regularizations in learning problems;
• mathematical concepts in eXplainable AI (XAI), Scalable Probabilistic Approximation family of learning methods and its extensions.

Contact Hours:

4 SWS / 56 h Lectures
2 SWS / 28 h Exercise Classes

Prerequisites (Contents):

A profound knowledge and skills in the following areas of mathematics are required:

• Analysis
• Linear Algebra
• Probability Theory and Statistics

Optional requirements (nice to have):

It is desirable (but not mandatory) that students participating in this course have basic knowledge and experience in programming with one out of the following three languages: MATLAB, Python, Julia. We recommend MATLAB.

Helpful for the course would also be mastering principles of more advanced Frechet-calculus with matrices (from the "matrix cookbook").

Frequency of occurence:

The lecture is irregularly given.

Link to KIS: [KIS]

OLAT-Kurs