The Seminars in Mathematical Foundations of Artificial Intelligence are open to employees and students of 91传媒在线.
Subscribe to mathfoundations-ai-seminar@lists.91传媒在线 for notification of future seminars.
Send an email to sympa@lists.91传媒在线 and in the subject/heading of the email write: subscribe mathfoundations-ai-seminar
Leave the email blank.
18 November 2025, 16:30-17:30
Speaker: Tolga Birdal, Imperial College
Abstract: Training deep learning models involves searching for a good model over the space of possible architectures and their parameters. Discovering models that exhibit robust generalization to unseen data and tasks is of paramount for accurate and reliable machine learning. Generalization, a hallmark of model efficacy, is conventionally gauged by a model's performance on data beyond its training set. Yet, the reliance on vast training datasets raises a pivotal question: how can deep learning models transcend the notorious hurdle of 'memorization' to generalize effectively? Is it feasible to assess and guarantee the generalization prowess of deep neural networks in advance of empirical testing, and notably, without any recourse to test data? This inquiry is not merely theoretical; it underpins the practical utility of deep learning across myriad applications. In this talk, I will show that scrutinizing the training dynamics of neural networks through the lens of topology, specifically using 'persistent-homology dimension', leads to novel bounds on the generalization gap and can help demystifying the inner workings of neural networks. Our work bridges deep learning with the abstract realms of topology and learning theory, while relating to information theory through compression.
Place: MIT.A.356
29 October 2025, 15:30-16:30
Speaker: Sebastian Mair, LiU
Abstract: Representation learning is often only seen from the perspective of changing the representation of each and every data point. In this talk, we consider an orthogonal perspective on representation learning and take a closer look at the representation of data in terms of its sample size. Using less but informative data points allows us to learn approximately the same model with high probability in less time; thus lowering the computational burden of model training. This translates to lower electricity consumption, reducing costs and carbon footprint. Using matrix factorization as an example, we first consider a geometric construction that has connections to well-known optimization methods. Then, we quantify data importance using straightforward statistics yielding a representative subset with approximation guarantees. We then leverage the same idea for efficient model initialization. Finally, we demonstrate using similar principles how to use importance scores within privacy amplification and how to accelerate radiation treatment planning.
Place: MIT.A.356
2025
15 October: Learning in the overparametrized regime with polynomial neural networks
Speaker: Kathlén Kohn, KTH
Abstract: This talk will be very interactive. The audience will explore neuromanifolds and why their geometry governs the learning behavior of neural networks. Since studying their geometry for real-life networks seems out of reach for current mathematics, we approximate arbitrary neural networks by polynomial ones. We will motivate this choice by turning the universal approximation theorem upside down. This then allows us to argue that training polynomial networks essentially means solving a closest-point problem on a neuromanifold in a finite dimensional ambient space. We then focus on the highly overparametrized setting common in modern large language models where the distance of the closest-point problem is degenerate and present preliminary results on formally comparing the minimization of degenerate distances with non-degenerate distance minimization. The preliminary results come from joint work with Paul Breiding, Erin Conelly, and Giovanni Marchetti.
9 September: Flexible block-iterative analysis for the Frank-Wolfe algorithm
Speaker: Zev Woodstock, James Madison University
Abstract: We prove that the block-coordinate Frank-Wolfe (BCFW) algorithm converges with state-of-the-art rates in both convex and nonconvex settings under a very mild “block-iterative” assumption, newly allowing for (I) progress without activating the most-expensive linear minimization oracle(s), LMO(s), at every iteration, (II) parallelized updates that do not require all LMOs, and therefore (III) deterministic parallel update strategies that take into account the numerical cost of the problem’s LMOs. Our results apply for short-step BCFW as well as an adaptive method for convex functions. New relationships between updated coordinates and primal progress are proven, and a favorable speedup is demonstrated using FrankWolfe.jl. If time permits, we will also discuss BCFW's relationship to the topic of Frank-Wolfe splitting algorithms.
14 May: Workshop
13:00 - 13:30: Aron Persson, Uppsala University
Title: A Glimpse of Intrinsic Statistics on Manifolds with an Application to Signal Processing on the Sphere
Abstract: This talk provides an introduction and overview of intrinsic statistics on manifolds, concluding with an application to signal processing on the sphere.
13:45 - 14:15: Fredrik Ohlsson, Umeå University
Title: Neural ODEs Beyond Manifolds
Abstract: An important class of models in geometric deep learning is based on the flows generated by vector fields. The ODEs corresponding to these flows describe the dynamics of information propagating through the model, and training amounts to learning the vector field that minimizes the loss in some machine learning application. Neural ODEs have been used to construct powerful generative models that can account for both the geometry and the symmetries of a problem, e.g., to generate 3D protein structures. In this talk I will discuss the current NODE framework on manifolds and in particular some open problems that prompt us to revisit the role of geometry in these deep learning models; adding more geometrical structure to the manifolds or even moving beyond the manifold concept itself.
14:30 - 15:00: Andreas Granath, Umeå University
Title: Low Rank Methods for Wave Dominated Problems
Abstract: Many problems of interest in engineering, ranging from electromagnetic to elastic and acoustic phenomena, can be modelled by wave equations. It is well known that solving wave equations accurately requires a high resolution in the spatial discretization. To decrease the overall computational cost of the discretization, we propose a low rank framework utilizing a singular value decomposition (SVD) to represent the numerical solution in 2D and tensor trains (TT) in 3D. Specifically, we focus on underwater acoustics, modelled by the Helmholtz equation in the frequency domain. We demonstrate how the SVD and TT representations can be combined with an iterative finite difference solver, decreasing the computational cost compared to a full rank solver, while maintaining the desired convergence and accuracy of the underlying numerical method.
15:00 - 15:30: FIKA
15:30 - 16:00: Oskar Nylén Nordenfors, Umeå University
Title: Group Equivariant Models and Training with Data Augmentation
Abstract: When training a neural network to perform a task with known symmetries, one would like the neural network to respect those symmetries, that is, to be equivariant. We compare two strategies for achieving equivariance: 1) restricting the network architecture to be equivariant by design; 2) training the neural network using data augmentation. We find the same set of equivariant stationary points for both strategies, and we see that the space of equivariant models is an invariant set of the augmented gradient flow. Analogously, for ensembles of neural networks we see that if the networks are invariantly initialized, then the ensemble mean is equivariant. In this talk I will highlight some of the techniques we used to obtain these results. The talk is based on joint work with Fredrik Ohlsson and Axel Flinth.
16:00 - 16:45: Emma Andersdotter Svensson, Umeå University
Title: A Bundle Formalism for Equivariant Neural ODEs
Abstract: Previous work by, e.g., M. Weiler et al has shown that using a bundle formalism to describe equivariant CNNs is a useful way to capture the underlying mathematical structures. There, feature maps are described as sections of an associated bundle of a homogeneous vector bundle. In my talk, I will introduce a way to use the bundle formalism for equivariant manifold neural ODEs – a neural network model where the network is defined by a vector field describing how the data evolves continuously with time. By considering neural ODEs on a homogeneous space, we can define a lift of the solution curve to an associated bundle, making it possible to extend previous formulations in terms of a parallel transport. I will include a description of how this formulation might be a generalization of our previous formulation of vector fields being transformed by neural ODEs through the pushforward.
11 April: From conformal modeling to O(n) equivariant deep learning
Speaker: Michael Felsberg, Linköping
Abstract: Felix Klein’s Erlangen Programme of 1872 introduced a methodology to unify non-Euclidean geometries. Within machine learning, geometric deep learning (GDL) constitutes a unifying framework for various neural network architectures and allows models to adjust to different geometric transformations. The orthogonal group O(n) fully encapsulates the symmetry structure of an nD sphere, including both rotational and reflection symmetries. Neurons with spherical decision surfaces — spherical neurons — can efficiently be constructed using a conformal embedding of Euclidean space and their activation transforms in an equivariant way under rotations. Replicating spherical neurons at the vertices of the regular simplex results in the steerable TetraSphere representation, useful in various applications, e.g. on pointclouds.
28 March: On the Geometry and Optimization of Polynomial Convolutional Networks
Speaker: Vahid Shahverdi, KTH
Abstract: In this talk, I will explore the rich interplay between algebraic geometry and convolutional neural networks (CNNs) with polynomial activation functions. At the heart of this study is the parameterization map, which translates network parameters into functions. We show that this map is a regular morphism and an isomorphism almost everywhere, up to scaling symmetries. The image of this map, which we call the “neuromanifold,” exhibits intricate geometric properties. I will discuss its dimension, degree, and singularities, and their implications for the learning process. Beyond structural insights, I will highlight a connection between the geometry of the neuromanifold and optimization: for large generic datasets, we compute the number of critical points that arise during training with a quadratic loss function, using tools from metric algebraic geometry. This is joint work with Giovanni Luca Marchetti and Kathlén Kohn.
13 March: Equivariant Neural Tangent Kernels
Speaker: Philipp Misof, Chalmers University of Technology
Abstract: In recent years, the neural tangent kernel (NTK) has proven to be a valuable tool to study training dynamics of neural networks (NN) analytically. In this talk, I will present how this NTK framework can be extended to equivariant NNs based on group convolutional NNs (GCNNs). Not only does this enable the analytic study of influences of hyperparameters, training biases etc. in equivariant NNs, but it also allows us to draw an interesting connection between data augmentation and manifestly equivariant architectures. In particular, we show that the mean predictions of an ensemble of data augmented non-equivariant networks coincide with the mean predictions of an ensemble of specific GCNNs at all training times in the infinite-width limit. We further provide explicit implementations of the equivariant NTK for roto-translations in the plane and 3d rotations. To evaluate the performance of the equivariant infinite width solution, we benchmark the models on quantum mechanical property prediction and medical image classification. This talk is based on joined work with Jan Gerken and Pan Kessel.
6 March: Learning-Based Surrogate Models for the Fluid Dynamics of a Pharmaceutical Bioreactor
Speaker: Umut Kaya, Daiichi Sankyo Europe GmbH / University of Ghent
Abstract: We developed learning-based surrogate models to predict fluid dynamics in pharmaceutical bioreactors. Traditional CFD simulations take too long to run, making real-time process optimization impractical. By using machine learning, including graph neural networks and reduced-order modeling, we built models that provide fast and accurate predictions of hydrodynamic stress and mixing behavior. These surrogate models significantly cut down computational costs while maintaining reliability, making them valuable for biopharmaceutical process development. Our work bridges the gap between physics-based modeling and data-driven approaches, helping improve bioprocess design, monitoring, and control.
13 February: Communication-efficient distributed optimization algorithms
Speaker: Laurent Condat, King Abdullah University of Science and Technology (KAUST)
Abstract: In distributed optimization and machine learning, a large number of machines perform computations in parallel and communicate back and forth with a distant server. Communication can be costly and slow, in particular in federated learning. To address the communication bottleneck, two strategies are popular: 1) communicate less frequently; 2) compress the communicated vectors. Also, a robust algorithm should allow for partial participation. I will present several randomized algorithms we developed recently in this area, with proved convergence guarantees and state-of-the-art communication complexity.
