Deep Learning & NLP

Expressivity & Universal Approximation of Neural Nets

Universal approximation theorems, depth-width trade-offs, and expressivity of modern architectures.

Overparameterization, NTK & Mean-Field Limits

Analyze overparameterized networks via neural tangent kernel and mean-field theory.

Implicit Bias of SGD & Loss-Landscape Geometry

How optimization algorithms implicitly regularize: margin maximization, flat minima, and edge of stability.

Generalization & Double Descent in Deep Nets

Modern generalization theory: interpolation, double descent, and benign overfitting in deep learning.

Invariance, Equivariance & Group-Theoretic Representations

Design neural architectures with built-in symmetries using group representation theory.

Transformer Theory & Sequence Modeling

Theoretical analysis of transformer architectures: attention mechanisms, positional encoding, and expressivity.

Convolutional & Spectral Networks

Theory of CNNs: translation equivariance, spectral methods, and connections to scattering transforms.

Regularization, Flat Minima & Sharpness-Aware Theory

Regularization techniques and their connection to flat minima for improved generalization.

Adversarial Robustness & Certified Defenses

Theory of adversarial examples, robustness certificates, and certified defenses for neural networks.

Lottery Ticket Hypothesis & Pruning Theory

Sparse subnetworks, lottery tickets, and theoretical foundations of neural network pruning.

Compression, Quantization & Information Bottleneck

Neural network compression theory: quantization, distillation, and information-theoretic perspectives.

Multimodal Representation Learning

Theory of learning joint representations across text, images, and other modalities.