Research

My research interests lie in the areas of information theory, statistics, and deep learning. My current focus is on understanding the implicit regularization and stability properties of neural network optimization algorithms. I am also interested in information-theoretic aspects of control and unsupervised reinforcement learning.

In my PhD, I developed new methods to decompose information into parts that allow for a fine-grained analysis of how information is distributed over composite systems consisting of multiple interacting parts or subsystems. These methods are potentially useful in applications ranging from neuroscience and representation learning, to robotics, and cryptography.

See my Google Scholar page for an updated list of publications.

• Pradeep Kr. Banerjee and Guido Montúfar (2021)
  Information Complexity and Generalization Bounds
  Proceedings of the IEEE International Symposium on Information Theory
  arXiv | ISIT 2021 | Slides of a talk at Google DeepMind

• Hui Jin, Pradeep Kr. Banerjee and Guido Montúfar (2021)
  Power-law asymptotics of the generalization error for GP regression under power-law priors and targets
  NeurIPS 2021 Bayesian Deep Learning Workshop
  PDF

• Pradeep Kr. Banerjee and Guido Montúfar (2021)
  PAC-Bayes and Information Complexity
  ICLR 2021 Neural Compression Workshop
  OpenReview | Poster

• Pradeep Kr. Banerjee and Guido Montúfar (2020)
  The Variational Deficiency Bottleneck
  Proceedings of the International Joint Conference on Neural Networks
  arXiv | IJCNN 2020 | Slides

• Johannes Rauh*, Pradeep Kr. Banerjee*, Eckehard Olbrich, and Jürgen Jost (2019)
  Unique Information and Secret Key Decompositions
  Proceedings of the IEEE International Symposium on Information Theory
  arXiv | ISIT 2019 | Slides

• Pradeep Kr. Banerjee, Johannes Rauh, and Guido Montúfar (2018)
  Computing the Unique Information
  Proceedings of the IEEE International Symposium on Information Theory
  arXiv | ISIT 2018 | Code | Slides of a talk by Guido Montúfar at the CVPR 2019 Workshop on Semantic Information

• Pradeep Kr. Banerjee, Eckehard Olbrich, Jürgen Jost, and Johannes Rauh (2018)
  Unique Informations and Deficiencies
  Proceedings of the 56th Annual Allerton Conference on Communication, Control and Computing
  arXiv | Allerton 2018

• Johannes Rauh, Pradeep Kr. Banerjee, Eckehard Olbrich, Jürgen Jost, Nils Bertschinger, and David H. Wolpert (2017)
  Coarse-graining and the Blackwell Order
  arXiv | Entropy 2017, 19(10), 527

• Johannes Rauh, Pradeep Kr. Banerjee, Eckehard Olbrich, Jürgen Jost, and Nils Bertschinger (2017)
  On Extractable Shared Information
  arXiv | Entropy 2017, 19(7), 328

Service: Reviewer for NeurIPS (Top 8%, 2021), ICML, ICLR, ISIT, IEEE TNNLS.

I co-organize the Math Machine Learning Seminar MPI MiS + UCLA with Guido Montúfar.