Research

My research interests lie in the areas of information theory, statistics, and deep learning. I am also interested in information-theoretic aspects of control and unsupervised reinforcement learning.

In my PhD, I developed new methods to decompose information into parts that allow for a fine-grained analysis of how information is distributed over composite systems consisting of multiple interacting parts or subsystems. These methods are potentially useful in applications ranging from neuroscience and representation learning, to robotics, and cryptography.

See my Google Scholar page for an updated list of publications.

• Kedar Karhadkar, Pradeep Kr. Banerjee, and Guido Montúfar (2022)
  FoSR: First-order spectral rewiring for addressing oversquashing in GNNs
  arXiv

• Pradeep Kr. Banerjee, Kedar Karhadkar, Yu Guang Wang, Uri Alon, and Guido Montúfar (2022)
  Oversquashing in GNNs through the lens of information contraction and graph expansion
  58th Annual Allerton Conference on Communication, Control and Computing
  arXiv | Allerton 2022

• Johannes Rauh, Pradeep Kr. Banerjee, Eckehard Olbrich, Guido Montúfar and Jürgen Jost (2022)
  Continuity and additivity properties of information decompositions
  12th Workshop on Uncertainty Processing (WUPES) 2022
  arXiv

• Hui Jin, Pradeep Kr. Banerjee and Guido Montúfar (2022)
  Learning Curves for Gaussian Process Regression with Power-Law Priors and Targets
  International Conference on Learning Representations (ICLR 2022)
  PDF

• Pradeep Kr. Banerjee and Guido Montúfar (2021)
  Information Complexity and Generalization Bounds
  IEEE International Symposium on Information Theory
  arXiv | ISIT 2021 | Slides

• Hui Jin, Pradeep Kr. Banerjee and Guido Montúfar (2021)
  Power-law Asymptotics of the Generalization Error for GP Regression under Power-Law Priors and Targets
  NeurIPS 2021 Bayesian Deep Learning Workshop
  PDF

• Pradeep Kr. Banerjee and Guido Montúfar (2021)
  PAC-Bayes and Information Complexity
  ICLR 2021 Neural Compression Workshop
  PDF | Poster

• Pradeep Kr. Banerjee and Guido Montúfar (2020)
  The Variational Deficiency Bottleneck
  IEEE International Joint Conference on Neural Networks
  arXiv | IJCNN 2020 | Slides

• Johannes Rauh*, Pradeep Kr. Banerjee*, Eckehard Olbrich, and Jürgen Jost (2019)
  Unique Information and Secret Key Decompositions
  IEEE International Symposium on Information Theory
  arXiv | ISIT 2019 | Slides

• Pradeep Kr. Banerjee, Johannes Rauh, and Guido Montúfar (2018)
  Computing the Unique Information
  IEEE International Symposium on Information Theory
  arXiv | ISIT 2018 | Code | Slides of a talk by Guido Montúfar at the CVPR 2019 Workshop on Semantic Information

• Pradeep Kr. Banerjee, Eckehard Olbrich, Jürgen Jost, and Johannes Rauh (2018)
  Unique Informations and Deficiencies
  56th Annual Allerton Conference on Communication, Control and Computing
  arXiv | Allerton 2018

• Johannes Rauh, Pradeep Kr. Banerjee, Eckehard Olbrich, Jürgen Jost, Nils Bertschinger, and David H. Wolpert (2017)
  Coarse-graining and the Blackwell Order
  arXiv | Entropy 2017, 19(10), 527

• Johannes Rauh, Pradeep Kr. Banerjee, Eckehard Olbrich, Jürgen Jost, and Nils Bertschinger (2017)
  On Extractable Shared Information
  arXiv | Entropy 2017, 19(7), 328

Service: Reviewer for NeurIPS (Top 8%, 2021), ICML, ICLR, TMLR, ISIT, IEEE TNNLS.

I co-organize the Math Machine Learning Seminar MPI MiS + UCLA with Guido Montúfar.