Research
My research interests lie in the areas of information theory, statistics, and deep learning. My current focus is on understanding the implicit regularization and stability properties of neural network optimization algorithms. I am also interested in information-theoretic aspects of control and unsupervised reinforcement learning.
In my PhD, I developed new methods to decompose information into parts that allow for a fine-grained analysis of how information is distributed over composite systems consisting of multiple interacting parts or subsystems. These methods are potentially useful in applications ranging from neuroscience and representation learning, to robotics, and cryptography.
See my Google Scholar page for an updated list of publications.
Hui Jin, Pradeep Kr. Banerjee and Guido Montúfar (2022)
Learning Curves for Gaussian Process Regression with Power-Law Priors and Targets
International Conference on Learning Representations (ICLR 2022)
PDFPradeep Kr. Banerjee and Guido Montúfar (2021)
Information Complexity and Generalization Bounds
Proceedings of the IEEE International Symposium on Information Theory
arXiv | ISIT 2021 | SlidesHui Jin, Pradeep Kr. Banerjee and Guido Montúfar (2021)
Power-law Asymptotics of the Generalization Error for GP Regression under Power-Law Priors and Targets
NeurIPS 2021 Bayesian Deep Learning Workshop
PDFPradeep Kr. Banerjee and Guido Montúfar (2021)
PAC-Bayes and Information Complexity
ICLR 2021 Neural Compression Workshop
OpenReview | PosterPradeep Kr. Banerjee and Guido Montúfar (2020)
The Variational Deficiency Bottleneck
Proceedings of the International Joint Conference on Neural Networks
arXiv | IJCNN 2020 | SlidesJohannes Rauh*, Pradeep Kr. Banerjee*, Eckehard Olbrich, and Jürgen Jost (2019)
Unique Information and Secret Key Decompositions
Proceedings of the IEEE International Symposium on Information Theory
arXiv | ISIT 2019 | SlidesPradeep Kr. Banerjee, Johannes Rauh, and Guido Montúfar (2018)
Computing the Unique Information
Proceedings of the IEEE International Symposium on Information Theory
arXiv | ISIT 2018 | Code | Slides of a talk by Guido Montúfar at the CVPR 2019 Workshop on Semantic InformationPradeep Kr. Banerjee, Eckehard Olbrich, Jürgen Jost, and Johannes Rauh (2018)
Unique Informations and Deficiencies
Proceedings of the 56th Annual Allerton Conference on Communication, Control and Computing
arXiv | Allerton 2018Johannes Rauh, Pradeep Kr. Banerjee, Eckehard Olbrich, Jürgen Jost, Nils Bertschinger, and David H. Wolpert (2017)
Coarse-graining and the Blackwell Order
arXiv | Entropy 2017, 19(10), 527Johannes Rauh, Pradeep Kr. Banerjee, Eckehard Olbrich, Jürgen Jost, and Nils Bertschinger (2017)
On Extractable Shared Information
arXiv | Entropy 2017, 19(7), 328
Service: Reviewer for NeurIPS (Top 8%, 2021), ICML, ICLR, TMLR, ISIT, IEEE TNNLS.
I co-organize the Math Machine Learning Seminar MPI MiS + UCLA with Guido Montúfar.