avatar
Umang Gupta

I recently earned my Ph.D. from the University of Southern California. I was advised by Prof. Greg Ver Steeg. I am interested in unsupervised learning, representation learning, and using machine learning for scientific advancement and improving and scaling decision-making capabilities. More recently, I have been working on fairness (1, 2, 3) and privacy (4, 5, 6) problems in machine learning and federated learning (7). This culminated in a Ph.D. thesis titled “Controlling Information in Neural Networks for Fairness and Privacy.”

During my Ph.D., I spent a summer at Amazon-Alexa AI as an applied scientist intern mentored by Aram Galstyan, Kai-Wei Chang, and Rahul Gupta, working on fairness in language models. I spent another summer at Morgan Stanley, working on time-series problems.

Before coming to USC, I spent two years at Visa Inc., Bangalore, as a senior software developer, where I led the development of web applications for card management. Before that, I spent five wonderful years at IIT Delhi, concluding with a Dual Degree (B.Tech and M.Tech) in Electrical Engineering and a silver medal.

Research Interest & Experience

Highlights

Selected Publications

See Google Scholar for full list.

  • Jointly Reparametrized Multi-Layer Adaptation for Efficient and Private Tuning

    Umang Gupta, Aram Galstyan, Greg Ver Steeg

    Findings of the Association for Computational Linguistics (ACL 2023)

    code / arXiv

    We finetune transformers by sharing parameters across layers, leading to better tuning than prompt tuning methods while using similar parameters and improving differential private finetuning for supervised tasks.

  • Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversal

    Umang Gupta, Jwala Dhamala, Varun Kumar, Apurv Verma, Yada Pruksachatkun, Satyapriya Krishna, Rahul Gupta, Kai-Wei Chang, Greg Ver Steeg, Aram Galstyan

    Findings of the Association for Computational Linguistics (ACL 2022)

    arXiv

    We mitigate gender disparity in text generation while performing knowledge distillation by exploiting counterfactual role-reversed texts for training.

  • Membership Inference Attacks on Deep Regression Models for Neuroimaging

    Umang Gupta, Dimitris Stripelis, Pradeep K. Lam, Paul Thompson, Jose Luis Ambite, Greg Ver Steeg

    Medical Imaging and Deep Learning (MIDL 2021)

    openreview / arXiv

    We illustrate that allowing access to model parameters can leak private information about the training set. We observed strong correlations between privacy leakage and overfitting, indicating that reducing overfitting may ensure privacy.

  • Controllable Guarantees for Fair Outcomes via Contrastive Information Estimation

    Umang Gupta, Aaron Ferber, Bistra Dilkina, Greg Ver Steeg

    The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021)

    video / code / arXiv

    We show that limiting the mutual information between the representations limits any classifier’s statistical parity. To this end, we propose a novel method for controlling fairness through mutual information based on contrastive information estimators.

  • Improved Brain Age Estimation with Slice-based Set Networks

    Umang Gupta, Pradeep K. Lam, Greg Ver Steeg, Paul M. Thompson

    18th International Symposium on Biomedical Imaging (ISBI 2021)

    code / arXiv / follow-up work about transfer learning at ISBI 2023

    We propose a new architecture for making predictions over 3D-MRIs prediction, which works by encoding a single 2D slice in an MRI with a deep 2D-CNN model and integrating the information from these 2D-slice encodings with permutation invariant layers.

  • Deep Generative Dual Memory Network for Continual Learning

    Nitin Kamra, Umang Gupta, Yan Liu

    arXiv

    Deriving inspiration from human complementary learning systems (hippocampus and neocortex), we develop a dual generative memory architecture that consolidates memory via generative replay and is capable of learning continuously from sequentially incoming tasks while averting catastrophic forgetting.

Not Research