Varun Kumar

I am passionate about solving meaningful problems that profoundly improve people’s lives globally through artificial intelligence. As an Applied Science Manager at AWS, I currently lead the science team for Amazon CodeWhisperer and Amazon Q in IDEs. My team focuses on developing large language models and generative AI to enable more natural human-computer interaction and streamline workflows for software developers.

My aim is to build AI systems with deep natural language understanding that seamlessly respond to and converse with people, unlocking new possibilities for human-technology collaboration. I thrive on tackling complex challenges at the intersection of language, AI, and human-computer interaction. My vision is a future enabled by collaborative AI that empowers people to focus on their most creative and fulfilling work.

Before joining AWS, I worked at Alexa Natural language understanding team where I led the development of data augmentation, few-shot learning and semi-supervised learning components. I graduated from University Of Maryland, College Park (UMD). At UMD, I primarily worked on Human-In-The-Loop topic modeling project to incorporate non-experts feedback into topic modeling.

selected publications

2021

BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation

Jwala Dhamala, Tony Sun, Varun Kumar, and 4 more authors

In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021

Abs

Recent advances in deep learning techniques have enabled machines to generate cohesive open-ended text when prompted with a sequence of words as context. While these models now empower many downstream applications from conversation bots to automatic storytelling, they have been shown to generate texts that exhibit social biases. To systematically study and benchmark social biases in open-ended language generation, we introduce the Bias in Open-Ended Language Generation Dataset (BOLD), a large-scale dataset that consists of 23,679 English text generation prompts for bias benchmarking across five domains: profession, gender, race, religion, and political ideology. We also propose new automated metrics for toxicity, psycholinguistic norms, and text gender polarity to measure social biases in open-ended text generation from multiple angles. An examination of text generated from three popular language models reveals that the majority of these models exhibit a larger social bias than human-written Wikipedia text across all domains. With these results we highlight the need to benchmark biases in open-ended language generation and caution users of language generation models on downstream tasks to be cognizant of these embedded prejudices.

2020

Data Augmentation using Pre-trained Transformer Models

Varun Kumar, Ashutosh Choudhary, and Eunah Cho

In Proceedings of the 2nd Workshop on Life-long Learning for Spoken Language Systems, Dec 2020

Abs

Language model based pre-trained models such as BERT have provided significant gains across different NLP tasks. In this paper, we study different types of transformer based pre-trained models such as auto-regressive models (GPT-2), auto-encoder models (BERT), and seq2seq models (BART) for conditional data augmentation. We show that prepending the class labels to text sequences provides a simple yet effective way to condition the pre-trained models for data augmentation. Additionally, on three classification benchmarks, pre-trained Seq2Seq model outperforms other data augmentation methods in a low-resource setting. Further, we explore how different pre-trained model based data augmentation differs in-terms of data diversity, and how well such methods preserve the class-label information.