Ananya Kumar
Research Lead At TBD Labs at Meta
Palo Alto, California, United States
Summary
👤
Senior🎓
Top SchoolAnanya Kumar is a research lead at Meta TBD Labs with 11 years of experience advancing large-scale ML and safety for next-generation reasoning models. Previously she was a core contributor to reasoning efforts at OpenAI (o1, o3, GPT-5) and completed a PhD in statistical machine learning at Stanford under Percy Liang and Tengyu Ma. Her work spans both theory and practice—from calibration and evaluation tooling in the widely used HELM framework to generative model and safety research at DeepMind. Based in Palo Alto, she combines deep academic rigor with hands-on engineering across research and production settings. Colleagues know her for turning nuanced evaluation metrics into robust, reproducible infrastructure that improves model reliability. She’s driven by applying AGI research toward beneficial, well-governed outcomes.
11 years of coding experience
2 years of employment as a software developer
NUS High School of Mathematics & ScienceComputer Science, Computer Science at Carnegie Mellon UniversityPhD Machine Learning, PhD Machine Learning at Stanford University
English, Hindi
Github Skills (7)
machine-learning10
nlp10
calibration10
python10
evaluation10
natural-language-processing10
metric10
Programming languages (5)
Github contributions (5)
stanford-crfm/helm
Jul 2022 - Nov 2022
Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparent evaluation of foundation models, including large language models (LLMs) and multimodal models.
Role in this project:
ML Engineer
ML EngineerContributions:42 commits, 3 comments, 1 issue in 3 months
Contributions summary:Ananya contributed to the evaluation framework of language models (HELM) by adding and refining metrics, particularly focusing on calibration within the context of multiple-choice and classification problems. Their work involved incorporating calibration statistics, such as ECE (Expected Calibration Error), and adjusting the metrics to handle different evaluation splits and problem types. The user also made modifications to handle multiple-choice scenarios within the HELM framework.
foundation-modelshelmlanguage-modellarge-language-modelsmultimodal
AnanyaKumar/transfer_learning
Nov 2020 - Jan 2023
Framework code with wandb, checkpointing, logging, configs, experimental protocols. Useful for fine-tuning models or training from scratch, and testing them on a variety of datasets (transfer learning)
Contributions:273 commits, 3 PRs, 85 pushes in 2 years 2 months
checkpointingfine-tuningloggingtestingtransfer-learning
Find and Hire Top DevelopersWe’ve analyzed the programming source code of over 60 million software developers on GitHub and scored them by 50,000 skills. Sign-up on Prog,AI to search for software developers.