Nicole Meister

Hello! 👋 I'm Nicole (she/her/hers), a PhD student and NSF graduate research fellow at Stanford University advised by Tatsu Hashimoto and Carlos Guestrin in the Stanford NLP group.

My research interests are broadly in building fair and robust AI systems.

Previously, I graduated from Princeton University with a BSE in electrical and computer engineering and minors in cognitive science and robotics. I was fortunate to work in the Princeton VisualAI Lab and be advised by Prof. Olga Russakovsky.

Email  /  CV  /  Google Scholar  /  Github  /  LinkedIn  /  Twitter

• Nov 2024: New preprint out! Benchmarking Distributional Alignment of Large Language Models
• Jan 2024: Paper accepted to ICLR: Proving Test Set Contamination for Black-Box Language Models (Outstanding Paper Honorable Mention)
• July 2023: Paper accepted to ICCV: Gender Artifacts in Visual Datasets
• July 2022: Paper accepted to ECCV: HIVE: Evaluating the Human Interpretability of Visual Explanations
• June 2022: Presented at CVPR workshops: Gender Artifacts in Visual Datasets and HIVE
• May 2022: Graduated from Princeton 🎓: received Calvin Dodd MacCracken Senior Thesis Award (one of 3 students across all of Princeton Engineering), Sigma Xi Book Award for Outstanding Undergraduate Research (one of 4 in 40 in department)
• April 2022: Awarded NSFGRFP
• Sept 2021: Interned at Adobe Research, developed new reading Adobe Acrobat tool for acronym definitions and built acronym disambiguation dataset)
• March 2021: Awarded Adobe Research Women-in-Technology Scholarship ($10k)
• March 2021: Published in the ReScienceC journal [Re] Don't Judge an Object by Its Context: Learning to Overcome Contextual Bias (one of 23/82 reproducibility reports accepted)

We explore to what extent gendered information can truly be removed from the dataset. We develop a framework to identify gender artifacts, or visual cues that are correlated with gender.

Human evaluation framework for diverse interpretability methods in computer vision. We identify two desiderata for explanations used to assist human decision making:
(1) Explanations should allow users to distinguish between correct and incorrect predictions.
(2) Explanations should be understandable to users.

Participated in ML Reproducibility Challenge 2020 and reproduced from scratch Singh et al. (CVPR 2020) that mitigates contextual bias in object and attribute recognition.
One of 23/82 reports accepted for publication to ReScience C, a peer-reviewed journal for new implementations and explicit replications of previously published papers.

Implemented machine learning algorithms to predict prognosis of Multiple Sclerosis patients (able to predict walk time within 1 second accuracy). Used k-means to cluster patients and produce more accurate predictions.

I have had the privilege of working with some amazing labs and groups.

New reading assistant tool in Acrobat (SmartAcronyms) that extracts acronyms and their definition from text with 95% accuracy, improved speed (21x) of existing acronym extraction method while maintaining precision and recall.
Selected to present internship project to Adobe CEO and Adobe Research VP at 2021 Adobe Award Symposium.
Adobe Research Women-In-Tech Scholarship Winner($10k)

Decision rule method to selectively choose when to apply a model that performed well on out-of-context images and one that performed well on in-context images.
Explored image prediction confidence estimates and saliency heatmaps (interpretability methods).

Web application to allow conservation groups to use CNNs to quantify and monitor mangroves. Finalist for NCWIT Collegiate Award.

Python based toolkit to identify and classify tactile interactions from a locust experiment video. Used toolkit to analyze locust limb interactions to better understand individual behavior.

I am grateful to my professors for their mentorship in projects exploring in algorithmic bias, interpretability, and more.

Exploring the role of contextual objects in gender bias of computer vision models. (Python, Pytorch)

Conducted human study to evaluate and understand if the representations and prototypes identified by a computer vision interpretability method capture a sense of typicality and align with a human's concept of prototypes. (Python)

Reproduced Liang et. al method to debias sentence representations, researched effects of size of context window on method (Python)

Omni-wheeled robot that plays DDR by using computer vision to process DDR videos to extract dance moves (up, down left, right) to move and flash LED strip colors accordingly (Arduino, Python)

CNN-LSTM to recognize ASL gestures with 95% accuracy (Pytorch, Python)

Website to schedule and book dance studios. I worked on front-end and back-end to implement a scheduling algorithm to replace 40 hours of administrative work (currently used by Princeton Arts Council to schedule spaces) Advisor: Prof. Dondero (Python, Django, SQL, HTML/CSS, JavaScript, JQuery, Bootstrap, Heroku)

Creating inclusive spaces is extremely important to me as supportive environments have been crucial to my decision to pursue a graduate degree. I am extremely privileged to study at Princeton and want to (1) help Princeton students succeed (2) liberate the resources of an elite institution to support and uplift others.

Created a free 6-week course to empower high school students from historically underrepresented minority groups with the skills to leverage data science and web development.
Taught 20 hours/week online, prepared detailed lesson plans, mentored 21 students to complete data science project.
Improved program in second year by recruiting 10 program alumni to teach, advertise, and coordinate logistics of program.

Organized high school science competition, managing 30 person team, $23k budget, 140 volunteers, 800 participants.
Prioritized accessibility, becoming first ever invitational to provide travel scholarships to teams and no attendance fee.

Design and source code from Jon Barron's website