Research Scientist, LLM Evaluation and Data (AI Startup)

A young AI company focused on the data behind model improvement is hiring one of its first dedicated researchers. This person will lead research into benchmarks, evaluations and training datasets, with emphasis on demanding domains such as life sciences, finance and similar fields.

Research scope

Design and own benchmarks, evaluation approaches and taxonomies for datasets that raise the bar on data quality

Run independent research and publish papers that build the company’s standing in AI evaluation

Locate where leading models fall short, then assemble datasets that close those gaps

Collaborate with subject-matter specialists and the people leading customer deployments to specify what strong training and eval data should contain in each vertical

Set the research direction yourself by pitching new ideas, planning experiments and pulling colleagues along

Profile

A doctorate in ML, computer science, vision or another quantitative field is strongly preferred; a master’s with an outstanding publication record can substitute

Several papers at leading venues such as ICML or NeurIPS, as lead or near-lead author

Deep knowledge of LLM training or post-training, how evaluations are designed, and how data quality drives model gains

Six to ten years of research across academia and industry

Python and PyTorch fluency for working next to evaluation infrastructure

Curiosity about how models behave and the ability to convert observed weaknesses into data-driven fixes

Hybrid in New York, NY or the San Francisco Bay Area, with most days in the office. Visa transfers considered.