Research Scientist, Safety Evaluations (Applied AI Research Lab)

A young applied AI research lab serving frontier model developers is looking for a hands-on researcher, not a manager, to start a safety evaluation program. The first focus is measuring how well models can carry out and speed up AI research themselves, followed by studying risky model behavior.

The work

Design benchmarks that gauge how well models can automate research and improve themselves, with explicit tasks, reference baselines, resource limits and definitions of success

Build test environments that surface exploitation of reward signals, sabotage, deceptive conduct and failure to respect stated limits

Examine ways results can mislead, such as data contamination, weak graders, ambiguous tasks and shortcuts

Test today’s models, study their transcripts and failure cases, and follow how abilities evolve from one model to the next

Partner with engineers to convert research designs into environments others can rerun, with transparent scoring and written caveats

Publish benchmark releases and reports, and advise lab partners on what the evidence means

Help steer the safety research agenda; some of the environments may later feed into model training

Profile

One to ten years as a research scientist or alignment researcher, with recent hands-on safety work on frontier models

Experience with post-training and reinforcement learning

Contributions to recognized benchmarks or papers at major ML venues

Python and PyTorch

Solid grasp of current frontier capabilities and where they are heading

Work on AI R&D or self-improvement evaluations, pre-deployment testing, or a PhD is a plus

On-site in the San Francisco Bay Area. Visa transfers considered; no new visa sponsorship.