Machine Learning Research Scientist, Evaluations

The shape of it

In the posting’s own words

Scale works with the industry's leading AI labs to provide high quality data and accelerate progress in GenAI research. We are looking for Research Scientists and Research Engineers with expertise in LLM post-training (SFT, RLHF, reward modeling) and evaluation. This role is on the evaluation pod within the GenAI Research Organization and will focus on building benchmarks and diagnosing model failure modes in both text and multimodal modalities.

What it asks for · 3

• Ph.D. or Master's degree in Computer Science, Machine Learning, AI, or a related field.
• Experience with post-training techniques such as RLHF, preference modeling, or instruction tuning, and with LLM evaluation or benchmark development.
• Previous experience in a customer facing role.

Degree language

• Ph.D. or Master's degree in Computer Science, Machine Learning, AI, or a related field.

Tools and skills named

Words the posting leans on

• model6×
• research6×
• evaluation5×
• failure4×
• learning4×
• llm4×
• benchmarks3×
• post-training3×
• conferences2×
• data2×
• deep2×
• development2×
• engineers2×
• experience2×
• expertise2×
• failure modes2×

Counted from the posting after the mission statement and the legal notices are set aside. The ones near the top are the ones a screener is looking for.

More open at Scale AI

• AI Advisory ConsultantSan Francisco, CA; New York, NY
• AI Advisory PrincipalSan Francisco, CA; New York, NY
• AI Applications Ops Manager, GPSDoha, Qatar
• AI Infrastructure Engineer, Model Serving PlatformSan Francisco, CA; New York, NY
• AI Infrastructure Engineer, Sandbox PlatformLondon, UK
• AI Infrastructure Engineer, Sandbox PlatformSan Francisco, CA; Seattle, WA; New York, NY