Performance Engineer, Inference Systems

The shape of it

In the posting’s own words

You'll work across all four areas. One week that might mean tracing a tail-latency regression from request timing down through routing and batching into a kernel overhead; the next it might mean tightening a correctness eval so it catches an output regression introduced by a quantization change. We're looking for performance engineers who treat correctness as part of performance.

What it asks for · 5

• Hands-on performance engineering experience: profiling, roofline analysis, latency/throughput optimization, and root-cause investigation in complex production systems
• Proficiency in Python, with the ability to read, instrument, and contribute to large production codebases you didn’t write
• Solid data analysis skills (e.g. SQL, pandas, or similar) sufficient to turn raw telemetry into clear findings
• Ability to communicate quantitative results clearly in writing to influence priorities on teams you don't manage
• Genuine interest in correctness as an engineering discipline: numerics, evaluation design, regression detection

Also a plus

• Experience with ML systems, especially training or inference infrastructure or general LLM serving stacks. Direct large-scale inference experience is a strong plus
• Familiarity with GPU/TPU/accelerator performance concepts (memory bandwidth, kernel overheads, quantization, collective communication). Reasoning about these matters more than having written kernels yourself
• Experience with reliability engineering for high-throughput services: autoscaling, load balancing, request routing, tail latency
• Experience with model evaluation or numerical regression-detection pipelines
• Experience building observability or telemetry for distributed systems
• Comfortable having impact through influence and evidence rather than direct ownership

What the job covers

• Run cross-layer performance investigations across throughput, latency, and reliability, sizing the gap between actual fleet performance and theoretical rooflines, identifying root causes, and quantifying the value of closing them
• Own and improve the correctness evaluation pipeline that validates model output quality across hardware platforms, numerics, and serving configurations, and lead the investigation when it catches a regression
• Build the observability, dashboards, and modeling tools that make throughput, latency, cost, reliability, correctness, and their interactions legible across the stack
• Partner with kernel, serving, routing, autoscaling, and capacity teams to prioritize and land the highest-impact optimizations your analysis surfaces
• Ruthlessly stack-rank a large surface area of opportunities by impact and effort, and say no to the ones that don't make the cut

Tools and skills named

Words the posting leans on

• correctness8×
• kernel7×
• performance7×
• experience6×
• latency6×
• model5×
• regression5×
• systems5×
• fleet4×
• gap4×
• inference4×
• investigation4×
• reliability4×
• routing4×
• serving4×
• analysis3×

Counted from the posting after the mission statement and the legal notices are set aside. The ones near the top are the ones a screener is looking for.

More open at Anthropic

• Account Executive, AI NativeNew York City, NY; San Francisco, CA | New York City, NY
• Account Executive - DNBSingapore
• Account Executive, Public SectorSydney, Australia
• Account Executive - Public Sector (ASEAN)Singapore
• Account Executive, StartupsSan Francisco, CA | New York City, NY
• Accounting, Revenue Internal ControlsSan Francisco, CA | Seattle, WA