Staff Software Engineer, Model Serving
Databricks · San Francisco, California · Engineering · listed October 14, 2025
The shape of it
Seniority
Staff
Experience asked
10+ years
Where
Not stated
Stated pay
$192,000 – $260,000 USD
Requirements listed
8
Length
821 words
In the posting’s own words
As a Staff Engineer, you’ll play a critical role in shaping both the product experience and the foundational infrastructure of Model Serving. You will design and build systems that enable high-throughput, low-latency inference across CPU and GPU workloads, influence architectural direction, and collaborate closely across platform, product, infrastructure, and research teams to deliver a world-class serving platform.
What it asks for · 8
- 10+ years of experience building and operating large-scale distributed systems.
- Deep expertise in model serving, inference systems, and related infrastructure (e.g., routing, scheduling, autoscaling, and observability).
- Strong foundation in algorithms, data structures, and system design as applied to large-scale, low-latency serving systems.
- Proven ability to deliver technically complex, high-impact initiatives that create measurable customer or business value.
- Experience leading architecture for large-scale, performance-sensitive CPU/GPU inference systems.
- Strong communication skills and ability to collaborate across teams in fast-moving environments.
- Strategic and product-oriented mindset with the ability to align technical execution with long-term vision.
- Passion for mentoring, growing engineers, and fostering technical excellence.
What the job covers
- Design and implement core systems and APIs that power Databricks Model Serving, ensuring scalability, reliability, and operational excellence.
- Partner with product and engineering leadership to define the technical roadmap and long-term architecture for serving workloads.
- Drive architectural decisions and trade-offs to optimize performance, throughput, autoscaling, and operational efficiency for CPU and GPU serving workloads.
- Contribute directly to key components across the serving infrastructure — from model container builds and deployment workflows to runtime systems like routing, caching, observability, and intelligent autoscaling — ensuring smooth and efficient operations at scale.
- Collaborate cross-functionally with product, platform, and research teams to translate customer needs into reliable and performant systems.
- Lead technical initiatives that improve latency, availability, and cost-effectiveness across both customer-facing and foundational serving layers.
- Establish best practices for code quality, testing, and operational readiness, and mentor other engineers through design reviews and technical guidance.
- Represent the team in cross-organizational technical discussions and influence Databricks’ broader AI platform strategy.
Tools and skills named
Models & research
- Inference4×
- GPU3×
- Machine learning2×
- LLM
Data
- Databricks5×
Cloud & infra
- Observability2×
- Distributed systems
Ways of working
- Mentorship
- Testing
Product & design
- Roadmap
Words the posting leans on
- serving11×
- model9×
- systems9×
- platform7×
- technical6×
- infrastructure5×
- model serving5×
- product5×
- customer4×
- data4×
- design4×
- inference4×
- autoscaling3×
- collaborate3×
- engineers3×
- experience3×
Counted from the posting after the mission statement and the legal notices are set aside. The ones near the top are the ones a screener is looking for.
The posting, your resume, and the gaps between them. One click loads all three.