Software Engineer - Model Serving Infrastructure
Experience Requirements:
Strong Systems Fundamentals (OS, networking, concurrency)
Production Experience building systems at scale
Experience with distributed systems frameworks (gRPC, Ray)
Background in ML/AI systems or serving infrastructure
Contributions to major open source projects
Other Requirements:
Ownership Mindset
High Code Quality
Responsibilities:
Design and implement sub-millisecond Model Routing
Build Zero-Downtime Model Updates traffic management systems
Create reactive systems for Autoscaling at Scale
Architect frameworks for Multi-Model Orchestration
Build deep introspection tools for Observability & Debugging
Show more details