Framework Software Engineer
Education Requirements:
Master's or higher degree in Computer Science, Electrical Engineering, or a related field
Experience Requirements:
Strong experience with Python, C++ and PyTorch, including model execution and runtime internals
Hands-on experience with inference serving or high-performance ML systems
Other Requirements:
Familiarity with Linux systems, profiling tools, and debugging performance bottlenecks
Strong problem-solving skills and the ability to reason about system-level trade-offs
Clear communication skills and ability to collaborate in a fast-paced environment
Responsibilities:
Design, develop, and optimize high-performance inference frameworks for large-scale distributed serving
Improve end-to-end serving performance (TTFT, ITL, throughput, tail latency)
Design scalable multi-node serving architectures
Analyze and optimize memory usage, device utilization, and communication overhead
Build and maintain benchmarks and simulators for AI serving workloads
Show more details