MLOps Engineer, LLM Performance (Kernels, Profiling, Serving)

Kurz zusammengefasst

The role is an MLOps Engineer focused on Large Language Model (LLM) performance, joining a leading AI lab's GenAI team. The lab builds foundational AI models and the position is staffed through Cincinnatus LLC, a global staffing partner. It is a fully remote, full‑time (40 hours per week) engagement with no other concurrent commitments, and the engineer will work on the cutting‑edge infrastructure that powers frontier LLMs. Day‑to‑day duties involve designing and solving challenging tasks that span accelerator kernel programming, performance profiling, debugging of distributed workloads, and high‑throughput inference serving. The engineer will craft adversarial problems that reveal failures in accelerator code (for example, Pallas kernels on TPU) and produce well‑structured solutions. They will guide research and engineering groups to close knowledge gaps, evaluate ML‑systems tasks, and deliver clear technical feedback. Additionally, the role requires developing detailed guidelines and rubrics for kernel optimization, profiler interpretation, distributed system reasoning, and serving latency‑throughput trade‑offs, while collaborating with other experts to keep training data accurate and consistent. The must‑have qualifications include at least two years of professional, hands‑on experience in ML systems, infrastructure, model serving, or accelerator performance engineering. Candidates must have practical experience in at least one of the following areas: custom kernel development (e.g., Pallas, Triton, CUDA, CUTLASS), performance profiling and trace analysis (e.g., JAX/XLA profiler, Kineto, torch.profiler, Nsight), debugging distributed or accelerator‑bound workloads, or serving large language models at scale (e.g., vLLM, SGLang, TensorRT‑LLM). Production‑level work with JAX and/or PyTorch is required, with deeper framework knowledge (custom operators, sharding, distributed training, compiler work) considered a strong plus. Experience with modern accelerators such as TPU v4‑v6e, H100, B200, Trainium or MI300, and the ability to assess throughput, latency and memory trade‑offs, is also essential. The candidate must demonstrate career progression, be able to commit 40 hours per week on weekdays, and possess strong written communication skills. Nice‑to‑have attributes include deeper expertise in multiple kernel or profiling tools, extensive work with compiler‑level technologies (XLA/StableHLO, torch.compile, MLIR), and familiarity with advanced serving techniques like KV cache, paged attention, or continuous batching. The remote nature of the role allows candidates worldwide to join, with employment handled by Cincinnatus LLC or its local employer‑of‑record partners, ensuring compliance and payroll support across jurisdictions. The position offers the chance to influence the performance of frontier AI models within a high‑impact, collaborative environment. What the role asks for: - 2+ years professional ML systems or accelerator performance experience - Hands‑on work with JAX and/or PyTorch in production - Experience writing or optimizing custom kernels (e.g., Pallas, Triton, CUDA) - Experience with performance profiling tools (e.g., JAX/XLA profiler, Kineto) - Experience debugging distributed or accelerator‑bound workloads - Experience serving LLMs at scale (e.g., vLLM, SGLang, TensorRT‑LLM) - Experience with modern accelerators (TPU, H100, B200, Trainium, MI300) - Strong written communication skills - Nice‑to‑have: deep framework knowledge (custom operators, sharding, FSDP) - Nice‑to‑have: compiler and graph‑level expertise (XLA/StableHLO, torch.compile)