Senior Software Engineer - Together Cloud Infrastructure

<h3>About the Role</h3> <p>Together AI is building the AI Acceleration Cloud, an end-to-end platform for the full generative AI lifecycle, combining the fastest LLM inference engine with state-of-the-art AI cloud infrastructure.</p> <p>As a Senior AI Infrastructure Engineer, you will play a key role in building the next generation AI cloud platform – a highly available, global, blazing-fast cloud infrastructure that virtualizes cutting-edge ML hardware (GB200s/GB300s, BlueField DPUs) and enables state-of-the-art ML practitioners with self-serve AI cloud services, such as on-demand + managed Kubernetes and Slurm clusters. This platform serves both our internal SaaS products (inference, fine-tuning) and our external cloud customers, spanning dozens of data centers across the world.</p> <h3><strong>Responsibilities</strong></h3> <ul> <li>Design, build, and maintain performant, secure, and highly-available backend services/operators that run in our data centers and automate hardware management, such as Infiniband partitioning, in-DC parallel storage provisioning, and VM provisioning.</li> <li>Design and build out the IaaS software layer for a new GB200 data center with thousands of GPUs.</li> <li>Work on a global multi-exabyte high-performance object store, serving massive datasets for pretraining.</li> <li>Build advanced observability stacks for our customers with automated node lifecycle management for fault-tolerant distributed pretraining.</li> <li>Perform architecture and research work for decentralized AI workloads</li> <li>Work on the core, open-source Together AI platform</li> <li>Create services, tools, and developer documentation</li> <li>Create testing frameworks for robustness and fault-tolerance</li> </ul> <p>To be successful, you’ll need to be deeply technical and possess excellent communication, collaboration, and diplomacy skills. You have strong fundamental software development skills. In addition, you have strong systems knowledge and troubleshooting a