Inference Runtime Engineer - Remote Position

About the Role

We're hiring an Inference Runtime Engineer to join our team at Inferact. This remote position offers an exciting opportunity to work on cutting-edge AI inference technologies. As an Inference Runtime Engineer, you will play a crucial role in optimizing the execution of large language models (LLMs) across diverse hardware and architectures, making inference cheaper and faster.

What You'll Do

• Develop and optimize inference engines for LLMs and diffusion models.
• Implement and enhance model architectures and inference techniques based on the latest research.
• Collaborate with cross-functional teams to push the boundaries of AI inference capabilities.
• Contribute to the vLLM core, ensuring performant and maintainable code.
• Debug complex ML codebases and improve the overall efficiency of the inference process.

Requirements

• Bachelor's degree or equivalent experience in computer science, engineering, or a related field.
• Strong programming skills in Python, with extensive experience in PyTorch internals.
• Deep understanding of transformer architectures and their variants.
• Experience with LLM inference systems such as vLLM, TensorRT-LLM, or SGLang.
• Ability to read and implement model architectures from research papers.

Nice to Have

• Familiarity with KV-cache memory management and hybrid model serving.
• Experience with multimodal inference (audio/image/video/text).
• Contributions to open-source ML projects.

What We Offer

• Annual salary range of $200,000 - $400,000, depending on experience.
• Equity options to share in the company's success.
• Comprehensive health, dental, and vision benefits.
• 401(k) company match to help you save for the future.
• Flexible remote work options.