Senior AI Engineer - Remote Role at Ruby Labs

About the Role

At Ruby Labs, we’re seeking a Senior AI Engineer remote to shape our AI infrastructure and drive production-ready LLM experiences. You’ll work in a modern stack, making data-driven decisions around model performance, reliability, and cost. This Senior AI Engineer remote role offers the opportunity to take full ownership of key AI features from experimentation to live production.

What You'll Do

• Design complex, dynamic prompt templates with conditional logic to maximize generation quality and reasoning.
• Implement various response schemes (JSON mode, function calling, Zod/JSON schemas) to ensure AI outputs are predictable and ready for seamless integration.
• Build robust evaluation pipelines and use Langfuse to collect feedback and score the quality of responses in real time.
• Perform deep debugging of complex LLM chains using Langfuse traces to identify bottlenecks and optimize for cost, latency, and context window usage.
• Run systematic experiments across different models via OpenRouter and analyze results based on quantitative metrics.
• Make deployment decisions for new prompts or models based strictly on quantitative benchmarks and trace data.
• Develop scoring systems to analyze the "Problem → Solution" chain and identify root causes of hallucinations or logic errors.
• Regularly re-evaluate model performance as new architectures emerge and perform fine-tuning when necessary.

Requirements

• Deep knowledge of Node.js & Next.js to build reliable services and handle complex LLM-generated data.
• Proven experience in building prompts where content is highly dependent on input variables and context injection.
• Experience working with OpenRouter for unified APIs and managing rate limits.
• Understanding of LLM observability principles, including setting up tracing and integrating scoring systems.
• Ability to transform raw generation logs into actionable business metrics and technical insights.
• Focus on continuous product improvement through constant feedback loops.

Nice to Have

• Practical experience in fine-tuning models for specific domain tasks.
• Understanding of Retrieval-Augmented Generation systems, including indexing and retrieval.
• Basic knowledge of Python for working with data science scripts or AI evaluation libraries.

What We Offer

• Remote Work Environment: Embrace the freedom to work from anywhere, promoting a healthy work-life balance.
• Unlimited PTO: Enjoy unlimited paid time off to recharge and prioritize your well-being.
• Paid National Holidays: Celebrate national holidays with paid time off.
• Company-provided MacBook: Experience seamless productivity with top-notch Apple MacBooks.
• Flexible Independent Contractor Agreement: Benefit from flexibility, autonomy, and entrepreneurial opportunities.

Interview Process

After submitting your application, we conduct a thorough review which typically takes 3 to 5 days. If we see a potential fit, we proceed with a recruiter screening, technical interview, and final interview.

Life at Ruby Labs

We’re not here to play small—we’re here to build, grow, and win. If you’re hungry to achieve, motivated by impact, and want to grow at the speed of your own ambition, Ruby Labs offers the platform to make it happen.