Software Engineer II - LLM Ops Engineer (Remote)

About the Role

We are looking for a Software Engineer II - LLM Ops Engineer (Remote) to join our dynamic team at Housecall Pro. In this role, you will support the development, deployment, and monitoring of production AI applications powered by large language models. Your contributions will directly impact the lives of home service professionals across the United States.

What You'll Do

• Build and maintain LLM-powered applications and agent systems using Python, FastAPI, LangChain, and LangGraph.
• Design and optimize agentic workflows for multi-step reasoning, tool usage, and state management.
• Deploy and manage LLM applications on AWS, ensuring system reliability, performance, and scalability.
• Implement observability using LangSmith to monitor token usage, latency, and prompt/response quality.
• Build and maintain Airflow-based data pipelines to support LLM workflows, embeddings, and retrieval.
• Implement and tune RAG (Retrieval-Augmented Generation) systems using PGVector for semantic search.
• Work with Snowflake to manage evaluation datasets, data analytics, and warehousing for AI features.
• Design RESTful APIs using FastAPI to expose LLM capabilities and handle streaming and async responses.

Requirements

• Bachelor’s degree in Computer Science, Engineering, Data Science, or related field, or equivalent work experience.
• 2–4 years of professional software engineering experience with strong Python development skills.
• Experience building and deploying production-grade RESTful APIs (i.e. FastAPI, Flask).
• Hands-on experience integrating large language models (i.e. OpenAI, Anthropic, Bedrock, or open-source LLMs).
• Familiarity with LangChain or similar frameworks for LLM orchestration.
• Understanding of prompt engineering, context handling, and model interaction patterns.
• Experience with AWS infrastructure (i.e. EC2, Lambda, ECS/EKS, S3).
• Knowledge of relational databases and SQL (PostgreSQL preferred).

Nice to Have

• Familiarity with vector databases and semantic search using vector embeddings.
• Production experience building agentic workflows using LangGraph.
• Hands-on use of LangSmith for LLM observability, tracing, and evaluation.
• Experience building data and ML pipelines with Airflow.
• Working knowledge of Docker and orchestration tools such as Kubernetes or ECS.

What We Offer

• Remote work environment with a culture that values innovation and collaboration.
• Self-managed PTO to encourage work-life balance.
• Flexible working hours to help you achieve personal and professional goals.
• MacBook (or PC) and a setup fee of $500 for your home office.
• Opportunities for career growth and development in a supportive environment.