AI Engineer

About Alaffia & Our Mission

Every year, U.S. health plans lose billions to improper payments and administrative waste. That wasted spending ultimately trickles down across the healthcare ecosystem, driving up costs for plans, providers, and patients alike. We’re here to change that paradigm.

Alaffia is a new kind of claims operations partner for health plans. Using expert clinicians and transparent AI, we deliver deeper insights, smarter automation, and consistently better outcomes across the entire lifecycle of claims. With Alaffia, health plans can cut wasted spending more effectively than ever — and provide their members the most affordable care.

We’re a high-growth, venture-backed Series B healthtech startup based in NYC and are actively scaling our company. Join us in helping to build a healthcare system that works better for everyone.

*This position requires current authorization to work in the United States. Unfortunately, we are not in a position to sponsor work visas at this time.

About the Role & What You’ll Be Doing

Alaffia Health is a healthcare AI startup revolutionizing medical billing and data automation. Our AI-driven platform leverages state-of-the-art generative AI and machine learning technologies to enhance accuracy, efficiency, and compliance in medical billing and auditing

As we scale, we’re looking for an AI Engineer to build the intelligence behind Alaffia’s AI agents. Our agentic systems are already in production reviewing real claims. As our customers’ needs grow more complex, we need someone to make our agents smarter, more accurate, and better grounded in the policies, contracts, and clinical evidence they reason over.

This is a model-focused role spanning four areas. You’ll design and build the agents themselves, including their planning, tool use, and multi-step reasoning across claims and medical records. You’ll develop and fine-tune the LLMs that power them on domain-specific healthcare data. You’ll build the RAG systems that keep their outputs accurate and traceable to source documents. And you’ll apply reinforcement learning techniques such as RLHF and DPO, turning feedback from our clinical auditors into measurable improvements in agent performance.

This role is for you if you like working at the frontier of applied LLM research and seeing your work land in production. You’ll take ideas from early experiments through to models that make real decisions on real claims, and you’ll have a direct hand in shaping how a Series B healthtech builds, trains, and improves its AI. You’ll work closely with our AI engineers, MLOps team, and clinical subject-matter experts.

Your Responsibilities

AI Agent Development: Design and build single- and multi-agent systems for claim review. This includes planning, tool use, memory, and multi-step reasoning across claims data, medical records, and coding guidelines.

RAG Development: Build and optimize retrieval-augmented generation pipelines over policies, contracts, clinical guidelines, and medical records. This covers chunking strategies, embeddings, hybrid search, re-ranking, and grounding and citation of outputs.

LLM Development & Fine-Tuning: Fine-tune open-source and proprietary LLMs (e.g., SFT, LoRA/QLoRA, instruction tuning) on domain-specific healthcare and claims data. Manage dataset curation, training runs, and model versioning.

Reinforcement Learning: Develop RL-based training and alignment methods (e.g., RLHF, DPO, reward modeling) that use human-in-the-loop feedback from clinical auditors. Use them to improve agent accuracy, reasoning, and decision quality.

Evaluation & Experimentation: Define evaluation frameworks, benchmarks, and metrics for agents, fine-tuned models, and RAG systems. These include hallucination, faithfulness, retrieval quality, and task-level accuracy. Run and track experiments rigorously.

Production Integration (in collaboration with the Software and DevOps teams): Take models and agents from prototype to production, optimizing for latency, cost, and reliability at scale.

Compliance & Safety: Ensure models and agents handle PHI securely and comply with HIPAA. Build guardrails that keep outputs accurate, explainable, and auditable.

What We’re Looking For

3+ years of experience in applied ML or AI engineering, ideally with experience developing agentic systems, fine-tuning LLMs, and building RAG pipelines in production.

Strong software engineering fundamentals and proficiency in Python.

Highly experienced in developing AI agents and agentic workflows, including tool calling, orchestration, and multi-step reasoning

Hands-on experience with LLM fine-tuning (SFT, PEFT/LoRA) and training pipelines using frameworks such as PyTorch and Hugging Face

Proven experience building production RAG systems , including vector databases, embedding models, and retrieval optimization

Practical experience applying reinforcement learning to LLMs (e.g., RLHF, DPO, PPO, reward modeling)

Strong understanding of LLM evaluation methodologies and experiment tracking

Nice to have

Experience with agent frameworks such as LangGraph, LangChain, LlamaIndex, or similar

Familiarity with LLM provider APIs and self-hosted model serving (e.g., vLLM, TGI)

Experience with distributed training and GPU infrastructure

Working knowledge of AI tooling such as MLFlow, LangFuse, and Weights & Biases

Experience with healthcare data (claims, EHRs, medical coding such as ICD-10/CPT) or other regulated domains

Our Culture

Alaffia was born out of our founders’ personal connection to the inefficiency of the U.S. healthcare system. We are deeply mission-driven, with an abiding belief that technology can help create a better future for everyone — and we’re looking for others who share our passion for change to join the team.

What Else Do You Get Working With Us?

Competitive Compensation

Medical, Dental, and Vision benefits

Flexible, paid vacation policy

Work in a flat organizational structure — direct access to Leadership