Software Engineer - ML Infrastructure
RFS Group
San Francisco, CA · Posted November 11, 2026
Will your résumé get you an interview here?
Drop your résumé and see in seconds how well it matches what RFS Group asks for. Free, no account needed.
Just laid off? Start here: a plan for the first weeks
See every open role at RFS Group
More in California job market · Software Engineer jobs in San Francisco, CA (382) · All Software Engineer jobs (1,599) · Healthcare & Hospital Systems · Data & Analytics jobs
More roles at RFS Group
- Strategic Account Manager · San Francisco, CA
- Software Engineer - Data Infrastructure · New York, NY
- Head of Data Partnerships · San Francisco, CA
- Head of Software Engineering · Brooklyn, NY
- Chief Engineer (Principal Engineer) · San Francisco, CA
- AI Engagement Manager · San Francisco, CA
Similar jobs
- Senior Software Frontend Engineer, Market Pricing · Pave · San Francisco, CA
- Software Engineer · Mechanize · San Francisco, CA
- Senior Software Engineer · Mechanize · San Francisco, CA
- Sr. Field Applications Engineer · Supermicro · San Jose, CA
- Senior Software Engineer, Frontend · Otter.ai · Mountain View, CA
- Senior Embedded Audio Software Engineer · Qualcomm · San Diego, CA
Where San Francisco, CA ranks
Where San Francisco, CA placed in our monthly rankings, month by month. Each month stays online at its own address.
November 2026
- #1Cities hosting the most engineering conferences18 conferences · of 27
- #1Cities hosting the most product conferences8 conferences · of 6
- #1Best cities for software and engineering jobs3,732 openings · of 100
- #1Best cities for software and engineering jobs in the West Coast3,732 openings · of 59
- #1Best all-round cities for mid-career professionals88 / 100 · of 100
- #1Best all-round cities for mid-career professionals in the West Coast84 / 100 · of 35
Rankings about data & analytics jobs
- Best cities for data and analytics jobs #1 New York (1,904 openings)
- Best cities for data and analytics pay after cost of living #1 San Mateo ($184k)
- Most requested AI skills in data and analytics jobs #1 Machine learning engineering (17.1%)
- AI tools named most often in data and analytics jobs #1 Databricks (7.6%)
- Employers with the most data and analytics jobs #1 Amazon (606 roles)
Data & Analytics jobs · Top Data & Analytics employers · All rankings
Pay compared with the market
This listing does not state pay. Across 402 open software engineer listings that state pay, the middle half runs $165,000 to $247,500 and the median is $209,500.
Listed pay is what employers advertise, not what people are paid.
How AI is used in this kind of work
Among software developers, AI is already used for some of the tasks. Measured exposure for software developers: 29% (some).
Share of this job’s tasks that AI is already used for at work, from real usage. It is not a forecast of job loss, and a low number means no use seen yet. Source: Anthropic Economic Index.
Skills this job asks for
Say which you have and which you want to build. We use it to match you to jobs like this one.
About the job
Who is Recruiting from Scratch :
Recruiting from Scratch is a premier talent firm that focuses on placing the best product managers, software, and hardware talent at innovative companies. Our team is 100% remote and we work with teams across the United States to help them hire.
Software Engineer – ML Infrastructure
Location
San Francisco, CA
Company Stage of Funding
Series A2 Total Funding: Approximately $28M
Office Type
Fully on-site — 5 days per week in the San Francisco office on Market Street.
Salary
$250,000–$300,000 USD annually.
Equity
Competitive equity.
Visa
Open to visa transfers and new visa sponsorships, including OPT, H-1B transfers, new H-1B, and TN.
Employment Type
Full-time
Hiring Count
1
Company Description
This company is building a radiology-as-a-service platform powered by a multimodal AI copilot, with the goal of improving diagnostic accuracy and patient care through AI-powered medical imaging.
The company combines AI technology with a radiology practice and works with major medical institutions across the United States. It has served more than 300,000 patients and is working toward reading 1% of all daily X-rays in the US by year-end.
The platform is built around multimodal medical imaging, including X-rays, MRI scans, CT scans, echocardiograms, and ultrasounds. The company has access to more than 250,000 unique radiology images for model training and has secured its first customer through a signed $300,000 annual contract.
Backed by Bedrock Capital, Trust Ventures, and Jack Altman, the company has raised approximately $28M and is operating as a small, fast-moving team.
The engineering and research teams work closely together, with a strong emphasis on building foundational AI systems and infrastructure that allow researchers to experiment, train, evaluate, and deploy models efficiently.
What You Will Do
1. Build Distributed Training Infrastructure
Design and build the core infrastructure required to train large foundation models at scale.
Own distributed training systems for large-scale medical imaging datasets.
Build infrastructure that supports long-context parallelism and checkpointing for volumetric CT and MRI training.
Ensure training workloads are scalable, efficient, and reliable.
Identify and remove system bottlenecks that slow down research and experimentation.
2. Build High-Throughput Data Loading and Preprocessing
Design high-throughput data loading and preprocessing systems for large volumetric and multimodal datasets.
Optimize data pipelines to keep GPUs fully utilized during model training.
Build efficient workflows for handling large-scale medical imaging data.
Improve data throughput and preprocessing performance to support faster experimentation.
3. Own Reinforcement Learning Infrastructure
Build the reinforcement learning training stack for foundation-model and post-training workflows.
Develop high-throughput rollout generation systems.
Build and manage reward-model serving infrastructure.
Implement experience collection systems to support large-scale training.
Enable researchers to run online, multi-reward reinforcement learning at scale.
4. Build Model Inference and Serving Systems
Own inference systems that take models from experimentation into production.
Build efficient model-serving infrastructure.
Contribute to production serving and deployment pipelines.
Work on systems that support reliable and scalable model inference.
Help ensure models can move from research environments into production applications.
5. Build Evaluation and Experimentation Systems
Develop evaluation infrastructure to support model experimentation and research.
Build reliable systems for testing and evaluating models throughout the development lifecycle.
Work closely with researchers to understand their workflows and anticipate evolving infrastructure needs.
Make experimentation and training faster and more reliable so researchers can focus on scientific work.
6. Partner With Research and Engineering Teams
Work directly with researchers to understand their training and experimentation workflows.
Prototype new ideas alongside researchers and translate them into production-ready systems.
Own end-to-end delivery, from experimentation through deployment and monitoring.
Collaborate with the backend team on model rollout, canary deployments, and monitoring.
Help establish the infrastructure foundations as an early member of the engineering team.
Ideal Candidate Background
Experience Requirements
Strong software engineering background with experience in ML infrastructure, distributed systems, or large-scale model training.
Experience building infrastructure for training, serving, or evaluating machine learning models.
Ability to work closely with research teams and translate research requirements into production-ready systems.
Experience with distributed training and high-performance computing environments.
Comfortable owning infrastructure end to end in a small, fast-moving startup.
Able to operate autonomously and make technical decisions with limited management overhead.
Strong problem-solving skills and ability to identify and remove infrastructure bottlenecks.
The intake discussion indicates flexibility on years of experience for ML Infrastructure candidates. Prioritize technical depth, relevant infrastructure ownership, and ability to execute independently over a rigid years-of-experience requirement.
Technical Requirements
Programming and ML Frameworks
Strong Python experience.
Hands-on experience with PyTorch and/or JAX.
Ability to work with research-oriented machine learning code and production systems.
Distributed Training
Experience with distributed model training.
Familiarity with FSDP, DeepSpeed, and/or Megatron.
Understanding of distributed training architecture, parallelism, and checkpointing.
Ability to optimize training efficiency across large-scale workloads.
Data Infrastructure
Experience building high-throughput data loading and preprocessing systems.
Understanding of GPU utilization and performance bottlenecks.
Experience handling large datasets and complex data pipelines.
Familiarity with multimodal or volumetric data is valuable.
Inference and Model Serving
Experience with vLLM and/or SGLang.
Familiarity with TensorRT and Triton.
Understanding of model inference, serving, and deployment.
Ability to build efficient production model-serving systems.
Infrastructure and Systems
Experience with Kubernetes and Docker.
Strong understanding of distributed infrastructure and production deployment environments.
Familiarity with CUDA and GPU-based workloads.
Ability to build reliable infrastructure for experimentation, training, and production.
Reinforcement Learning Experience
Experience building or supporting reinforcement learning infrastructure.
Familiarity with high-throughput rollout generation.
Experience with reward-model serving or experience collection.
Understanding of online reinforcement learning workflows.
Ability to support multi-reward RL training at scale.
Ownership and Execution
Can own infrastructure projects from initial design through deployment and monitoring.
Comfortable working directly with researchers to understand and solve infrastructure challenges.
Able to prototype quickly and turn experimental systems into reliable production code.
Takes initiative in identifying performance and scalability bottlenecks.
Comfortable operating in a small engineering team with minimal management layers.
Strong ability to prioritize and execute independently.
Collaboration and Communication
Strong communication skills when working with research and engineering teams.
Comfortable translating research requirements into infrastructure solutions.
Able to explain technical decisions and performance trade-offs.
Collaborative and proactive when debugging complex systems.
Comfortable working in a high-ownership startup environment.
Nice-to-Have Qualifications
Experience training large foundation models.
Experience with multimodal models.
Experience working with medical imaging data, including CT, MRI, X-rays, or other imaging modalities.
Experience with volumetric data processing.
Experience with long-context parallelism and distributed checkpointing.
Experience building online reinforcement learning infrastructure.
Experience with GPU optimization and CUDA.
Background at a strong AI research lab, infrastructure company, or technically demanding startup.
Experience across both Big Tech and early-stage startup environments.
Compensation & Benefits
Base salary of $250,000–$300,000 USD annually.
Competitive equity.
Full-time position.
On-site work, 5 days per week in San Francisco.
Visa transfers and new sponsorships supported, including OPT, H-1B, and TN.
Why Join
Work on foundational ML infrastructure for a multimodal AI platform focused on healthcare and medical imaging.
Own distributed training, reinforcement learning, inference, and evaluation systems end to end.
Work directly with researchers and influence how large models are trained and deployed.
Build infrastructure for large-scale medical imaging datasets, including CT, MRI, X-rays, and other modalities.
Join a small engineering team with significant ownership and minimal management layers.
Help establish core ML infrastructure at an early stage, with direct impact on research velocity and production capabilities.
Work on a product with an existing customer contract and access to a substantial medical imaging dataset.