Unlock a $300K Remote Role: Lead Medical AI Evaluations at Sophont

Sophont, a public-benefit corporation, is seeking a Founding Lead Researcher – Medical AI Evaluations to spearhead the evaluation strategy for its medical AI systems. This fully remote position offers a salary between $100,000 and $300,000 USD and is open to candidates globally, including those in the Global South.

The role involves developing evaluation standards for medical foundation models, clinical AI, and other systems. You'll work at the intersection of machine learning, statistics, and biomedical research, ensuring these technologies are rigorously assessed for readiness.

About the Role

As the Founding Lead Researcher, you will own the evaluation layer across Sophont's portfolio, including language models, pathology models, neuroimaging, and multimodal systems. Your work will influence how medical AI is tested and compared.

Key responsibilities include:

• Designing evaluation methodologies for various AI systems.
• Building benchmark infrastructure for public and internal leaderboards.
• Creating reproducible evaluation jobs and standardized result packages.
• Collaborating with researchers and clinicians to design high-quality datasets.
• Establishing standards for source-grounded medical AI.
• Conducting statistically defensible model comparisons and robustness checks.
• Translating findings into research papers and technical reports.

Required Qualifications

Ideal candidates will have:

• Strong Python and software engineering skills.
• Experience with data pipelines and reproducible experiments.
• Expertise in model evaluation infrastructure.
• Ability to design annotation protocols and reason about ground truth in noisy biomedical data.
• Experience with LLMs, computer vision, or biomedical foundation models.
• Strong scientific writing and communication skills.

Preferred Experience

Experience with medical, biological, or clinical datasets is preferred. Additional expertise in retrieval-augmented generation, LLM-as-judge systems, or public benchmarks is advantageous. A strong publication record or open-source contributions are also valued.

Remote Work and Benefits

This role offers full remote flexibility with core collaboration hours. The compensation package includes meaningful equity, 401(k) with 4% match, and medical, dental, and vision insurance. You'll also have opportunities to publish research and contribute to open-source projects.

How to Apply

Submit your CV to hiring@sophontai.com with the subject line: "[Your Full Name] CV – Medical AI Evaluations". Highlight your experience in machine learning evaluation, biomedical AI, and related areas.

For more details, visit the official website.