Key Skills

Job Description

Help evaluate how AI systems approach scientific and clinical work. Use your subject expertise to design realistic problems, define verifiable outcomes and evaluate proposed solutions for scientific accuracy and quality of reasoning. What you’ll work on Develop Linux-terminal benchmarking tasks grounded in your field, such as interpreting evidence, analyzing data or testing a scientific workflow. Define how to judge whether a result is scientifically sound and reproducible. Document errors, uncertainty and the evidence behind your assessment. Requirements A completed PhD in a relevant scientific field, or an MD with a clinical specialization. Applicants must be out of current degree-program enrollment. At least seven years of relevant experience, including PhD and postdoctoral work. An alternate experience route is at least three relevant years and three peer-reviewed papers as first author or coauthor. Clinical applicants must be PGY-4 or later, or at an equivalent training stage. Senior residents, fellows and attendings can qualify. Practical comfort using a Linux terminal and at least one authored or co-authored academic or professional publication. Compensation and commitment The project pays $1,350 per task, with an expected commitment of 20 hours per week for 1–6 months.

Core Responsibilities

Design realistic, field-specific Linux-terminal benchmarking tasks for evaluating AI systems on scientific or clinical work. Define verifiable standards for scientifically sound and reproducible results, then assess proposed solutions and document errors, uncertainty, and supporting evidence.

Requirements

Applicants must have a relevant PhD or a clinically specialized MD, be out of current degree-program enrollment, and have at least seven years of relevant experience; an alternative is at least three years of relevant experience plus three peer-reviewed papers. Clinical applicants must be PGY-4 or later or at an equivalent training stage, and all applicants need practical Linux-terminal comfort and at least one authored or co-authored publication.