Member of Technical Staff, AI Researcher (New Stealth Startup)

About Stealth Startup

We're launching a new startup that is redefining how mission-critical organizations adopt AI. Our platform helps organizations across healthcare, government, and other regulated industries transform institutional knowledge into intelligence assets that improve the accuracy, reliability, and impact of AI in real-world workflows. We're a fast-moving team of builders, researchers, and operators working to help customers create, measure, and continuously improve AI capabilities they can own and control.

About the Role 

We are seeking a Member of Technical Staff, AI Researcher to help turn complex, real-world workflows into the evaluation frameworks, reinforcement learning environments, and feedback systems needed to advance frontier AI. 

This is a blended research and engineering role for someone who operates at the intersection of scientific inquiry and technical execution. You will investigate how data, evaluations, and feedback systems influence model behavior, while building the infrastructure and tools required to test and apply those ideas. 

Responsibilities 

• Design and build evaluation frameworks, benchmarks, and methodologies for frontier large language models, AI agents, multimodal models, and reinforcement learning environments 

• Develop reward models, programmatic verifiers, graders, and other feedback systems that make model behavior measurable and improvable 

• Design and run controlled supervised fine-tuning and reinforcement learning experiments to measure the effects of datasets on model capability, generalization, and alignment 

• Research and define what makes evaluations representative of real-world work and operational environments 

• Build systems for producing high-quality human data, including expert task design, annotation methodologies, data-quality signals, and data-attribution techniques 

• Translate research questions into lightweight, rigorous experiments that generate actionable findings 

• Prototype quickly, evaluate results, diagnose failure modes, and convert findings into improved benchmarks, datasets, or technical systems 

• Move promising methods from experimentation into scalable, production-quality implementations 

• Work across research, engineering, product, and operations to advance a shared technical roadmap 

• Contribute to the broader technical community through benchmarks, open-source tools, research, and technical writing 

This Role is a Great Fit if You: 

• Are equally comfortable investigating an open-ended research question and building the system needed to test it 

• Approach AI research with a builder mindset and focus on producing measurable, usable outcomes 

• Are deeply interested in how data structure, selection, attribution, and quality affect model behavior 

• Can design focused experiments, interpret imperfect or ambiguous results, and translate findings into technical decisions 

• Are eager to move quickly without sacrificing rigor, maintainability, or engineering quality 

• Are comfortable working across unfamiliar domains and converting real-world workflows into measurable tasks 

• Demonstrate high agency, sound technical judgment, and a strong sense of ownership 

• Communicate complex technical concepts clearly to both technical and nontechnical stakeholders 

• Are comfortable operating in a fast-moving environment where priorities and technical approaches may evolve 

• Are motivated by the purpose and real-world impact of the work, not simply by access to emerging technologies 

Skills & Qualifications 

• 5+ years of relevant experience in machine learning, artificial intelligence, computer science, data science, software engineering, or a related technical field 

• Demonstrated ability to operate at the intersection of AI research and software engineering 

• Strong Python skills and experience building reliable, scalable software