AI Evaluation Engineer
This is the employer's own posting, not a copy on a job board.
What we know
Is it still open?
Confirmed still open
Last checked 2d ago — checked against the employer's own applicant tracking system, which is the company answering directly.
We re-read the employer's own applicant tracking system and the posting was still there. That is the company answering directly.
How old is it?
Posted 17d ago
The date the source published, not the day we noticed it (2026-10-15). Last seen at its source 1h ago.
We have tracked this listing since 23 Oct 2026 (9 days). The employer's own board has carried it every time we have read it, most recently 1 hour ago.
Is it remote?
Marked remote on the employer's board
Their board carries a remote setting on this posting — a field they filled in, not wording we read. The location field names somewhere specific, which is usually where the team or the entity sits.
Who may apply?
United States
The description agrees: it names United States, Canada.
What the ad says
…Remote (…) US or Canada…
Pay
$200k–300k/yr
Read out of the job description by us, not from a structured field. Shown in the posting's own currency and period; we never convert.
Carried by 1 source
-
Workable employer's own board first seen 9d ago · last seen 1h ago
The listing
About the company
Our client is a software company.
The role
Raydar is recruiting for this role on behalf of our client. Lead the creation of a measurement approach for an AI system that carries out technical design work. Draw on both hands-on engineering judgment and evaluation know-how, and collaborate with domain experts and other teams to convert findings into clear improvement priorities.
What you'll do
- Establish a new assessment framework that defines quality for design work where several answers can be valid.
- Collaborate with domain experts, external users and internal teams to see how seasoned engineers operate and where the system struggles.
- Reproduce realistic design tasks and typical failure cases in repeatable test setups that developers can run again.
- Create and tune automated graders alongside human review processes, and confirm that the graders agree with expert opinion.
- Convert assessment results into a ranked list of improvements to the system's reasoning.
Requirements
What we're looking for
- About five or more years of professional experience in hardware engineering, AI, or a blend of both, with depth in one and practical familiarity with the other.
- Substantial hands-on background assessing AI systems, including authoring test tasks, building automated graders and benchmarking them against human review.
- Solid grasp of how autonomous, agent-style AI systems behave.
- Grounding in a hardware engineering field, enough to tell sound design from weak design.
- Comfort working directly with external users, specialists, partners and technical colleagues across functions.
- Self-assurance to guide work among many stakeholders in a role with high ownership.
- Based in the US or Canada, and able to travel to an office for one week every 6 weeks.
Bonus points
- Prior time at a robotics or physical AI company.
- Experience building things from zero in an early-stage startup.
Benefits
Compensation and benefits
- Base salary: USD 200,000 to 300,000 per year
- Equity
Location and work model
- Remote (US or Canada)
- Full-time