LLM Red Team Specialist - AI Job at Mercor, New York, NY

OEgxcWJsMnV3cXA0eGxXTHdrcmdGMlV5eHc9PQ==
  • Mercor
  • New York, NY

Job Description

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: LLM Red Team Specialist — Failure Modes & Edge Cases
Type: Contract
Compensation: $60–$90/hour
Location: Remote
Commitment: 35 hours/week

Role Responsibilities

  • Evaluate frontier AI models on coding, ML , and analysis tasks to identify vulnerabilities and failure modes.
  • Design complex tasks that challenge models and are fair for grading.
  • Document findings with clear evidence and reproducible steps.
  • Collaborate with task authors to close loopholes and improve grading.
  • Share insights with researchers to enhance benchmark quality.
  • Work independently and asynchronously to meet deadlines and improve AI model performance .

Qualifications

Must-Have

  • MSc or PhD in a STEM field or equivalent experience.
  • 1+ years in research, research-engineering, security, or AI evaluation.
  • Experience identifying vulnerabilities in LLMs or ML systems .
  • Proficiency in Python and Git .
  • Familiarity with LLM capabilities and evaluation techniques.
  • Ability to work 35 hours/week .

Preferred

  • Experience in AI training , model evaluation, or benchmark/task authoring.

Resources & Support

  • For details about the interview process and platform information, please check:
  • For any help or support, reach out to: support@mercor.com

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Job Tags

Contract work, Summer work, Remote work

Similar Jobs

Lenmar Consulting Inc

RPA Data Analytics Automation Analyst Job at Lenmar Consulting Inc

 ...RPA Data Analytics Automation Analyst Contract LOCATION: can sit in either Jersey City or NYC (1 NY Plaza)- Hybrid: 3 days in office, 2 days remote *** HM: "Consulting experience is a plus however the role is primarily an automation and data analytics position.... 

Mariani Premier Group

Landscape Designer Job at Mariani Premier Group

## Landscape DesignerApply: NJ, Basking Ridge: Full time: Posted Yesterday: R2416## **Work With The Best**We are building the premier outdoor...  ...properties into exceptional outdoor living spaces. We design, build, and maintain customized landscapes that reflect each client... 

HearingLife

Audiologist or Hearing Aid Specialist Job at HearingLife

Overview Audiologist or Hearing Aid SpecialistPosition Type Full-Time In-OfficeLocation Wooster, OHAbout HearingLife HearingLife is a national hearing care provider operating more than 600 hearing care centers across 42 states. As part of the Demant Group and in association... 

Boone Health

Sterile Processing Technician, Full Time - Evenings/Nights Job at Boone Health

 ...Sterile Processing Technician Performs a variety of Sterile Processing functions, which include the decontamination, preparation, sterilization...  ...Qualifications High School Diploma or GED No Experience No Supervisor Experience Preferred Qualifications... 

Riot Games, Inc.

Principal Game Designer - League of Legends, ARAM Mayhem Job at Riot Games, Inc.

 ...Principal Game Designer - League of Legends, ARAM Mayhem As a game designer, you will design, tweak, and optimize the player experience...  ...of materially-impactful world-class innovations on a commercial video game For this role, you'll find success through craft...