AI Model Policy Trainer, Generalist (Seattle)

ABOUT HANDSHAKE

Handshake was founded on a simple belief that everyone deserves a path to a great career, regardless of where they went to school or who they know. Today, we power 25 million job seekers, 1 million+ employers, and 1,600 educational institutions.

Handshake AI works directly with frontier AI lab researchers to create evaluations, publish benchmarks, and improve AI models through human expertise.

ABOUT THE ROLE

As an AI Model Policy Trainer, Generalist, you will turn complex customer policies into consistent, well-reasoned evaluations of AI model behavior.

You will read user requests, model responses, and relevant conversation history, then determine which policy category best applies. The most interesting cases will not have obvious answers. Two examples may look almost identical until a single word, contextual detail, or difference in intent changes the correct classification.

We are looking for people who enjoy splitting hairs in a healthy way. You form clear opinions, explain precisely why two cases should be treated differently, challenge interpretations respectfully, and change your mind when better evidence emerges. Productive disagreement helps the team find the most accurate and consistent interpretation.

Policies cannot anticipate every possible edge case. You will balance the policy’s text and intent with customer expectations, conversation context, precedent, and team calibration.

The subject matter will vary across projects, from distinguishing benign assistance from meaningful facilitation of harm to evaluating emotional reliance or nuanced sexual-safety boundaries. Success requires learning each customer’s framework on its own terms.

WHAT YOU WILL DO

- Learn customer policies, definitions, taxonomies, and evaluation rubrics, and adapt as projects and guidance change.

- Evaluate user requests and AI model responses within the full relevant conversation context.

- Distinguish between closely related labels, severity levels, and policy boundaries to select defensible classifications.

- Write concise, evidence-based rationales citing relevant policy language and conversation details.

- Identify policy gaps, contradictions, and emerging edge cases, and raise questions when guidance does not resolve a case.

- Participate in calibration discussions with evaluators, project leads, policy teams, and researchers; challenge interpretations respectfully and update your judgment when evidence warrants it.

- Apply customer policy consistently, maintain accuracy across repeated evaluations, and incorporate feedback into future work.

- Help improve evaluation frameworks, examples, decision rules, and quality standards.

YOU MAY BE A FIT IF

- You enjoy precise distinctions and notice when one word, contextual detail, or change in intent affects the answer.

- You can hold a strong opinion without becoming attached to being right and discuss disagreement respectfully.

- You explain judgment calls clearly enough that another person can audit your reasoning.

- You ask productive questions when a policy is ambiguous instead of guessing or forcing certainty.

- You separate personal views from the customer’s standard while understanding both the letter and purpose of a policy.

- You remain careful and consistent during repetitive work and apply feedback quickly.

- You learn unfamiliar subjects quickly, work independently, and recognize when you need more context.

- You treat sensitive information and difficult subject matter with maturity and sound judgment.

Strong candidates may come from quality assurance, research, editing, law, teaching, operations, trust and safety, content moderation, social science, policy, investigations, compliance, customer support, or other fields that require careful interpretation and defensible decision-making. We care more about how you reason than where you learned to reason.

NICE TO HAVE

- Professional experience evaluating language models or wo …