The project

Job description

Join our contributor community

PortAnnotate recruits annotators, evaluators and transcribers directly across 12 countries and 13 languages. Work is remote, paid by the hour, and scheduled around you. One profile carries across every project you apply to.

About the role

The model has just given a confident, fluent, wrong answer. Your job is to say exactly where it went wrong, and then write the answer it should have given.

Much of the day is comparison work: two responses, one rubric, a decision, and a written defence of that decision that another specialist could disagree with on the merits. The rest is adversarial, going looking for the prompt that makes a model fail and documenting how you found it.

Key responsibilities

• Write reference answers that a specialist in the field would sign off on.
• Grade paired model responses against a written rubric and defend the call.
• Probe for failure: find prompts that break the model and record the steps.
• Flag factual, safety and tone problems with a specific citation or reason.
• Feed recurring failure patterns back so the rubric can be tightened.

---

Your fit

Qualification path

Required qualifications

• Based in the United States, with proof of work authorisation before the first assignment.
• Native or fully fluent written English.
• Demonstrable experience writing to a brief: research, journalism, technical documentation or similar.
• Comfortable being overruled and arguing the point in writing.

Technical expertise

Applicants should be comfortable with most of the following:

• Adversarial testing
• Analytical writing
• Fact-checking
• Rubric-based grading
• Subject-matter expertise

Also useful

Experience in one or more of the following is an advantage:

• A degree or professional background in law, medicine, finance or software.
• Prior red-teaming, model evaluation or annotation experience.
• Experience writing or applying assessment rubrics.

Engagement details

• Contract type: Freelance / independent contractor
• Location: United States
• Language: English
• Commitment: 3 to 5 hours per day
• Schedule: Flexible, project-based
• Rate of pay: USD 32 - 45 per hour
• Payment: Local bank transfer, Payoneer, Airtm, PayPal, or ACH deposit
• Start date: 1 Sep 2026

Every application goes through screening before a decision. You can follow each stage on your application page.

Service rates

Compensation is based on tasks completed and project availability. Actual earnings vary with task volume and with the quality and productivity standards set for the programme.

Independent contractor relationship

This opportunity is offered on an independent contractor basis. Contributors choose when and how much they work, subject to project availability and quality requirements. It does not create an employment relationship with PortAnnotate.

---

Frequently asked questions