📝 AI Task Creator, are you there?

Hey there! We're looking for an AI Task Creator for a US-based AI research company building custom reinforcement learning environments and coding agent benchmarks. If you are systematic and autonomous with Python or Go and Docker, keep reading!

🌎 100% Remote

💵 Salary in USD

📍 LATAM (Brazil, Argentina, or Colombia) - Reporting to US, EST

🕒 Contract, hourly (20+ hours/week)

🇬🇧 Advanced English

About our client 🤓

You will be working for our client, a US-based AI research company that builds custom reinforcement learning environments and benchmarks for advanced AI coding agents. They design training and evaluation data from scratch to improve specific agent capabilities, and each task you build contributes directly to that pipeline. This is a contract role that can be part-time or full-time depending on your availability.

About Athyna 🙋🏻

Athyna is a Global Talent Platform, helping align world-class talent with the best employment opportunities across the globe.

We believe in the future of work. That's why we are a 100% globally distributed team across 5 continents. We believe in people, and we value freedom, integrity, quality, and sustainability.

We're proudly carbon neutral as an organization. We're very keen on social impact - we truly care about everyone on our team, and we want to make the world a better place for all of us.

In your day-to-day, you will 👇🏼

Source and pin large, complex open-source repositories that host a real planning challenge for the agent
Write task prompts that state the problem without revealing the plan or solution, ensuring genuine difficulty comes from the code and not the wording
Design roughly ten binary yes/no rubrics per task, each testing one dimension of planning or research without leaking the solution
Generate and review a gold trajectory that passes every rubric, then annotate each tool call on a 1 to 5 scale for how essential it was to the solution
Build deterministic test suites with pass-to-pass and fail-to-pass coverage that verify whether a final implementation is actually correct
Own the full task lifecycle from ideation through QC and refinement iterations, typically 20 to 30 hours per task

Skills you have 🚀

Strong, hands-on proficiency in Python and Go, with the ability to read and reason about unfamiliar codebases quickly
Solid understanding of the software development lifecycle in a verifiable, binary-outcome domain where correctness is what matters
Comfortable working with the concept of agent trajectories and judging how essential a given step was to reaching a solution
Familiarity with common task shapes such as bug fixes and basic service failures, and the ability to reason about which shape fits a given problem
Experience using AI coding tools as part of your daily workflow
Familiarity with Docker and containerized test environments
Ability to debug real codebases without a test suite or a clear error message pointing to the cause

Skills that will make you stand out 💥

Reinforcement learning experience, ideally with long-horizon tasks
Experience contributing to or maintaining a large real-world open-source repository
Prior task-creation or data-annotation work for an RL or agent-training pipeline
Comfort reading unfamiliar languages or codebases beyond Python and Go
Master's degree in Computer Science from a top-tier university