👋 Hey there! We are looking for a Technical Program Manager, AI to join our team! In this role, you will own the program behind Athyna Intelligence's benchmark Datasets: the tasks we deliver to frontier AI labs, the experts who author them, and the testing that decides what ships. If you have run an AI evaluation or data program, are comfortable with LLM agents and Docker, and like turning a hand-run process into a tight, repeatable one, keep reading!

🌏 100% Remote role
📍 Reporting to the General Manager
🕒 Full time
🇬🇧 Advanced English

A bit about us 🏡

Athyna is a platform that helps companies hire exceptional talent from around the world with AI precision and a human-first approach. We’re not a recruitment agency. We manage the entire hiring journey, from sourcing and vetting to matching, onboarding, and global compliance, making it easy for companies to build high-performing remote teams without the usual hiring headaches.

We're also a fully remote, globally distributed team ourselves. We believe great work isn't limited by geography, and neither are great careers. Our culture is built around trust, ownership, curiosity, and kindness, creating an environment where talented people can do their best work together.

And it's working. With an engagement score of 91.5% (yes, seriously), we don’t just talk culture, we live it.

Job description 📜

Athyna Intelligence authors and sells benchmark tasks to AI labs: coding tasks written by senior engineers in the Terminal-Bench/Harbor format, measured with Pier and mini-swe-agent, and expert knowledge-work tasks in the style of GDPval, written by professionals in finance, legal, and other fields.

As Technical Program Manager, AI, you'll run the program end to end: manage the authors, test every task, decide what ships, deliver it, and keep the process documented, measured, and increasingly automated.

We're scaling this new product from a handful of task types to a full portfolio across domains, and as our first hire on it, you'll build that infrastructure from scratch, growing the team as volume increases, with engineering support for tooling and automation.

Key responsibilities 🎯

Design and document the end-to-end delivery workflow from scratch: client intake, author assignment, testing, packaging, delivery, and acceptance tracking.
Own the program as its product owner, with targets per customer and task type covering throughput, cost, and delivery.
Manage the network of authors, engineers, and domain experts: onboard them on the playbook, set pace and expectations, keep the best authors busy, and act as their single point of contact.
Run testing for every task type, including pass@k evaluation against frontier models for coding tasks and blind expert grading with LLM-judge scoring for GDPval-style tasks.
Decide verdicts from measured results and give authors clear, fast feedback on where a model got past their task design.
Build the minimum tooling needed to run the program without repeated manual work (tracking, templates, queues), and automate the testing pipeline in collaboration with product and engineering.
Act as the direct point of contact with the client (AI lab) for specs, partial deliveries, and post-delivery feedback.
Keep the playbook current, track spend, and report weekly on what holds, what is queued, and what was dropped.

The successful applicant 🙋🏻

English at a near-native level (oral and written).
Experience building and running an AI evaluation, benchmark, or data program, on the vendor side or within a lab's data team. You know what a hidden test suite, a reference solution, and a pass@k figure mean, and what a GDPval-style rubric-graded expert evaluation looks like.
Hands-on technical fluency: comfortable with Docker, a terminal, APIs, agent transcripts, and test output, and enough scripting (Python or shell, CI or job orchestration) to have replaced a manual process with an automated one before.
Domain judgment: enough working knowledge of how LLM agents behave on coding and knowledge-work tasks to read a transcript or deliverable and explain where the model went wrong.
A strong operator who turns a hand-run process into queues, checklists, templates, and SLAs.
Direct experience with Terminal-Bench, GDPval, or SWE-bench is a plus.
Rigor with numbers: you'd rather rerun a test than report a figure you don't trust.
Experience managing remote freelance engineers and domain experts with fast, direct, written feedback.
Strong analytical and project management skills, with close attention to detail.
Ability to thrive in a dynamic startup environment: goal-oriented, able to execute independently, with guidance available when needed.
Having been part of a team that shipped tasks a frontier model failed reliably is a strong plus.

Benefits we offer ♥️

🏝 30 days paid leave each year to refresh.

🎄 Enjoy all your local public holidays.

🎁 Your birthday also, that one is on us.

👶🏼 If you have a kiddo, we have mum and dad covered with our Parental Leave Policy.

🛩 Work part-time from the beach under our Digital Nomad Policy if you like.

🗣 Enjoy English classes provided by us if you are a non-native speaker.

💥 We have a budget for learning and development opportunities.

💆🏻 Enjoy our Meeting Free Friday policy.

🏥 Healthcare cover.

🍿 Co-work passes to get out of the house from time to time.

🎯 We will gift you equity stock in the company.

We are currently taking applications so if this role sounds like you let us know. Seriously, people don’t apply when they don’t feel confident. Don’t let that be you. Throw your hat in the ring.

We’re excited to dive into this one. ✌️

Tech Project Manager AI