Skip to main content
StrideRecruit

ML Inference Engineer – Terminal-Bench Task Designer

Toptal

Source: RemoteJobs.org — authorized feed

Location
Remote
Job type
Full-time · Remote
Experience
Not specified
Posted
4 Nov 2026
Last seen live
17 Nov 2026
Source
RemoteJobs.org
Employer
Toptal
View on RemoteJobs.org and apply ↗

This link opens the listing on RemoteJobs.org, where the application continues. It is not confirmed to be the employer's own website.

Job description

Headquarters:

Overview

We are building a benchmark that tests how well AI agents handle real expert work in a terminal. We are hiring ML inference and serving experts to turn real, paid tasks from their jobs into expert-level Terminal-Bench challenges. Each task includes the environment, a working solution, and an automated check that proves the result is correct.

***THIS IS A TASK BASED COMPENSATION MODEL. PLEASE CHECK THE APPLICATION QUESTIONS FOR MORE DETAILS.***

Responsibilities

• Design expert-level Terminal-Bench tasks that turn real, paid ML inference work from your own experience into challenges an AI agent must solve end to end.

• Define the files, systems and environments each task runs on, such as model serving with streaming, so the challenge is realistic, reproducible and command-line solvable.

• Write a working reference solution for every task and verify it runs end to end, sized for about 60 to 75 minutes of expert effort.

• Build automated checks that objectively prove a result is correct, including cases where ML outputs go wrong silently, such as numerical drift or metric bugs.

• Package tasks with Docker and Git so they are reproducible, and document the systems, tools, data and rules involved so reviewers can validate difficulty.

• Iterate on tasks based on reviewer feedback and agent test runs, adjusting difficulty and fixing ambiguities so each task stays expert-level, fair and objectively gradable.

Skills/Experience

• 3+ years doing ML inference or serving work hands-on (not only studying, selling or managing it), or a PhD plus active research in the subfield

• Did this work in the last 3 years and can name the systems, tools, data and rules used

• Trained, served or evaluated models at real scale (multi-GPU or production serving), not only notebooks or API calls

• Hands-on with PyTorch or JAX internals and serving stacks such as vLLM, SGLang or TensorRT-LLM

• Can explain how ML results go wrong without errors (numerical drift, metric bugs) and how to catch them

• Technical fluency in Python and the command line, and with files, CSV/JSON, APIs or databases

• Availability to start by early November 2026 and complete tasks of about 60 to 75 minutes each

Nice-to-have

• AI lab or LLM infrastructure experience

• Contributions to PyTorch, vLLM or SGLang

• GPU profiling experience

• Docker and Git proficiency

• Experience writing automated tests

Process

• Task vetting instead of an interview: complete a vetting task so the client can assess your task design

• Paid per task; each task takes about 60 to 75 minutes

• Work is delivered task by task, with reviewer feedback on submissions

• Expected start in early November 2026

To apply: https://weworkremotely.com/remote-jobs/toptal-ml-inference-engineer-terminal-bench-task-designer

Apply directly on RemoteJobs.org: https://remotejobs.org/remote-jobs/ml-inference-engineer-terminal-bench-task-designer-toptal

Jobs provided by RemoteJobs.org · remotejobs.org

About this job

This page describes one live job: ML Inference Engineer – Terminal-Bench Task Designer at Toptal, listed as Remote.

Frequently asked questions

Where is this job based?
This job is listed as Remote.
What kind of work is this?
This is a Full-time · Remote role at Toptal.
When was this job posted?
This job was posted on 4 Nov 2026.
Where do I apply for this job?
Use the Apply link on this page. It opens the original listing on RemoteJobs.org, where you apply — StrideRecruit does not take applications.
Report this listing

Tell us if this listing looks wrong. We review reports; a report does not remove the listing by itself.

We do not ask for your name or email and do not store your IP address with the report.