← WorkMundi · 1M+ jobs from around the world, liveSign inCreate free account

Founding Software Engineer - AI Task Benchmark & Evaluation Infrastructure

LH2 AI Labs · Bangalore

📅 13/08/2026
🔔 Alert me about jobs like this
No password, no sign-up. Just the email — and you can leave the list anytime.
🔓 Apply — free →
Opens this job on WorkMundi. The account is free and takes under a minute.

View and apply on WorkMundi →

🎁 Before you apply, rehearse this interview. Create your free WorkMundi account and get an Interview Training on HelpsYouSpeak — no cost, no card. I want my training →
About LH2 AI Labs Built by second-time founders who have built and sold companies before, LH2 AI Labs is building the post-training infrastructure for frontier AI models. We bring private, high-quality institutional datasets and vetted domain experts into frontier AI pipelines across verticals such as coding, computer use, agentic workflows, medical, audio, and more. For AI to keep progressing, it needs high-quality training data drawn from real production use cases. The public web has already been crawled and trained onthere is limited new signal left there. That is where we come in. Our vision is to create a world where frontier models can access high-quality data on tap, the same way they access compute today. About the roleYou will create high-quality, SWE-bench-style benchmark tasks from real-world software repositories, and build the pipelines and tooling that make task creation faster, more reliable, and more scalable. This is a dual-mandate role: roughly half hands-on task authoring and review, and half engineering the automation that the whole team depends on. Key responsibilitiesEvaluate real-world software repositories for reproducibility, test quality and suitability for benchmark task development.Understand unfamiliar architectures, dependencies and services, and establish reliable local or containerized execution environments.Identify self-contained bugs and feature changes from issues, pull requests, commits and repository history.Construct complete benchmark tasks from selected bugs and feature changes, including the task setup, specification, tests and containerized execution environment.Write precise, implementation-neutral task specifications that define the expected behavior without revealing the solution.Create and review tests that measure observable behavior, cover important edge cases and allow valid alternative implementations.Verify that the original revision demonstrates the expected failure, the reference solution passes and unrelated behavior remains stable.Independently review tasks created by other engineers and document clear, evidence-based release decisions.Improve authoring standards, validation automation and review practices while mentoring less-experienced engineers.Design and maintain the task-development pipeline: automated environment builds, validation harnesses (fail-before/pass-after checks, determinism and flakiness runs) and task packaging for evaluation at scale.Build tooling that reduces per-task authoring effort, including repository ingestion, candidate mining from issues and pull requests, containerized execution templates and CI integration for task validation.Automate quality gates so that validation evidence is generated, checked and archived without manual steps.Improve authoring standards, validation automation and review practices while mentoring less-experienced engineers. Required qualifications4+ years of professional software-engineering experience with strong debugging and code-review skills.Proficiency in at least one relevant ecosystem, such as Python, JavaScript/TypeScript, Java or Go.Hands-on experience with Git, Linux, Docker, CI/CD, dependency management and automated testing.Experience building internal tooling or CI/CD automation that other engineers depend on, such as test harnesses, build pipelines or containerized development environments.Ability to understand unfamiliar codebases, multi-service dependencies and complex build or test failures.Strong technical writing and sound judgment about whether requirements and tests evaluate behavior fairly. Preferred experienceExperience with agent-evaluation harnesses and benchmark tooling such as Harbor (agent eval framework), SWE-bench / SWE-gym style task pipelines, Terminal-Bench, or equivalent internal systems.Platform or infrastructure engineering: orchestration of containerized workloads, artifact registries, build caching or large-scale batch execution.Monorepos, microservices, open-source projects or containerized test infrastructure. What success looks likeReleased tasks are reproducible, deterministic and supported by complete validation evidence.Task specifications and tests remain aligned without leaking or over-constraining the implementation.Reviews identify defects early, maintain a low rework rate and improve team quality and throughput over time.Validation and environment setup are increasingly automated, measurably reducing time-per-released-task and eliminating manual verification steps.Tooling you build is adopted by the team and reduces onboarding time for new task authors. .
Read the rest of the job →
For people searching Engineer

144,883 engineer jobs are open right now

Here's how to pick the right one and stand out in your application.

144.883Jobs
31.687IN
81%EN

That number is real. WorkMundi's database shows 144,883 open engineer roles across the world. India has the most with 31,687 jobs, followed by the United States with 30,084. If you just finished reading one job ad and felt paralyzed by choice, you're not alone—but this scale is actually an advantage. It means you can afford to be selective.

Start by geography and language. The majority of engineer ads—117,837 of them—have the job posting text written in English. Use that as one filter, but remember: the ad text language tells you nothing about whether the role actually requires you to speak English day-to-day. Read the job description carefully. Then check which countries have the volume you're targeting. Singapore, Poland, and Australia round out the top five after India and the US.

Next, learn who's hiring. Accenture has posted 2,801 engineer roles. andurilindustries, speechify, and jobgether are also actively recruiting. If you're applying to one of these names, research their hiring patterns and interview style before you apply. That homework pays off.

When you interview, expect the question every engineer hears: 'Tell me about a time you had to debug a problem that wasn't in your job description.' Have a specific story ready—not a general one. Name the tools, the deadline pressure, and what you learned. Hiring managers listen for whether you see problem-solving as part of the role itself, not a favour.

👁 21 have read this
0 comments
Want to comment?

Leave your e-mail to comment, react and follow the posts for your role. It is free.

Similar jobs

Job on WorkMundi — the world's largest job board. See more jobs from every continent, updated live.

📢
🎁

Before you apply, rehearse this interview.

Create your free WorkMundi account and get an Interview Training on HelpsYouSpeak — no cost, no card.

I want my training →