🎁 Before you apply, rehearse this interview. Create your free WorkMundi account and get an Interview Training on HelpsYouSpeak — no cost, no card. I want my training →
The ideal candidate will have strong hands-on experience with Python, APIs, Generative AI, LLMs, and Agentic Systems, along with a strong understanding of AI evaluation and benchmarking. You will work closely with engineering and product teams to assess agent performance, identify gaps, design evaluation frameworks, and drive improvements in overall agent quality and user outcomes. Key Responsibilities Test and evaluate LLM-powered Agentic Systems across different business scenarios and user journeys. Develop and implement evaluation frameworks, datasets, and benchmarks for AI agents. Define and measure key agent performance metrics, including:Tool-calling accuracy and success rateResponse accuracy and relevanceReliability and consistencyTask completion rateMulti-turn conversation performanceOverall system effectiveness Design evaluation datasets, including questions, expected answers, test cases, and multi-turn conversational scenarios. Benchmark agent performance across different models, prompts, tools, workflows, and system configurations. Analyze agent failures and identify root causes and areas for improvement. Evaluate the agent's ability to correctly understand user intent, reason through tasks, select appropriate tools, and generate accurate responses. Work with large volumes of business documentation, product information, and data to quickly understand complex business domains. Develop Python-based scripts and tools for automated evaluation, testing, data processing, and analysis. Work with APIs and system integrations to test agent interactions with external tools and services. Collaborate with AI/ML engineers, software engineers, product managers, and domain experts to continuously improve the agent. Monitor evaluation results and provide data-driven recommendations to improve agent quality and user experience. Contribute to building scalable processes for continuous evaluation and regression testing of Agentic AI systems. Required Skills & Experience Technical Skills Strong programming experience in Python. Solid understanding of REST APIs, API integrations, and system interactions. Strong understanding of Generative AI and Large Language Models (LLMs). Deep understanding of Agentic AI systems, including:AI agents and workflowsTool/function callingPrompting and context managementMulti-turn conversationsAgent orchestrationRetrieval and external knowledge integration Experience with AI/LLM evaluation and benchmarking. Experience designing datasets and test scenarios for AI systems. Strong analytical skills with the ability to work with evaluation data and identify patterns, failures, and performance gaps. .
Here's how to pick the right one and stand out in your application.
144.883Jobs
31.687IN
81%EN
That number is real. WorkMundi's database shows 144,883 open engineer roles across the world. India has the most with 31,687 jobs, followed by the United States with 30,084. If you just finished reading one job ad and felt paralyzed by choice, you're not alone—but this scale is actually an advantage. It means you can afford to be selective.
Start by geography and language. The majority of engineer ads—117,837 of them—have the job posting text written in English. Use that as one filter, but remember: the ad text language tells you nothing about whether the role actually requires you to speak English day-to-day. Read the job description carefully. Then check which countries have the volume you're targeting. Singapore, Poland, and Australia round out the top five after India and the US.
Next, learn who's hiring. Accenture has posted 2,801 engineer roles. andurilindustries, speechify, and jobgether are also actively recruiting. If you're applying to one of these names, research their hiring patterns and interview style before you apply. That homework pays off.
When you interview, expect the question every engineer hears: 'Tell me about a time you had to debug a problem that wasn't in your job description.' Have a specific story ready—not a general one. Name the tools, the deadline pressure, and what you learned. Hiring managers listen for whether you see problem-solving as part of the role itself, not a favour.