Testing Lead -QA (Bengaluru)

Job Title: Testing Lead Deep Learning / LLM / VLM Location Remote / Hybrid / On -site Experience 34 years (hands -on ownership in DL / LLM / GenAI testing) Employment Type Full -time Role Overview We are seeking a hands -on Testing Lead to own quality and documentation for our Deep Learning, LLM, and Vision -Language Model (VLM) products. You will define how we test, measure, document, and communicate AI qualityworking closely with ML, Engineering, and Product teams in a fast -paced startup setting. This role is ideal for someone who believes clear documentation is as critical as good testing , especially for non -deterministic AI systems. What Youll Do Own Quality & Documentation End -to -End - Define testing strategy for LLMs, VLMs, and DL pipelines . - Create and maintain clear, lightweight documentation covering: - Model testing strategies and assumptions - Evaluation metrics and acceptance criteria - Known limitations, risks, and failure modes - Release readiness and quality sign -off - Ensure documentation evolves with models, data, and prompts. LLM / GenAI Testing - Design tests for: - Prompt templates and prompt changes - RAG pipelines (retrieval quality, grounding, hallucination control) - Multi -turn conversations and long -context behaviour - Maintain golden datasets , regression test suites, and test result summaries. - Document prompt behaviour, edge cases, and known model quirks. Vision & Multimodal Testing - Test VLMs for image -text alignment, OCR, captioning, and reasoning. - Document model performance across different image types, quality levels, and domains. - Track and publish model behaviour changes between versions. Automation, MLOps & Reporting - Build Python -based automation for evaluation and regression testing. - Integrate tests into CI/CD and MLOps pipelines . - Produce readable quality reports and dashboards for engineers and leadership. - Monitor and document production issues such as model/data drift and degradation . Build a Quality -First Culture - Establish QA and documentation standards that scale with a startup. - Mentor engineers on writing testable code and meaningful documentation. - Act as the single source of truth for AI quality, testing, and known risks. What were looking For Must -Have - Strong background in software testing with lead or ownership experience . - Hands -on experience testing LLMs, DL models, or GenAI systems . - Strong Python skills for test automation and data validation. - Proven ability to write clear, structured technical documentation . - Understanding of: - Transformer -based models and DL workflows - Model evaluation metrics and non -deterministic system testing - Comfortable working in ambiguity and moving fast in a startup. Nice -to -Have - Experience with VLMs, multimodal models, or computer vision . - Exposure to RAG architectures , vector databases, and embeddings. - Familiarity with tools like LangChain, LlamaIndex, MLflow, or similar. - Experience documenting AI risks, limitations, or compliance requirements. .