🎁 Before you apply, rehearse this interview. Create your free WorkMundi account and get an Interview Training on HelpsYouSpeak — no cost, no card. I want my training →
Purpose of the role Build and maintain all data pipelines feeding the Databricks lakehouse, ensuring clean, timely, and governed data flows from every source across the Investment Division’s five portfolio clusters and, in later phases, the Group’s operating divisions. Own the bronze-to-silver-to-gold transformation logic, data quality monitoring, security master management, and the Document Intelligence Engine’s ingestion and classification pipelines. Key responsibilities Build and maintain automated ingestion pipelines: Addepar API (daily positions, transactions, cash), Capital IQ API (prices, fundamentals, consensus), Canoe (alternative fund documents), eVestment/Mercer (manager database), email (Outlook Graph API). Implement the medallion architecture: bronze layer (raw immutable data), silver layer (cleaned, conformed, security master mapped, FX normalised, GICS classified, entity ownership resolved), gold layer (analytics-ready tables optimised for ML and SQL queries). Build and maintain the security master: mapping ISIN, CUSIP, SEDOL, Bloomberg ticker, and custodian internal identifiers across all sources, handling corporate actions (mergers, ticker changes, share class conversions). Build the data quality monitoring framework: automated checks for completeness, timeliness, and reconciliation, with exception alerts to the operations analyst. Build the Document Intelligence Engine ingestion pipeline: receive documents from email, file drop, and Canoe; route to the classification and extraction models (built by the ML engineer); store structured output in gold layer tables and raw text in the vector store. Build the reverse pipeline: push model outputs (factor scores, anomaly flags, risk metrics) from the lakehouse to Addepar via API as custom position attributes. In later phases, build ingestion pipelines for operating division ERP systems (Nama core banking, UMG DMS, hotel PMS, IFM CMMS, electronics POS). Monitor pipeline health daily: investigate and resolve ingestion failures, data quality alerts, and schema changes from upstream sources. Optimise pipeline performance and cost: right-size compute clusters, implement incremental processing, manage storage lifecycle. Qualifications and experience 5–8 years of data engineering experience, with at least 2 years on Databricks or equivalent Spark-based platforms. Strong proficiency in Python, SQL, PySpark, and Delta Live Tables. Experience building API integrations and working with financial data feeds (custodian files, market data APIs, fund administrator reports). Understanding of security master management and corporate action processing in financial services. Experience with Azure Data Factory, Azure Blob Storage, and Azure Functions. Familiarity with document processing pipelines (PDF parsing, OCR, text extraction). Bachelor’s degree in computer science, engineering, or equivalent. Detail-oriented with strong debugging and problem-solving skills.
Here's how to pick the right one and stand out in your application.
144.883Jobs
31.687IN
81%EN
That number is real. WorkMundi's database shows 144,883 open engineer roles across the world. India has the most with 31,687 jobs, followed by the United States with 30,084. If you just finished reading one job ad and felt paralyzed by choice, you're not alone—but this scale is actually an advantage. It means you can afford to be selective.
Start by geography and language. The majority of engineer ads—117,837 of them—have the job posting text written in English. Use that as one filter, but remember: the ad text language tells you nothing about whether the role actually requires you to speak English day-to-day. Read the job description carefully. Then check which countries have the volume you're targeting. Singapore, Poland, and Australia round out the top five after India and the US.
Next, learn who's hiring. Accenture has posted 2,801 engineer roles. andurilindustries, speechify, and jobgether are also actively recruiting. If you're applying to one of these names, research their hiring patterns and interview style before you apply. That homework pays off.
When you interview, expect the question every engineer hears: 'Tell me about a time you had to debug a problem that wasn't in your job description.' Have a specific story ready—not a general one. Name the tools, the deadline pressure, and what you learned. Hiring managers listen for whether you see problem-solving as part of the role itself, not a favour.