🎁 Before you apply, rehearse this interview. Create your free WorkMundi account and get an Interview Training on HelpsYouSpeak — no cost, no card. I want my training →
Role Description Core Responsibilities Data Pipeline Development Design and implement robust ETLELT pipelines using PySpark Spark SQL and Python Performance Optimization Optimize Databricks clusters and code for efficient largescale data processing Data Lakehouse Management Implement Delta Lake architecture for highquality data storage management and performance Data Modeling Architecture Create scalable data models and analytical structures System Integration Automation Integrate Databricks with data lakes eg ADLS and orchestrate workflows eg Azure Data Factory Airflow Monitoring Support Monitor job performance troubleshoot failures and ensure data quality Key Qualifications Skills Experience 35 years in data engineering or related roles Technical Proficiency Strong skills in PySpark Python SQL and Spark optimization Platforms Handson experience with Azure Databricks preferred AWS Databricks or GCP Databricks Tools Technologies Knowledge of Delta Lake Unity Catalog Databricks SQL and CICD tools Git Databases Experience with Data Warehousing and Big Data technologies .