Azure Data Engineer – Business Intelligence Operations Lead (AVP)

pune
Posted 4 hours ago

The Azure Data Engineer – BI Operations Lead will be responsible for driving the design, development, and oversight of large-scale data engineering and BI solutions across the organization. This role involves working closely with CXOs and business stakeholders to translate requirements into scalable technical implementations using Azure Databricks, Synapse, and ADF. The position ensures operational excellence across ETL/ELT pipelines, data governance, visualization platforms, and BI audits while leading and mentoring a high-performing team.

Key Responsibilities:

  • Collaborate with stakeholders and CXOs to understand source systems and translate business requirements into technical solutions using Databricks, Azure Synapse, and Azure Data Factory.
  • Design, implement, and manage end-to-end ETL/ELT pipelines for MIS automation, system integrations, and streaming ingestion using Delta Live Tables.
  • Lead cloud cost estimation, optimization strategies, and cost controls for new and existing data engineering solutions.
  • Define and implement Datalake governance policies including data purging, archival, and retention strategies.
  • Guide teams on advanced analytics requirements in Qlik and Power BI, including time series, forecasting, correlation, bucketing, hierarchical data, and What-If analysis.
  • Oversee change management processes, including reviewing existing pipelines, SQL/Python code, and visualizations; ensure proper approval workflows through the Change Approval Board.
  • Estimate, deliver, and optimize high-volume data processing pipelines (structured & unstructured).
  • Supervise and ensure accurate delivery of all adhoc data requirements.
  • Set up proactive ETL monitoring and lead troubleshooting and resolution of ETL issues.
  • Ensure adherence to data governance, security standards, and masking policies.
  • Plan and oversee Disaster Recovery (DR) drills for the BI data platform.
  • Ensure all developments follow standard coding practices and maintain audit-ready documentation.
  • Coordinate and execute BI platform audits with internal audit teams.

Qualifications

Must-Have:

  • 10+ years of experience in Azure Data & Analytics engineering.
  • Proven expertise in Databricks, PySpark, Spark SQL, Delta Live Tables, Kafka, Azure Synapse, Azure SQL DB, Azure Data Factory, ADLS, Azure Logic Apps.
  • Strong hands-on skills in Python, PySpark, Azure DevOps, CI/CD pipelines, ETL/ELT design, SSIS, and data ingestion into Azure storage.
  • Experience in data warehouse concepts including denormalization, fact/dimension modeling, and conformance.
  • Deep understanding of Power BI and/or QlikSense with advanced analytics capabilities (Time Intelligence, Time Series, Forecasting, What-If, etc.).
  • Strong SQL development and performance tuning skills for large-scale workloads.
  • Excellent communication skills (written and verbal).
  • Experience leading technical teams and managing BI operations.

Good-to-Have:

  • Experience functioning as a BI Technical Project Manager.
  • Knowledge of NBFC/Banking domain functions (Sales, Credit, Collections, Finance, Accounts).
  • Exposure to large enterprise data governance frameworks and audit processes.
  • Certifications in Azure Data Engineering, Databricks, or related cloud technologies.

In Brief:

Title: Azure Data Engineer – Business Intelligence Operations Lead (AVP)
Qualification: Experienced Azure Data & Analytics professional skilled in Databricks, Synapse, ADF, and BI platforms. Responsible for leading BI operations, managing data platforms, designing ETL/ELT solutions, and driving governance and audits.
Experience Needed: 10+ Years
No. of Openings: 2
Joining Type: Immediate / Within 15 Days

Apply For This Job

A valid email address is required.
A valid phone number is required.