{"id":1248,"date":"2026-08-17T08:59:57","date_gmt":"2026-08-17T08:59:57","guid":{"rendered":"https:\/\/www.hardwinsoftware.com\/blog\/?post_type=jobpost&p=1248"},"modified":"2026-08-17T08:59:58","modified_gmt":"2026-08-17T08:59:58","slug":"senior-data-engineer","status":"publish","type":"jobpost","link":"https:\/\/www.hardwinsoftware.com\/blog\/?jobpost=senior-data-engineer","title":{"rendered":"Senior Data Engineer"},"content":{"rendered":"\n

Location:<\/strong> Bangalore
Employment Type:<\/strong> Full-Time
Experience:<\/strong> 10+ Years
Department:<\/strong> Data & Analytics<\/p>\n\n\n\n

About the Role<\/h2>\n\n\n\n

We are seeking a highly skilled and experienced Senior Data Engineer<\/strong> to design, develop, optimize, and maintain scalable data platforms and high-performance ETL\/ELT pipelines across Microsoft Azure and Snowflake<\/strong>.<\/p>\n\n\n\n

The ideal candidate will have strong expertise in Python, SQL, PySpark, Azure Databricks, Azure Data Factory, Snowflake, Apache Airflow, Azure Synapse, and modern data architecture<\/strong>. This role requires a hands-on technical leader who can work closely with architects, business stakeholders, DevOps teams, and data consumers to deliver reliable, scalable, secure, and cost-effective data solutions.<\/p>\n\n\n\n

Key Responsibilities<\/h2>\n\n\n\n
    \n
  • Lead the design, development, testing, deployment, and maintenance of high-performance ETL\/ELT pipelines<\/strong> using Azure and Snowflake.<\/li>\n\n\n\n
  • Design and optimize modern Azure Lakehouse architectures<\/strong> using ADLS Gen2, Delta Lake, Azure Databricks, and Azure Synapse Analytics.<\/li>\n\n\n\n
  • Build scalable data ingestion frameworks supporting:\n
      \n
    • Batch data processing<\/li>\n\n\n\n
    • Real-time\/streaming data<\/li>\n\n\n\n
    • API-based ingestion<\/li>\n\n\n\n
    • External data sources<\/li>\n<\/ul>\n<\/li>\n\n\n\n
    • Develop and optimize complex SQL queries, stored procedures, analytical functions, and data transformation logic<\/strong>.<\/li>\n\n\n\n
    • Perform performance tuning and cost optimization across Spark jobs, Snowflake warehouses, SQL pools, and data pipelines<\/strong>.<\/li>\n\n\n\n
    • Develop scalable data models supporting BI, analytics, reporting, and machine learning<\/strong> use cases.<\/li>\n\n\n\n
    • Implement robust data quality, validation, governance, metadata, and lineage<\/strong> processes.<\/li>\n\n\n\n
    • Work with Microsoft Purview and Collibra for data cataloging, classification, governance, and lineage.<\/li>\n\n\n\n
    • Integrate APIs, streaming platforms, IoT sources, and other external data systems.<\/li>\n\n\n\n
    • Implement automated testing using unit tests, integration tests, Great Expectations, and dbt tests<\/strong>.<\/li>\n\n\n\n
    • Design and maintain CI\/CD pipelines using GitHub Actions, Azure DevOps, and Jenkins<\/strong>.<\/li>\n\n\n\n
    • Implement Infrastructure as Code using Terraform or Bicep<\/strong>.<\/li>\n\n\n\n
    • Monitor and troubleshoot production data systems using Azure Monitor, Log Analytics, and other observability tools<\/strong>.<\/li>\n\n\n\n
    • Collaborate with DevOps and Platform Engineering teams to automate infrastructure and application deployments.<\/li>\n\n\n\n
    • Troubleshoot complex issues involving distributed systems, cloud infrastructure, networking, data processing, and data platforms.<\/li>\n\n\n\n
    • Establish and improve engineering standards, development practices, automation, CI\/CD, and IaC processes.<\/li>\n\n\n\n
    • Conduct code reviews and mentor junior and mid-level data engineers.<\/li>\n\n\n\n
    • Maintain comprehensive architectural, technical, and operational documentation.<\/li>\n<\/ul>\n\n\n\n

      Required Skills & Qualifications<\/h2>\n\n\n\n
        \n
      • 5+ years<\/strong> of professional experience in Data Engineering or a related field.<\/li>\n\n\n\n
      • Strong programming skills in Python, SQL, and PySpark<\/strong>.<\/li>\n\n\n\n
      • Advanced hands-on experience building and optimizing ETL\/ELT pipelines using:\n
          \n
        • Azure Data Factory (ADF)<\/li>\n\n\n\n
        • Azure Databricks<\/li>\n\n\n\n
        • Apache Spark<\/li>\n\n\n\n
        • Delta Lake<\/li>\n\n\n\n
        • Apache Airflow<\/li>\n\n\n\n
        • Azure Functions or Azure Synapse Pipelines<\/li>\n<\/ul>\n<\/li>\n\n\n\n
        • Experience integrating APIs, streaming data, and external data sources.<\/li>\n\n\n\n
        • Strong understanding of data governance and data quality frameworks<\/strong>.<\/li>\n\n\n\n
        • Hands-on experience with Microsoft Purview<\/strong> and\/or Collibra<\/strong>.<\/li>\n\n\n\n
        • Experience implementing CI\/CD pipelines using GitHub Actions, Azure DevOps, or Jenkins<\/strong>.<\/li>\n\n\n\n
        • Experience with Infrastructure as Code using Terraform or Bicep<\/strong>.<\/li>\n<\/ul>\n\n\n\n

          Good to Have<\/h2>\n\n\n\n
            \n
          • Experience with Azure Kubernetes Service (AKS)<\/strong> and\/or Dockerized workloads.<\/li>\n\n\n\n
          • Experience developing high-performance APIs using FastAPI, Flask, or Django<\/strong>.\n
              \n
            • Azure Event Hub<\/li>\n\n\n\n
            • Azure Stream Analytics<\/li>\n\n\n\n
            • Apache Kafka<\/li>\n\n\n\n
            • OneLake<\/li>\n\n\n\n
            • Lakehouse<\/li>\n\n\n\n
            • Data Pipelines<\/li>\n\n\n\n
            • Warehouses<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n

              Senior-Level Behavioral Expectations<\/h2>\n\n\n\n
                \n
              • Demonstrate a strong ownership mindset and proactively resolve technical challenges.<\/li>\n\n\n\n
              • Translate complex business and technical requirements into clear, actionable engineering tasks.<\/li>\n\n\n\n
              • Promote engineering excellence, continuous learning, experimentation, and innovation.<\/li>\n\n\n\n
              • Mentor team members and contribute to the development of engineering best practices.<\/li>\n\n\n\n
              • Take responsibility for the reliability, scalability, security, and performance of data platforms.<\/li>\n<\/ul>\n\n\n\n

                What We Offer<\/h2>\n\n\n\n
                  \n
                • Opportunity to work on modern Azure, Snowflake, Lakehouse, and Data Engineering technologies<\/strong>.<\/li>\n\n\n\n
                • Exposure to large-scale data platforms, analytics, cloud, streaming, and AI\/ML workloads.<\/li>\n\n\n\n
                • A collaborative and technically driven work environment.<\/li>\n\n\n\n
                • Opportunities to take ownership of challenging data engineering initiatives.<\/li>\n\n\n\n
                • Career growth and continuous learning opportunities.<\/li>\n<\/ul>\n\n\n\n

                  Apply now and join our team to build the next generation of scalable and intelligent data solutions.<\/strong><\/p>\n\n\n\n

                  <\/p>\n","protected":false},"excerpt":{"rendered":"

                  Location: BangaloreEmployment Type: Full-TimeExperience: 10+ YearsDepartment: Data & Analytics About the Role We are seeking a highly skilled and experienced Senior Data Engineer to design, develop, optimize, and maintain scalable data platforms and high-performance ETL\/ELT pipelines across Microsoft… <\/p>\n","protected":false},"author":1,"featured_media":0,"menu_order":0,"template":"","jobpost_category":[],"jobpost_job_type":[],"jobpost_location":[],"jobpost_tag":[],"class_list":["post-1248","jobpost","type-jobpost","status-publish","hentry"],"yoast_head":"\nSenior Data Engineer - Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.hardwinsoftware.com\/blog\/?jobpost=senior-data-engineer\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Senior Data Engineer - Blog\" \/>\n<meta property=\"og:description\" content=\"Location: BangaloreEmployment Type: Full-TimeExperience: 10+ YearsDepartment: Data & Analytics About the Role We are seeking a highly skilled and experienced Senior Data Engineer to design, develop, optimize, and maintain scalable data platforms and high-performance ETL\/ELT pipelines across Microsoft...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.hardwinsoftware.com\/blog\/?jobpost=senior-data-engineer\" \/>\n<meta property=\"og:site_name\" content=\"Blog\" \/>\n<meta property=\"article:modified_time\" content=\"2026-08-17T08:59:58+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.hardwinsoftware.com\/blog\/?jobpost=senior-data-engineer\",\"url\":\"https:\/\/www.hardwinsoftware.com\/blog\/?jobpost=senior-data-engineer\",\"name\":\"Senior Data Engineer - Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.hardwinsoftware.com\/blog\/#website\"},\"datePublished\":\"2026-08-17T08:59:57+00:00\",\"dateModified\":\"2026-08-17T08:59:58+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.hardwinsoftware.com\/blog\/?jobpost=senior-data-engineer#breadcrumb\"},\"inLanguage\":\"en\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.hardwinsoftware.com\/blog\/?jobpost=senior-data-engineer\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.hardwinsoftware.com\/blog\/?jobpost=senior-data-engineer#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.hardwinsoftware.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Jobs\",\"item\":\"https:\/\/www.hardwinsoftware.com\/blog\/?post_type=jobpost\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Senior Data Engineer\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.hardwinsoftware.com\/blog\/#website\",\"url\":\"https:\/\/www.hardwinsoftware.com\/blog\/\",\"name\":\"Blog\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.hardwinsoftware.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.hardwinsoftware.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.hardwinsoftware.com\/blog\/#organization\",\"name\":\"Blog\",\"url\":\"https:\/\/www.hardwinsoftware.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en\",\"@id\":\"https:\/\/www.hardwinsoftware.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.hardwinsoftware.com\/blog\/wp-content\/uploads\/2025\/01\/HSS-logo-for-social-media-copy.png\",\"contentUrl\":\"https:\/\/www.hardwinsoftware.com\/blog\/wp-content\/uploads\/2025\/01\/HSS-logo-for-social-media-copy.png\",\"width\":1080,\"height\":1080,\"caption\":\"Blog\"},\"image\":{\"@id\":\"https:\/\/www.hardwinsoftware.com\/blog\/#\/schema\/logo\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Senior Data Engineer - Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.hardwinsoftware.com\/blog\/?jobpost=senior-data-engineer","og_locale":"en_US","og_type":"article","og_title":"Senior Data Engineer - Blog","og_description":"Location: BangaloreEmployment Type: Full-TimeExperience: 10+ YearsDepartment: Data & Analytics About the Role We are seeking a highly skilled and experienced Senior Data Engineer to design, develop, optimize, and maintain scalable data platforms and high-performance ETL\/ELT pipelines across Microsoft...","og_url":"https:\/\/www.hardwinsoftware.com\/blog\/?jobpost=senior-data-engineer","og_site_name":"Blog","article_modified_time":"2026-08-17T08:59:58+00:00","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.hardwinsoftware.com\/blog\/?jobpost=senior-data-engineer","url":"https:\/\/www.hardwinsoftware.com\/blog\/?jobpost=senior-data-engineer","name":"Senior Data Engineer - Blog","isPartOf":{"@id":"https:\/\/www.hardwinsoftware.com\/blog\/#website"},"datePublished":"2026-08-17T08:59:57+00:00","dateModified":"2026-08-17T08:59:58+00:00","breadcrumb":{"@id":"https:\/\/www.hardwinsoftware.com\/blog\/?jobpost=senior-data-engineer#breadcrumb"},"inLanguage":"en","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.hardwinsoftware.com\/blog\/?jobpost=senior-data-engineer"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.hardwinsoftware.com\/blog\/?jobpost=senior-data-engineer#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.hardwinsoftware.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Jobs","item":"https:\/\/www.hardwinsoftware.com\/blog\/?post_type=jobpost"},{"@type":"ListItem","position":3,"name":"Senior Data Engineer"}]},{"@type":"WebSite","@id":"https:\/\/www.hardwinsoftware.com\/blog\/#website","url":"https:\/\/www.hardwinsoftware.com\/blog\/","name":"Blog","description":"","publisher":{"@id":"https:\/\/www.hardwinsoftware.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.hardwinsoftware.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en"},{"@type":"Organization","@id":"https:\/\/www.hardwinsoftware.com\/blog\/#organization","name":"Blog","url":"https:\/\/www.hardwinsoftware.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en","@id":"https:\/\/www.hardwinsoftware.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.hardwinsoftware.com\/blog\/wp-content\/uploads\/2025\/01\/HSS-logo-for-social-media-copy.png","contentUrl":"https:\/\/www.hardwinsoftware.com\/blog\/wp-content\/uploads\/2025\/01\/HSS-logo-for-social-media-copy.png","width":1080,"height":1080,"caption":"Blog"},"image":{"@id":"https:\/\/www.hardwinsoftware.com\/blog\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/www.hardwinsoftware.com\/blog\/index.php?rest_route=\/wp\/v2\/jobpost\/1248","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.hardwinsoftware.com\/blog\/index.php?rest_route=\/wp\/v2\/jobpost"}],"about":[{"href":"https:\/\/www.hardwinsoftware.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/jobpost"}],"author":[{"embeddable":true,"href":"https:\/\/www.hardwinsoftware.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"wp:attachment":[{"href":"https:\/\/www.hardwinsoftware.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1248"}],"wp:term":[{"taxonomy":"jobpost_category","embeddable":true,"href":"https:\/\/www.hardwinsoftware.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fjobpost_category&post=1248"},{"taxonomy":"jobpost_job_type","embeddable":true,"href":"https:\/\/www.hardwinsoftware.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fjobpost_job_type&post=1248"},{"taxonomy":"jobpost_location","embeddable":true,"href":"https:\/\/www.hardwinsoftware.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fjobpost_location&post=1248"},{"taxonomy":"jobpost_tag","embeddable":true,"href":"https:\/\/www.hardwinsoftware.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fjobpost_tag&post=1248"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}