Sr. Databricks Engineer

Job Description: Azure / Databricks Data Engineer (9-15 Years Experience) Job Title Senior Azure Databricks Data Engineer Experience 9-15 Years of IT Experience Location Hybrid Job Summary We are seeking a skilled Databricks Engineer with minimum of 5+ years of hands-on in Databricks who can design, develop, and optimize scalable data platforms and analytics solutions using the Databricks Lakehouse Platform. The ideal candidate will have expertise in data engineering, ETL/ELT development, cloud technologies preferably Azure, and big data processing to support enterprise analytics and AI initiatives. Key Responsibilities • Design, develop, and maintain data pipelines using Databricks, Apache Spark, and cloud-native services. • Build and optimize ETL/ELT workflows for large-scale structured and unstructured data. • Develop data models and implement data quality, validation, and governance frameworks. • Integrate data from multiple sources into a unified Lakehouse architecture. • Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency. • Implement security controls, access management, and data governance using Unity Catalog. • Collaborate with business, analytics, and AI/ML teams to deliver trusted data products. • Monitor, troubleshoot, and resolve data pipeline issues. • Support CI/CD, DevOps, and infrastructure automation practices. • Maintain technical documentation and best practices. Required Technical Skills Core Technologies • Databricks Lakehouse Platform • Apache Spark / PySpark • Delta Lake • SQL • Python Data Engineering • ETL / ELT Development • Data Modeling • Data Warehousing • Data Quality & Validation • Streaming & Real-Time Processing Governance & Security • Unity Catalog • Data Lineage • Row-Level Security • Access Control & Compliance • Data Governance Frameworks Cloud & DevOps • Azure / AWS / GCP • Terraform • GitHub Actions / Azure DevOps • CI/CD Pipelines Analytics & AI • Semantic Layers • Data Products • BI Platforms • Machine Learning Support • Generative AI & RAG Architectures Required Qualifications • Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field. • 9-15 years of experience in Data Engineering and Data Warehousing. • Minimum 5+ years of hands-on experience with Azure Data Engineering technologies. • Minimum 4+ years of hands-on experience with Azure Databricks and Spark ecosystem. • Strong understanding of data lake, lakehouse, and cloud-native architecture patterns. • Experience in handling large-scale structured and unstructured datasets. • Strong analytical, problem-solving, and troubleshooting skills. Preferred Qualifications • Microsoft Certified: Azure Data Engineer Associate (DP-203). • Databricks Certified Data Engineer Associate/Professional. • Experience with Snowflake, Power BI, or Microsoft Fabric. • Experience in real-time streaming solutions using Kafka/Event Hubs. • Exposure to Data Governance and Master Data Management initiatives. Soft Skills • Strong stakeholder management and communication skills. • Ability to lead technical initiatives and drive architecture discussions. • Experience working in Agile/Scrum environments. • Excellent documentation and presentation skills. • Strong mentoring and team leadership abilities. Nice to Have • Microsoft Fabric • Power BI • Azure Event Hubs • Kafka • Machine Learning data pipelines • Data Governance tools such as Purview Job Description: Azure / Databricks Data Engineer (9-15 Years Experience) Job Title Senior Azure Databricks Data Engineer Experience 9-15 Years of IT Experience Location Hybrid Job Summary We are seeking a skilled Databricks Engineer with minimum of 5+ years of hands-on in Databricks who can design, develop, and optimize scalable data platforms and analytics solutions using the Databricks Lakehouse Platform. The ideal candidate will have expertise in data engineering, ETL/ELT development, cloud technologies preferably Azure, and big data processing to support enterprise analytics and AI initiatives. Key Responsibilities • Design, develop, and maintain data pipelines using Databricks, Apache Spark, and cloud-native services. • Build and optimize ETL/ELT workflows for large-scale structured and unstructured data. • Develop data models and implement data quality, validation, and governance frameworks. • Integrate data from multiple sources into a unified Lakehouse architecture. • Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency. • Implement security controls, access management, and data governance using Unity Catalog. • Collaborate with business, analytics, and AI/ML teams to deliver trusted data products. • Monitor, troubleshoot, and resolve data pipeline issues. • Support CI/CD, DevOps, and infrastructure automation practices. • Maintain technical documentation and best practices. Required Technical Skills Core Technologies • Databricks Lakehouse Platform • Apache Spark / PySpark • Delta Lake • SQL • Python Data Engineering • ETL / ELT Development • Data Modeling • Data Warehousing • Data Quality & Validation • Streaming & Real-Time Processing Governance & Security • Unity Catalog • Data Lineage • Row-Level Security • Access Control & Compliance • Data Governance Frameworks Cloud & DevOps • Azure / AWS / GCP • Terraform • GitHub Actions / Azure DevOps • CI/CD Pipelines Analytics & AI • Semantic Layers • Data Products • BI Platforms • Machine Learning Support • Generative AI & RAG Architectures Required Qualifications • Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field. • 9-15 years of experience in Data Engineering and Data Warehousing. • Minimum 5+ years of hands-on experience with Azure Data Engineering technologies. • Minimum 4+ years of hands-on experience with Azure Databricks and Spark ecosystem. • Strong understanding of data lake, lakehouse, and cloud-native architecture patterns. • Experience in handling large-scale structured and unstructured datasets. • Strong analytical, problem-solving, and troubleshooting skills. Preferred Qualifications • Microsoft Certified: Azure Data Engineer Associate (DP-203). • Databricks Certified Data Engineer Associate/Professional. • Experience with Snowflake, Power BI, or Microsoft Fabric. • Experience in real-time streaming solutions using Kafka/Event Hubs. • Exposure to Data Governance and Master Data Management initiatives. Soft Skills • Strong stakeholder management and communication skills. • Ability to lead technical initiatives and drive architecture discussions. • Experience working in Agile/Scrum environments. • Excellent documentation and presentation skills. • Strong mentoring and team leadership abilities. Nice to Have • Microsoft Fabric • Power BI • Azure Event Hubs • Kafka • Machine Learning data pipelines • Data Governance tools such as Purview Job Description: Azure / Databricks Data Engineer (9-15 Years Experience) Job Title Senior Azure Databricks Data Engineer Experience 9-15 Years of IT Experience Location Hybrid Job Summary We are seeking a skilled Databricks Engineer with minimum of 5+ years of hands-on in Databricks who can design, develop, and optimize scalable data platforms and analytics solutions using the Databricks Lakehouse Platform. The ideal candidate will have expertise in data engineering, ETL/ELT development, cloud technologies preferably Azure, and big data processing to support enterprise analytics and AI initiatives. Key Responsibilities • Design, develop, and maintain data pipelines using Databricks, Apache Spark, and cloud-native services. • Build and optimize ETL/ELT workflows for large-scale structured and unstructured data. • Develop data models and implement data quality, validation, and governance frameworks. • Integrate data from multiple sources into a unified Lakehouse architecture. • Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency. • Implement security controls, access management, and data governance using Unity Catalog. • Collaborate with business, analytics, and AI/ML teams to deliver trusted data products. • Monitor, troubleshoot, and resolve data pipeline issues. • Support CI/CD, DevOps, and infrastructure automation practices. • Maintain technical documentation and best practices. Required Technical Skills Core Technologies • Databricks Lakehouse Platform • Apache Spark / PySpark • Delta Lake • SQL • Python Data Engineering • ETL / ELT Development • Data Modeling • Data Warehousing • Data Quality & Validation • Streaming & Real-Time Processing Governance & Security • Unity Catalog • Data Lineage • Row-Level Security • Access Control & Compliance • Data Governance Frameworks Cloud & DevOps • Azure / AWS / GCP • Terraform • GitHub Actions / Azure DevOps • CI/CD Pipelines Analytics & AI • Semantic Layers • Data Products • BI Platforms • Machine Learning Support • Generative AI & RAG Architectures Required Qualifications • Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field. • 9-15 years of experience in Data Engineering and Data Warehousing. • Minimum 5+ years of hands-on experience with Azure Data Engineering technologies. • Minimum 4+ years of hands-on experience with Azure Databricks and Spark ecosystem. • Strong understanding of data lake, lakehouse, and cloud-native architecture patterns. • Experience in handling large-scale structured and unstructured datasets. • Strong analytical, problem-solving, and troubleshooting skills. Preferred Qualifications • Microsoft Certified: Azure Data Engineer Associate (DP-203). • Databricks Certified Data Engineer Associate/Professional. • Experience with Snowflake, Power BI, or Microsoft Fabric. • Experience in real-time streaming solutions using Kafka/Event Hubs. • Exposure to Data Governance and Master Data Management initiatives. Soft Skills • Strong stakeholder management and communication skills. • Ability to lead technical initiatives and drive architecture discussions. • Experience working in Agile/Scrum environments. • Excellent documentation and presentation skills. • Strong mentoring and team leadership abilities. Nice to Have • Microsoft Fabric • Power BI • Azure Event Hubs • Kafka • Machine Learning data pipelines • Data Governance tools such as Purview