Senior Data Engineer (Databricks Lead)

We are looking for a Senior Data Engineer (Databricks Lead)  with 12-16 years of experience to lead the design, development, and optimization of enterprise-scale data platforms. The ideal candidate will have deep expertise in Databricks, Apache Spark, Snowflake, Kafka, Python, SQL, and AWS , along with experience building scalable data architectures that support analytics, real-time processing, AI/ML, and Generative AI initiatives. .

Lead the architecture, design, and implementation of scalable, secure, and high-performance data platforms using Databricks, Snowflake, and AWS . Design and optimize large-scale batch and real-time data pipelines using Spark and Kafka. Develop and maintain robust ETL/ELT frameworks to ensure data reliability, scalability, and performance. Implement real-time data ingestion and processing solutions using Kafka and Spark Streaming . Build automated data integration frameworks leveraging REST APIs, GraphQL APIs, and Databricks APIs . Design cloud-native data solutions aligned with enterprise governance, security, and compliance standards. Develop data models using Star Schema, Snowflake Schema, Kimball, and Inmon methodologies . Provide technical leadership, architecture guidance, and mentorship to engineering teams. Collaborate with Analytics, Data Science, and AI teams to enable Machine Learning, Generative AI, and LLM-based solutions . Troubleshoot and resolve platform performance bottlenecks and scalability challenges. Drive engineering best practices, code quality standards, and data governance initiatives.

Technical Skills Strong expertise in Databricks and Apache Spark . Hands-on experience with Snowflake and modern data warehouse architectures. Proficiency in Kafka and real-time stream processing. Strong programming skills in Python and advanced SQL . Experience with AWS Cloud Services . Expertise in designing and implementing ETL/ELT architectures . Strong understanding of data modeling concepts and methodologies . Experience integrating systems using REST and GraphQL APIs .

Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field. 12-16 years of experience in Data Engineering and Data Platform Architecture. Proven experience leading enterprise-scale data modernization and cloud migration initiatives.

Experience with AI/ML data platforms, Generative AI, and LLM enablement . Exposure to Data Governance and Data Quality frameworks . Strong stakeholder management, communication, and technical leadership skills.