Senior Data Platform Engineer

What if the work you did every day could impact the lives of people you know? Or all of humanity? At Illumina, we are expanding access to genomic technology to realize health equity for billions of people around the world. Our efforts enable life-changing discoveries that are transforming human health through the early detection and diagnosis of diseases and new treatment options for patients. Working at Illumina means being part of something bigger than yourself. Every person, in every role, has the opportunity to make a difference. Surrounded by extraordinary people, inspiring leaders, and world changing projects, you will do more and become more than you ever thought possible. Senior Data Engineer Role Overview The Databricks Platform Engineer is the technical owner of our Databricks Lakehouse platform — the shared foundation on which enterprise data products, analytics, reporting and AI/ML workloads are built. This is a hands-on individual-contributor role responsible for the reliability, scalability, security, automation, governance and cost efficiency of the Databricks platform. The successful candidate will have production experience operating Databricks as an enterprise platform, not just building on it. They will work AI-natively and set that standard for the team. You will own the Data Platforms domain within a cross-skilled Data & Analytics Platforms engineering team. You will establish platform engineering standards, design and build infrastructure automation, create reusable reference architectures and engineering patterns, and ensure the platform can be operated reliably at enterprise scale. You will also mentor engineers and help build platform engineering capability within the India team.

Platform ownership Own workspace- and account-level architecture: Unity Catalog, compute policies, Lakeflow Jobs and shared compute infrastructure, identity and service principals, secrets management, audit logging, system tables, storage integration, platform upgrades and environment promotion across development, test and production. Design, build and review infrastructure-as-code, CI/CD and observability capabilities for the platform, establishing reusable standards and automation for the wider engineering team. Own platform reliability: availability, capacity, incident management, root-cause analysis, and DR/BCP; define and meet platform SLOs for shared platform services. Run Databricks FinOps: consumption monitoring, cost optimisation, budget and anomaly management, and chargeback/showback with transparent unit costs for consuming teams. Implement platform security and governance controls: identity federation/SCIM, service principals, network controls, secrets, RBAC/ABAC, governed tags, row/column security and masking, PII handling, lineage and audit. Enable and operate the platform capabilities required for AI/ML workloads — MLflow, Model Serving, vector search and GenAI infrastructure patterns — partnering with analytics and AI teams on workload architecture, governance and cost controls. Build and maintain reference implementations and paved-road patterns (PySpark, SQL, Delta Lake, Auto Loader) that show engineering teams how to use the platform; enable consuming teams rather than owning delivery of their data products. Treat the platform as an internal product: understand consuming-team needs, define paved-road capabilities and self-service patterns, track adoption and reliability, and continuously reduce developer friction. Own the platform roadmap: upgrades, new capabilities, deprecations. Engineering standards Contribute to the team engineering handbook and architecture decisions; lead code reviews on platform and foundation work. Take part in architecture and engineering forums, communicating trade-offs clearly to peers and global stakeholders. Maintain reusable patterns and automation used across the team. Establish AI-native engineering practice for the team: how AI assistants are used