Senior CE Data Engineer

At Lilly, the work is demanding because patients are waiting. We unite caring with discovery to help make life better for people around the world, knowing that every decision, every detail, and every day matters. Headquartered in Indianapolis, Indiana, our over 50,000 employees around the globe take on complex challenges to discover and deliver life-changing medicines, strengthen how health is understood and managed, and support the communities we serve. This is hard, urgent, selfless work—but it’s work worth doing. If you’re driven by purpose and ready to bring your best to work that truly matters for patients, we invite you to join us.  Organization Overview Lilly CIA powers the enterprise with governed, AI-ready data products, a converged semantic layer, and the identity and policy infrastructure behind Lilly's digital experiences. Within CIA, the LillyDirect Covered Entity (CE) Pharmacy pod builds and operates the data platform that sits inside the pharmacy's covered-entity trust boundary — handling identified PHI under HIPAA with strict isolation from parent-side systems. This role implements that platform on Databricks.

Summary The CE Data Engineer builds and operationalizes the CE Pharmacy data platform on Databricks — implementing the medallion pipelines, data contracts, identity crosswalk logic, and isolation controls that the CE Data Architect designs. This is a hands-on engineering role: writing production pipelines, evaluating and applying Databricks capabilities and integration patterns, and holding the line on data quality, testing, and monitoring so the platform is reliable, auditable, and built to scale. Works in close, day-to-day partnership with the CE Data Architect and the Associate Director of CE Data Engineering — turning architectural intent into running code — and with Analytics Engineers and the DataHub Product Owner to ship governed data products.

Design and build scalable, efficient Databricks pipelines implementing the CE Architect's canonical data models across the medallion (bronze → silver → gold), scoped entirely within the CE trust boundary. Evaluate and apply Databricks capabilities and integration patterns — Unity Catalog, Delta Lake, Databricks Workflows, serverless compute, Lakebase, ingestion connectors — selecting the right tool for each pipeline given performance, cost, and scalability constraints. Implement the identity crosswalk and PHI classification logic defined by the CE Architect (Reltio MDM, Auth0/Passport, Datavant tokens, Scriptly, Genesys, Prescryptive, Transcend), safeguarding merge/split integrity in code. Build to ODCS data contracts — implement schema, quality, freshness/SLA, and lineage requirements as enforced pipeline logic, not documentation. Implement row/column-level security, masking, and tokenization boundaries so PHI isolation is enforced at the platform layer, in partnership with the Policy-as-Code Engineer's OPA/Rego policies. Own end-to-end pipeline development lifecycle — requirements to prototyping to production deployment and maintenance — for CE data products, in partnership with Analytics Engineers and the DataHub Product Owner. Build and maintain CI/CD pipelines (Github Actions [CI], Git-based promotion dev → test → prod) for CE data, contract, policy, and agent artifacts. Automate data ingestion and product creation to reduce manual pipeline maintenance and onboarding time for new CE data sources. Partner with the CE Data Architect on reference architecture and patterns, providing implementation feedback that keeps designs buildable and performant at scale. Build the data pipelines that lets CE Skills and agents consume governed data — ensuring PHI classification and consent travel with the data into agentic consumption paths.

Bachelor's degree in Computer Science, Engineering, Information Technology, or similar degree. Experience in data engineering, with a focus on