Staff Data Engineer (Data & AI solutions)

At Palo Alto Networks®, we’re united by a shared mission—to protect our digital way of life. We thrive at the intersection of innovation and impact, solving real-world problems with cutting-edge technology and bold thinking. Here, everyone has a voice, and every idea counts. If you’re ready to do the most meaningful work of your career alongside people who are just as passionate as you are, you’re in the right place.

In order to be the cybersecurity partner of choice, we must trailblaze the path and shape the future of our industry. This is something our employees work at each day and is defined by our values: Disruption, Collaboration, Execution, Integrity, and Inclusion. We weave AI into the fabric of everything we do and use it to augment the impact every individual can have. If you are passionate about solving real-world problems and ideating beside the best and the brightest, we invite you to join us! We believe collaboration thrives in person. That’s why most of our teams work from the office full time, with flexibility when it’s needed. This model supports real-time problem-solving, stronger relationships, and the kind of precision that drives great outcomes.

IDIRA by Palo Alto Networks seeks an experienced Senior Staff Data Engineer to join our AI and Data Solutions unit and champion an AI-first strategy. You will spearhead the development of Data Lakehouse processes, model core product domains, and define data engineering best practices across the organization. Serving as a technical project lead, you will provide strong direction and collaborate with Data Science, Analytics, R&D, and IT teams to elevate onboarding workflows. This role demands a proactive approach to optimizing system capabilities and driving cross-functional data initiatives.

Own the design, implementation, and optimization of scalable Data Lakehouse and Data Warehouse architectures across diverse cloud ecosystems. Spearhead the development of robust ETL/ELT pipelines and data ingestion processes utilizing SQL, Python, PySpark, and DBT. Apply advanced data modeling techniques to structure complex data assets into optimized facts, dimensions, and partitions. Partner with cross-functional leadership across data science, analytics, R&D, and IT to identify requirements and improve workflows. Orchestrate complex data workflows combining tools like Airflow and DBT with advanced Gen-AI development workflows using CloudCode. Define, document, and popularize data engineering best practices, design patterns, and data quality standards across the unit. Evaluate and implement modern data technologies spanning vendor-specific and open-source landscapes including GCP, AWS, and Apache Iceberg. Provide technical direction and hands-on guidance to ensure high-performance execution, reliable pipeline delivery, and proactive problem-solving.

Required Qualifications 3+ years of experience with advanced data modeling techniques and concepts (e.g., Facts, Dimensions, Partitions). 3+ years of experience designing and implementing end-to-end data ingestion and ETL processes within Data Lakehouse architectures. 3+ years of hands-on experience programming in SQL and Python, PySpark, Java, or Scala. Demonstrated experience leading technical projects, driving stakeholder alignment, and navigating cross-functional dynamics. Proven experience working within cloud environments (AWS or GCP), with a strong requirement for AWS. Deep understanding of the modern data landscape and open-source tools (e.g., Iceberg, DBT, Airflow). Proven capability in using and orchestrating Gen-AI workflows based on CloudCode and Cursor.

Deep expertise in the AWS ecosystem, including Glue, Athena, SageMaker, and EMR. Advanced experience with Data Warehouse technologies, especially Apache Iceberg or BigQuery. Leadership or mentorship experience within a data engineering or analytics