← All roles

Lead Data Engineer

Elastic· IT
Apply on company site
Remote$113.1k - $178.9kfull-timePosted 2w ago

About the role

Lead Data Engineer to design, maintain, and improve the data platform pipelines and infrastructure, collaborating with partners to turn data into usable, trusted assets across the company.

Responsibilities

  • Design, maintain, and improve data pipelines and infrastructure powering the data platform
  • Work across the full data lifecycle—from ingestion and streaming to transformation and delivery—using Apache Airflow, Spark, Iceberg, Kafka, Flink, and dbt Labs
  • Collaborate with stakeholders to understand needs, co-create data contracts, write documentation, and guide teammates on platform usage
  • Monitor and refine pipeline performance and processes to ensure smooth operations
  • Use Terraform to provision and manage resources on Google Cloud Platform (GCP)
  • Help steward the platform by reviewing code and ensuring best practices and standard processes
  • Process access requests and support compliance with platform policies

Qualifications

  • 8+ years of experience in data engineering or related roles
  • Experience with Apache Airflow for orchestrating data workflows
  • Experience with Spark, Flink, Kafka, and dbt Labs for data processing and transformation
  • Understanding of Apache Iceberg for data table formats and data warehouse/lakehouse strategies
  • Skilled in using Terraform for infrastructure-as-code deployments
  • Proficient in Bash, Python, and SQL
  • Knowledge of OLAP, OLTP, Document, and Vector databases
  • Experience with Google Cloud Platform (GCP) for cloud resource management
  • Bonus: Proven experience with data security and access controls for SOX compliant data
  • Bonus: Experience and knowledge of the Elastic stack

Skills

Required:Apache AirflowSparkFlinkKafkadbt LabsApache IcebergTerraformBashPythonSQLGCP
Nice to have:SOX complianceElastic stack

Benefits

  • Stock program eligibility
  • Company-matched 401k with dollar-for-dollar matching up to 6%
  • Holistic benefits package focused on well-being
  • Flexible locations and schedules
  • Competitive pay often aligned to location and experience
  • Health coverage for employee and family
  • Generous vacation days
  • Donation matching up to $2000
  • Volunteer time off (up to 40 hours/year)
  • Parental leave minimum 16 weeks
Last updated Oct 8, 2026. Interested? Apply on the company site.
Elastic logo

About Elastic

The open source platform that powers search, observability, security, and more ...

Power insights and outcomes with The Elastic Search AI Platform. See into your data and find answers that matter with enterprise solutions designed to help you accelerate time to insight.

searchobservabilitysecurityAIenterprise solutionsvector database
Company profileWebsite

More open roles at Elastic