← All roles

Senior Software Engineer, Data Platform

Handshake· Engineering
Apply on company site
San Francisco, CA$166k - $207kFull-timePosted 3w ago

About the role

Role focused on building and operating Handshake's data platform and infrastructure to move data reliably across marketplace and AI products. Hands-on technical leadership shaping Airflow-based workflows, data ingestion, pipelines, and data warehouse/streaming infrastructure.

Responsibilities

  • Lead the technical direction and roadmap for our Airflow and Astronomer platform while owning its production operations: deployment, DAG packaging, scheduler and worker capacity, upgrades, observability, and incident response.
  • Build paved paths for authoring, testing, deploying, and operating workflows, including reusable operators, CI checks, local and staging environments, and clear runbooks.
  • Design and operate streaming and change data capture pipelines using tools such as Pub/Sub, Dataflow or Beam, and Datastream to serve analytics and product use cases.
  • Make data delivery resilient to retries, duplicates, schema changes, late events, backfills, and replay; define useful latency, freshness, and reliability targets.
  • Improve the cloud foundation behind data workloads with Kubernetes, Terraform, IAM, secrets, CI/CD, and cost-aware capacity management.
  • Lead cross-team design decisions on data contracts, interfaces, and operational ownership; mentor engineers and align application, analytics, ML, and cloud partners on scalable platform approaches.
  • Participate in on-call, troubleshoot production failures across systems, and turn incidents into durable improvements.
  • Build dependable data and orchestration foundations for AI workloads, including batch inference, evaluation datasets, and agent-facing data.

Qualifications

  • Strong software engineering skills in Python and experience building and operating production data or distributed systems.
  • Deep hands-on Airflow experience beyond writing DAGs: scheduling and execution behavior, deployment, scaling, upgrades, monitoring, and debugging failures.
  • Experience with event-driven or streaming data infrastructure, including a message broker or managed event bus and a stream processing system.
  • Sound understanding of CDC, delivery guarantees, idempotency, ordering, schema evolution, and recovery or replay in production pipelines.
  • Experience with cloud infrastructure and infrastructure as code; comfortable working with containers, Kubernetes, CI/CD, access controls, and production observability.
  • A track record of leading ambiguous infrastructure initiatives: setting technical direction, making pragmatic architecture tradeoffs, and driving adoption across teams.
  • Clear communication and a track record of partnering across teams while owning systems through production support.

Skills

Required:PythonAirflowStreaming data / CDCCloud / IaC

Benefits

  • Equity in a fast-growing company
  • 401(k) match, competitive compensation, financial coaching
  • Paid parental leave, fertility benefits, parental coaching
  • Medical, dental, & vision, mental health support, $500 wellness stipend
  • $2,000 learning stipend, ongoing development
  • Internet, commuting, & free lunch/gym in our SF office
  • Flexible PTO, 15 holidays + 2 flex days
  • Team outings & referral bonuses
Last updated Oct 22, 2026. Interested? Apply on the company site.
Handshake logo

About Handshake

Careers start here

The better career platform for Gen Z changing how, where, and why the next generation of talent builds their career.

educationcareer platformtalent acquisitionstudent engagement
Company profileWebsite

More open roles at Handshake