← All roles

Sr. Software Engineer - Ingestion Core team

Databricks
Apply on company site
San Francisco, CA$166k - $225kPosted 1mo ago

About the role

Role focused on building distributed platform systems to ingest high-volume, petabyte-scale data from diverse sources into Delta Lake, with emphasis on streaming, performance, cost, and observability.

Responsibilities

  • Build distributed infrastructure to ingest data from diverse sources and support streaming ingestion, incremental processing, and replication.
  • Reduce end-to-end latency, increase throughput, and reduce costs from the time data appears in source systems to when it is available in Delta Lake.
  • Design and optimize streaming and distributed workloads for throughput, cost, latency, reliability, and scale.
  • Optimize streaming workloads by exploring and applying ML techniques.
  • Build monitoring and observability capabilities (customer-facing and internal) that provide visibility into ingestion workflows and the systems running them.
  • Collaborate with partner teams to enable use cases like RAG and AI agents.
  • Build Agentic SDKs with strong evaluations.

Qualifications

  • 5+ years of experience writing production code in one of: Java, Scala, Go, C++, or Python.
  • Experience architecting, developing, and deploying large-scale distributed and asynchronous systems.
  • Experience with distributed systems, streaming, Spark, databases, data processing, or CDC.
  • Experience building or operating systems where scale, throughput, latency, reliability, and cost are important considerations.
  • Comfortable using AI tools and defining effective evals to shorten the development loop.

Skills

Required:Production codeDistributed systemsStreamingPerformance optimization
Nice to have:AI tooling and evalsCollaboration

Benefits

  • For specific details on the benefits offered in your region click here.
Last updated Sep 17, 2026. Interested? Apply on the company site.

About Databricks

Leading Data and AI Platform for Enterprises

Databricks offers a unified platform for data, analytics and AI. Build better AI with a data-centric approach. Simplify ETL, data warehousing, governance and AI on the Data Intelligence Platform.

AIanalyticsbig datadatadata warehousingenterprise-infrastructuregovernanceliquid-ai-recruiting-target
Company profileWebsite

More open roles at Databricks