← All roles

Lead Data Engineer - Founding Member (Contract) [HR208] (UK)

Smart Working· Engineering
Apply on company site
Hybrid (UK)Part-Time | HybridPosted 2mo ago

About the role

Lead Data Engineer to build and lead the data infrastructure powering an AI assistant platform. Architect and scale data systems, real-time pipelines, vector databases, and ML data workflows. Founding data hire shaping culture, standards, and early data function growth.

Responsibilities

  • Architect and build scalable data pipelines and infrastructure to support AI and product systems.
  • Design and maintain data ingestion, transformation and storage architectures for operational and AI workloads.
  • Develop and manage batch and real-time data pipelines.
  • Build and optimise systems for vector search, retrieval and machine learning data pipelines.
  • Ensure data reliability, security and governance across the platform.
  • Collaborate with AI and backend engineering teams to support training, inference and product features.
  • Implement monitoring, observability and data quality frameworks.
  • Optimise the performance of large-scale datasets and query systems.
  • Contribute to technical architecture decisions and long-term data strategy.
  • Act as the founding data hire, defining culture, standards and the hiring bar for the data function as it scales.
  • Partner directly with founders and product leadership to translate data capabilities into product decisions.

Qualifications

  • 7+ years of professional experience, with the majority of that experience in dedicated data engineering roles.
  • Strong experience designing and building data pipelines and distributed data systems.
  • Experience working with relational databases, with PostgreSQL preferred, although MySQL or similar is acceptable.
  • Experience working with NoSQL databases.
  • Strong programming experience in Python.
  • Demonstrated ability to make and justify architectural decisions, rather than only implementing them.
  • Experience building scalable backend systems.
  • Experience designing data models and storage architectures.
  • Strong understanding of data processing performance and optimisation.
  • Experience with some of the following data frameworks and infrastructure technologies is highly desirable: Apache Spark, Apache Airflow, Kafka, and Elasticsearch or OpenSearch.
  • Experience with relevant database technologies is highly desirable, including PostgreSQL, MongoDB, and vector databases such as Qdrant, Milvus or pgvector.
  • Experience with Python data-processing libraries such as Pandas or Polars is highly desirable.

Skills

Required:Data pipeline design and implementationRelational databases (PostgreSQL preferred)NoSQL databasesPython programming
Last updated Oct 21, 2026. Interested? Apply on the company site.

About Smart Working

Hire top 1% vetted Software Developers | Smart Working

Expertise in 120+ skills and technologies, Smart Working helps you hire skilled developers. We excel in vetting and matching. Save up to £42,000 per hire.

SoftwareDevelopmentOutsourcingVettingRemote
Company profileWebsite