Staff AI Systems Engineer
About the role
Hiring a Sr. AI Systems Engineer to support product Night Shift, an AI research assistant for law enforcement investigations, within the Machine Learning team, focusing on system architecture for agentic AI and evaluation frameworks.
Responsibilities
- Support the development and deployment of Night Shift, an AI research assistant for law enforcement investigations.
- Work closely with engineering teams (Backend, Frontend, Design) in a fast-paced environment.
- Design and own the AI evaluation framework to measure performance metrics such as accuracy, speed, safety, and robustness.
- Contribute to system architecture for agentic AI including LLM API integration, agent design, retrieval, memory, grounding/attribution, and multi-agent orchestration.
- Build and maintain ML platform components including backend Python/JavaScript, web services, cloud infrastructure (AWS, Terraform), database management, observability tools, and possibly compute orchestration (Kubernetes).
- Develop offline and online evaluation harnesses for LLMs, covering retrieval, recommendation, safety, and efficiency metrics.
- Support scalable evaluation and observability infrastructure, including dashboards, alerting, regression tests, and data pipelines.
Qualifications
- 6+ years experience in developing and deploying ML systems.
- Hands-on experience with LLMs and agentic AI frameworks (LangChain, LangGraph, vLLM, OpenAI, Anthropic, etc.).
- Deep understanding of agent design, tool use, retrieval, grounding, context management, and RAG search.
- Proficiency in backend development (Python, JavaScript); Typescript or Golang familiarity is a plus.
- Experience with cloud infrastructure (AWS, Terraform), databases (Postgres, Redis), observability tools (Prometheus, Grafana, OpenTelemetry).
- Experience with building scalable eval frameworks and metrics for search, safety, cost, and system performance.
Skills
Benefits
- Salary between $200,000 and $240,000.
- Stock options.
- Remote work (US-based).
- Be an early technical contributor to innovative agentic AI systems.
- Participate in a mission-driven company aiming to improve public safety.
About Flock Safety
The first public safety operating system that eliminates crime.
Flock Safety provides the first public safety operating system that empowers private communities and law enforcement to work together to eliminate crime. We are committed to protecting human privacy and mitigating bias in policing with the development of best-in-class technology rooted in ethical design, which unites civilians and public servants in pursuit of a safer, more equitable society. Our Safety-as-a-Service approach includes affordable devices powered by LTE and solar that can be installed anywhere. Our technology detects and captures objective details, decodes evidence in real-time and delivers investigative leads into the hands of those who matter. While safety is a serious business, we are a supportive team that is optimizing the remote experience to create strong and fun relationships even when we are physically apart. Our flock of hard-working employees thrive in a positive and inclusive environment, where a bias towards action is rewarded. Flock Safety is headquartered in Atlanta and operates nationwide. We have raised $150M in our Series E led by Tiger Global at a $3.5B valuation.
More open roles at Flock Safety
- Senior ML Engineer, Enrichment7mo agoRemote · Engineering / Machine Learning
- Operations Research Scientist7mo agoRemote · Engineering, Machine learning
- Senior Software Engineer, Video FullStack Experience11mo agoRemote · Engineering
- Staff Software Engineer - Rust11mo agoRemote · Engineering
- Senior Software Engineer, Backend Integrations11mo agoHybrid (Atlanta, GA) · Engineering