Senior Software Engineer - LLM Ops & Evals

Amsterdam6 days ago

Full-timeIn English

About the job

Datasnipper is looking for a Senior Software Engineer to own the LLM gateway and evaluation platform. You will build and operate production AI infrastructure, working closely with product and ML teams in Amsterdam.

What you will do

Own the LLM gateway for routing, failover, rate limits, retries, and cost attribution
Deploy, version, and deprecate models across clouds and regions using infrastructure as code
Build the shared evaluation platform with versioned datasets and experiment tracking
Own observability for AI traffic including latency, token usage, and errors
Participate in on-call rotation and incident management
Define integration contracts and partner with product and ML engineers

What you bring

5+ years in backend or platform engineering with strong production Python
Experience building LLM inference infrastructure with multiple providers
Experience with cloud infrastructure and infrastructure as code (Azure, GCP, Terraform)
Experience running a shared service in production with on-call and SLOs
Hands-on experience with observability tools like OpenTelemetry and Grafana
Experience with privacy and compliance work including PII handling and access control

Nice to have

Experience with LLM or agent evaluation
Experience with Temporal or another durable workflow engine
Self-hosted inference, capacity planning, or load testing
Synthetic data or document anonymisation pipelines
Document AI experience with VLMs, OCR, or structured extraction
Domain experience in audit, accounting, or fintech

Summary by Personeel.com. The full job description and the application form are on Datasnipper's website (jobs.ashbyhq.com); Apply takes you straight there.

More jobs at Datasnipper

All 26 jobs at Datasnipper