Key Skills

Job Description

Senior AI Engineer Utrecht, Netherlands | Valencia, Spain About Engaige Engaige Technologies B.V. is a Dutch company building an AI customer-service platform for e-commerce. Millions of customer cases flow through our infrastructure every year, across more than 30 countries. Our agents answer customer questions and take actions on the merchant's behalf, such as looking up orders, creating returns, paybacks, changing subscriptions, etc. through 50+ integrations such as Shopify, Zendesk, Gorgias, Intercom, Klaviyo, PostNL and Sendcloud. About the team The engineering team consists of 4 engineers. You will work closely with the Head of Product and the CEO, and engineers own their work from initial thought through production. About the role As a Senior AI Engineer, you will be responsible for the quality and reliability of the platform and agents that handle these customer cases. This includes how we evaluate them, how we retrieve and classify information, how agents follow merchant policies, and how changes reach production safely. A wrong promise or a wrong action toward a customer is a refund and a merchant at risk, so measurement, precision and safe rollout are part of the job. The core of this role is classical production ML depth: you have trained, evaluated and shipped models before LLMs became widely available, and you know how models fail once they meet real data. Responsibilities Own the evaluation methodology for our agents, including our benchmark that replays real production conversations with simulated tools, and LLM judges that check for unsupported promises, customer language and correct outcomes Design offline evaluations that predict online behavior, and close the gap between benchmark scores and production results Build live guardrails, monitoring and staged rollout for model, prompt and policy changes across multiple model providers Improve retrieval and classification, including RAG with vector search and reranking, and embedding-based contact-reason classification Develop the policy graphs that let agents follow merchant policies step by step instead of relying on a single large prompt Extend our AI manager, which investigates conversations, identifies knowledge and policy gaps, and runs weekly improvement rounds Partner with the Head of Product and the CEO to set technical standard for engineering and evaluation across the team You may be a good fit if you: Have trained and shipped machine learning models to production before 2022 Have a working understanding of offline and online evaluation, A/B testing, data leakage, training/serving skew, and working with noisy or delayed labels Have hands-on experience with ranking, recommendation or classification systems Have built monitoring for model quality and drift, and rolled out model changes safely Are proficient in Python and comfortable owning backend production code Have high agency and low ego, prefer shipping over planning, and are comfortable working on problems that are not yet fully defined Strong candidates may also have: Experience with LLMs in production, such as tool calling, structured output, RAG or LLM-as-judge evaluation Experience with search, retrieval or agent systems Experience with Weaviate, Langfuse, OpenTelemetry or Google Cloud Platform Experience with E-commerce or building SaaS tools. Our stack Claude, OpenAI and open-weight models (Kimi, GLM, DeepSeek, Qwen), Weaviate, Python (FastAPI), Native Postgres, Cloud Tasks, Cloud Run and Terraform Location This role is ideally based full-time in our office in Utrecht. Hybrid from Utrecht or fully remote is possible. Valencia is also an option. We hire within Europe only. Compensation We offer a very generous salary and stock options. For the right candidate, this role has a possible growth path toward CTO. How to apply We encourage you to apply even if you do not meet every qualification. Not all strong candidates will meet every qualification as listed. Apply via [email protected] Please include your Linkedin, Github and shipped projects

Core Responsibilities

Own the evaluation, reliability, and safe deployment of AI agents, including benchmarks, live guardrails, monitoring, and staged rollouts. Improve retrieval, classification, and policy-following systems, and partner with product and company leadership to set engineering and evaluation standards.

Requirements

Candidates should have trained and shipped machine-learning models to production before 2022, with practical knowledge of evaluation, A/B testing, data leakage, training-serving skew, noisy labels, and model monitoring. They should be proficient in Python, comfortable owning backend production code, and experienced with ranking, recommendation, or classification systems; production LLM, retrieval, or agent experience is an advantage.

Benefits

• Stock Options