Evals for High-Stakes AI Agents — Doneyli De Jesus & Annabell Schäfer, Langfuse
A hands-on workshop on evaluating AI applications where mistakes are expensive.
Proof1 independent outlet
Key points
- Annabell Schäfer and Doneyli De Jesus from Langfuse use a banking assistant to walk through the whole AI engineering loop.
- They cover tracing with PII masking, building datasets, turning domain-expert comments into failure modes, LLM-as-judge and code evaluators, and CI that blocks a prompt change when scores drop.
- 12:47 Demo: a banking assistant with tracing and PII masking
- 45:58 Working with your coding agent
Sources (1)
- [1]Evals for High-Stakes AI Agents — Doneyli De Jesus & Annabell Schäfer, LangfuseAI Engineer (YouTube) · Oct 18, 08:00 AM
A hands-on workshop on evaluating AI applications where mistakes are expensive.
Annabell Schäfer and Doneyli De Jesus from Langfuse use a banking assistant to walk through the whole AI engineering loop.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 17, 2026How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
- Oct 17, 2026justxor/deepseek-local-guide-ru: 🐋 Полный гайд на русском: как установить и запустить DeepSeek локально (V4, R1) на ПК, Mac и сервере — Ollama, llama.cpp, LM Studio, vLLM, O
- Oct 16, 2026AI Agents Cut 3-Day Portfolio Builds to Minutes — Jack McDonald, Apollo & Michael Manapat, Rowspace
- Oct 16, 2026Why One AI Agent Isn’t Enough (And How to Run 1,000) - CJ Avilla & Harrison Stall, Anthropic
- Oct 15, 2026open source framework that makes your AI agents generate UI elements
- Oct 15, 2026NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning
