We are used to trusting our feelings: it seems that the processes are working and the product is of high quality. But feelings cannot be scaled. In this report, I will show how we moved from intuitive decisions to a system of metrics that measures the quality of products and processes in real time. How teams, with a “dashboard,” manage the development of their products in terms of quality. And most importantly, how technical metrics become understandable to the business, help to talk about risks in one language, and make decisions on a large scale.

I will talk about the validation and monitoring of AI agents using the example of a mobile application that interacts with a multi-agent system via OpenAPI. I will demonstrate practical approaches to testing agent logic, methods for collecting performance metrics, and setting up an observability system. I will share my experience in tracking agent behavior in real time, detecting anomalies, and ensuring the reliability of a multi-agent architecture in production.

The session covers architectural strategies that keep Elasticsearch stable under heavy load: proper index and shard design, ILM policies, persistent queues in Logstash, metrics downsampling, and self-monitoring of the observability stack. Insights are drawn from operating platforms that handle terabytes of logs and millions of events per day.