The Analytics Bottleneck
Data accessibility is a persistent challenge inside massive technology organizations. Historically, when a Spotify employee needed specific business metrics, they followed a frustratingly familiar pattern. They would search for a relevant dashboard, discover it did not exist, and subsequently ping a human data expert on Slack. The employee then had to wait hours or days until the analyst had free time to manually pull the numbers.
To break this bottleneck, the engineering team introduced Vedder, an internal artificial intelligence data assistant. Deployed to over 2,100 users, the assistant scales data insights without forcing every query through a human bottleneck. However, pointing a raw large language model at a corporate database is a recipe for hallucinations. Without strict guidance, an AI will confidently generate SQL queries that pull incorrect or misleading metrics.
Encoding the Domain Expert
To ensure the assistant outputs reliable data, Spotify engineered a dedicated "context layer". This architectural layer sits between the language model and the raw data, acting as a translator that captures the specific nuances of Spotify's business logic.
The context layer is divided into distinct "clusters," each owned and maintained by a designated domain expert. A cluster contains three vital components: datasets featuring full schema definitions and data profiling, highly vetted SQL query pairs, and vital business documentation. By forcing the language model to reference this verified context before answering a prompt, Spotify grounds the AI in reality.
The Necessity of Human Curation
The success of the system relies entirely on ruthless human curation. Spotify does not allow the model to ingest documentation blindly. Domain experts must manually review and approve the exact query examples that influence the assistant's behavior.
During the initial curation phase, the strict review process meant that only 12.5 percent of proposed query pairs were actually accepted into the active context layer. This high rejection rate highlights the absolute necessity of human judgment. By fiercely protecting the training examples, Spotify guarantees the accuracy and trustworthiness of the final output.
Furthermore, business data is not static. Upstream schemas change, and metric definitions evolve over time. The Vedder system continuously monitors the health of the individual clusters. When underlying data shifts, the system alerts the domain experts, ensuring the context layer accurately reflects the current reality of the database. This hybrid approach proves that scaling AI in the enterprise requires marrying raw compute power with heavily structured, human verified knowledge.