Encyclopedia Evalica / Deployment / Brainstore

Brainstore

/'brayn.stawr/Braintrust's purpose-built database for AI observability data, combining object storage, columnar analytics, full-text search, vector indexing, and mutable row semantics. It's designed to support both deep debugging and high-performance analytics at scale. (noun)

Why it matters

AI observability data has unusual requirements that general-purpose databases struggle with. Traces are deeply nested, variably structured, and need to be both searchable (find me every trace where the model hallucinated) and analytically queryable (show me the p95 latency by model over the last week). They also need to be mutable, because scores, feedback, and annotations arrive after the trace is initially written. A traditional data warehouse handles analytics but not full-text search or real-time updates. A document database handles flexible schemas but not columnar aggregations at scale. Brainstore combines these capabilities in a single engine purpose-built for AI traces, so you can go from a dashboard-level view of score trends to a full-text search for a specific failure pattern to a deep dive into a single trace without switching tools or waiting for ETL pipelines. The result is that debugging and analysis happen at the speed of thought rather than the speed of data infrastructure.

We query Brainstore to slice eval results by model, customer, and failure mode.

Related Deployment terms

From the docs

Get started with Evals

Braintrust is the AI observability and eval platform for production AI. By connecting evals and observability in one workflow, teams at Notion, Stripe, Zapier, Vercel, and Ramp ship quality AI products at scale.

Start building