Encyclopedia Evalica / Deployment / Token
Token
/'toh.kuhn/A single unit of text that language models process. Token counts determine both cost and latency because models bill and compute proportionally to the input and output of tokens. (noun)
“We reduced token usage by shortening the retrieved context.”
Customer example
As Notion's prompts grew from thousands to hundreds of thousands of tokens, they needed to search massive traces for specific tool calls and patterns. Read more
Related Deployment terms
From the docs
Get started with Evals
Braintrust is the AI observability and eval platform for production AI. By connecting evals and observability in one workflow, teams at Notion, Stripe, Zapier, Vercel, and Ramp ship quality AI products at scale.
Start building