Skip to main content

Metrics

The Metrics tab shows how much an agent is used and how well it is doing over a period you choose. Where the Sessions tab shows one conversation at a time, Metrics shows the trend: whether volume is growing, whether scores are slipping, and which tools are failing. It is the first tab on the agent page.

Before You Start​

  • Role: the Admin or Organisation Admin role.
  • A published agent that has had at least one session. Until then, every chart shows its empty state.

Choosing a Period​

  1. Open Agents and select the agent. The Metrics tab opens first.
  2. Choose 7d, 30d or 90d in the top right. The tab opens on 30d.

The caption beside the switch confirms the period, for example "Last 30 days · compared with previous 30 days". Every change shown on the tab compares the chosen period with the period of the same length just before it. While new figures load, the caption reads Refreshing… and the previous figures stay on screen.

Each day on the charts is a calendar day in UTC, so a day's figures may not line up exactly with your local midnight.

Metrics count every session, including playground tests. A burst of testing therefore shows up in sessions, messages and tokens. To see only real traffic, filter the Sessions tab to Live.

The Totals​

A strip of five totals runs across the top of the tab. Each one shows the change against the previous period.

TotalWhat it counts
SessionsCalls and runs in the period, with a small chart of the daily trend.
MessagesMessages exchanged across all sessions in the period.
Tool callsTimes the agent called one of its tools, with the share that succeeded, for example "96% success".
Avg scoreThe average score out of 10 across scored sessions. The change is shown in points rather than as a percentage.
TokensThe amount of text the AI model read and wrote, with a small chart of the daily trend.

Tokens are the units an AI model measures text in. Longer instructions and longer conversations both use more tokens. The change in tokens is shown without a good or bad colour, because higher usage can mean more customers as easily as a less efficient agent.

The Charts​

Sessions Over Time​

Sessions per day across the period, with the total, the busiest day ("peak 42 on 14 Aug") and the change against the previous period. Use it to see when the agent is busiest and whether a change you published affected volume.

When the agent has no sessions yet, the chart says so. For an agent with a playground, it offers Open playground so you can create the first session.

Token Usage​

Input tokens (what the model read, including instructions and the conversation so far) against output tokens (what it wrote) per day, with totals for each. Input usually far outweighs output, because the model rereads the instructions and the conversation on every turn. A sudden jump in input tokens with no change in sessions often follows a longer system prompt.

Score Distribution​

How many scored sessions fell at each score, with the average score as the headline and the number of scored sessions underneath. Bars are coloured green, amber or red, using the same bands as the Sessions tab.

Read the shape as well as the average. A healthy agent has most sessions bunched at the high end. A second cluster of low scores usually points to one kind of conversation the agent handles badly. Filter the Sessions tab and read those sessions to find out which.

If nothing has been scored yet, the chart reads Nothing scored yet. Scores are written when a session is summarised after it ends.

Tool Usage​

One row per tool, with the number of calls, the success rate, and the split between succeeded and failed calls. A success rate that is low enough to need attention is highlighted.

A failing tool is one of the most common reasons for a low score. Open a session where the tool failed and read its Input and Output in the transcript to see whether the agent sent the wrong details or the system it called returned an error. See Tools and Knowledge.

When the agent has not called any tools in the period, the chart reads No tool calls yet.

Reading Metrics Well​

  • Compare like with like. A 7-day period after a public holiday compares badly with the week before it. Switch to 30 or 90 days before drawing conclusions from a short dip.
  • Check scores against volume. An average built on a handful of sessions moves sharply. The Score distribution subtitle tells you how many sessions it is based on.
  • Separate testing from traffic. Heavy playground testing raises sessions and tokens and can pull the average score either way. Check the Live filter in Sessions before reporting figures.
  • Go from trend to transcript. Metrics tell you that something changed. The transcripts in Sessions tell you why.

Problems Loading Metrics​

If the figures cannot be loaded, the tab shows Unable to load agent metrics. Select Retry.