Instantly checking a response you just received mid-conversation
Chat response action bar › link icon (tooltip View Trace)
③ Search by ID
Investigating a specific trace from a forwarded ID
Evaluation › Tracing › Chat ID / Message ID search
① From Conversation Logs (Admin)
② Directly from Chat
③ Search by ID
The standard path where an admin picks a specific conversation from the operations screen and enters tracing.
1
Open conversation logs
Go to Monitoring › Conversation Logs.
2
Select a conversation
Click the conversation (request) row you want to investigate to open its detail.
Expanding a conversation shows a 'Trace' button on the right
Arrive at Evaluation › Tracing
Clicking the Trace button in the detail automatically opens the Evaluation › Tracing screen.
The path for tracing a response you just received right from the chat, on the spot.
1
Click 'View Trace' on the response
Press the link icon (tooltip View Trace) in the action bar below the response message you want to trace. Evaluation › Tracing opens in a new tab and lists trace cards for every message in that chat — pick the card by time and message content to find the response you just received. To open one specific message directly, use ① Conversation Logs › Trace (it looks up by message ID).
The link icon (tooltip 'View Trace') in the action bar below a response
The button shown depends on your permissions.
Permission
Button Shown
Admin or evaluation read/write permission
Tooltip View Trace → opens Evaluation › Tracing in a new tab
Other users
Tooltip Copy Message ID → copies two lines, the Chat ID and the Message ID, to the clipboard. Forward them to an admin to request an investigation
Find a trace by directly entering a Chat ID or Message ID. Mainly used when an admin investigates an ID that a regular user copied and forwarded.
1
Obtain the Chat ID
A chat URL has the form https://<domain>/c/{Chat ID}. In the address bar, the UUID after /c/ (the blue selection below) is that chat’s Chat ID — copy this value. (Or obtain the Message ID that a user forwarded.)
Chat URL address bar — the blue selection after /c/ is the Chat ID
Look up in Evaluation › Tracing
Paste it into the search box in Admin panel › Evaluation › Tracing to look it up.
The Chat ID in a chat URL is held by the channel owner/user. Admins should ask the user for the URL (or a Copy Message ID) and investigate after receiving it.
Destination · Evaluation › Tracing
No matter which of the three paths you enter through, the destination is a single screen: Evaluation › Tracing below.
Evaluation › Tracing arrival screen — search list (message cards)
Searching by Chat ID or Message ID on the Evaluation › Tracing screen produces a list of message cards like the screen above.
Each card is a summary of one user message’s trace, and clicking a card opens the detail view.
The search-criteria and card-item tables are collapsed below.
Search criteria — Chat ID vs. Message ID
Search Criteria
Scope
Where to Get It
Chat ID
All message traces in that chat
The part after /c/ in the chat URL
Message ID
The trace of one specific message
User selects Copy Message ID from the message options
Search retrieves all traces matching the entered ID with no time limit. (There are no separate filters such as date or status.) Results are paged 20 at a time, so use the page controls at the bottom when there are many.
Message card items
Card Item
Description
User message
Original input (cards show up to 2 lines)
Trace ID
Abbreviated identifier (e.g., 3fb13ae9…)
Time
Request time (MM/DD HH:MM:SS)
Latency
Sum of the top-level step durations (the unit switches automatically, e.g. 2.39s, 1m 22s)
Type badges
Types of that trace’s top-level steps — Agent · LLM · Embed, etc.
Run count
Number of top-level steps (e.g., 4 runs). Steps nested inside the agent are not counted; open the card to see every step in the detail view
Here you see what steps a message went through and what went into and came out of each step.
(Detailed notations such as badges and symbols are collapsed inside each tab — expand them when you encounter an unfamiliar marker.)
Trace detail — pick a step in the left Run tree and its inputs/outputs appear on the right
The screen is split into the left Run tree (lists processing steps in order and hierarchy, with total latency and total tokens at the top) and the right detail panel (inputs, outputs, and tokens of the selected step). The left panel header switches between three views: Tree · Timeline · Tokens.A single message typically goes like this — the agent (CH) repeats LM reasoning and tool calls to gather data, and finally final_answer (LM) writes the answer.
Left — Run Tree
Right — Detail Panel
Each line (Run) is shown as colored badge + name + ● status + latency. Knowing just the frequently seen badges is enough to read the tree. (For the full list including colors, expand the legend below.)
Badge
What It Does
CH
Agent run — the step wrapping the entire message processing (the name is the agent name)
LM
LLM call — reasoning or writing the final answer (final_answer)
TL
Tool execution — SQL queries, function calls (run_sql_read, etc.)
EM
Embedding — generating vectors for search
ACT
Action group — tool + sub-steps bundle (expand with the arrow)
● Status — green means OK, red means an error at that step.
Adjacent time — that step’s latency. Note that a parent step (CH, ACT) includes the time of its children, so the largest raw value is not necessarily the bottleneck. The bottleneck the product computes is shown with its latency highlighted in red.
The view toggle in the left panel header shows the same step list three ways.
View
What It Shows
Tree
Steps listed as a parent-child hierarchy (default)
Timeline
Each step’s duration laid out as a horizontal bar, with a time scale and a Slowest extraction step: <name> · time (n%) summary underneath
Tokens
A bar for the trace’s overall token mix (input, cache, output) plus prompt cache hit rate, effective tokens, and per-step token bars (clicking a bar switches the detail panel to that step)
Auxiliary tasks such as auto-generating the title, tags, and search query are recorded separately from the response in the background_tasks trace.
All Run types (color legend)
The detail tree distinguishes step types by colored badges. (On search-result cards, they are grouped under labels like Agent · LLM · Embed.)
Badge
Type
Color
Description
CH
Chain
Purple
Composite work — full message processing / agent run
LM
LLM
Blue
LLM call (reasoning, final answer)
TL
Tool
Green
Tool execution (SQL, functions, etc.)
RG
Retrieval
Orange
Knowledge Base document search
WB
Web Search
Cyan
Web search
GD
Guardrail
Red
Guardrail check
EM
Embedding
Yellow
Embedding (vector) generation
FL
Filter
Pink
Filter function execution
PP
Pipeline
Navy
Pipeline processing
TK
Task
Gray
Background task
IM
Image
Gray
Image generation (no dedicated color, gray)
ACT
Action
Purple
Tool + sub-step group (expandable, synthetic badge)
Some types may not appear depending on your environment and version — real traces mostly record CH, LM, TL, RG, GD, EM, and TK.
Status display (● color)
Status
Display
Success
Green dot
Error
Red dot
Running
Amber dot (blinking pulse animation)
Pending
Gray dot
All four states use the same dot shape and differ only in color.
The trace’s overall status is Error if any included Run is Error; if there’s no Error but a Running exists, it shows as Running.
Clicking a step in the tree shows that step’s details on the right.
Section
Content
Top summary
Status · latency · model used (steps with a response format set also show an ‘Answer Format’ badge)
Inputs
What went into the step (system prompt, normalized question, etc.)
Outputs
What the step produced (response, source bundle count sources_count · source_names, etc.)
Error
Shown only when the step failed, as a red area below Outputs — start here when diagnosing tool failures
Token Usage
Input · Output · Total tokens. On a prompt cache hit, a Prompt Cache line with M of N input tokens cached (x%) and a Cache write value are appended
The Tree · JSON · Text toggle switches inputs/outputs between tree, raw JSON, and plain text.
The Search... box in Outputs lets you find text within long output.
The user message stored in a trace is capped at 500 characters. A trailing ... in Inputs comes from that recording cap — it does not mean the content sent to the model was truncated. (System prompts can also be length-capped depending on the path.)
Input/output view modes · text search
View Mode
Description
Tree
Hierarchical tree (default)
JSON
Raw JSON
Text
Plain text
Text search works in the Outputs area. Entering a query highlights matches; Enter moves to the next match and Shift+Enter to the previous one, and the match count is shown next to the search box like 1/5.
Reading the agent’s response in two phases helps you find the cause quickly.
Phase
What
What to Look At
Phase 1
The agent (CH) calls tools (TL) via LM reasoning to gather data
Which tools were used, whether results were non-empty
Phase 2
final_answer (LM) writes the answer from the gathered data
Whether the gathered data is properly reflected in the answer
The left panel header also shows a bar that computes these two spans for you, so you don’t have to add up step times — Extraction (trace start → final answer start) with its share, First token (total) (extraction + time to the answer’s first token = what the user actually waited), and First token (answer) (first token after the final answer step starts). Traces with no final answer step (background tasks, etc.) don’t show it.
Check tool_descriptions (the names and descriptions of the tools attached to that run) in the Inputs of Phase 1’s first LM reasoning Run. To keep traces small, this entry is recorded only on the first LLM call, so later LM Runs may not have it.
Desired tool not in tool_descriptions → that feature (Knowledge Base, DB, etc.) isn’t connected to the agent
Tool present but not called → the model judged low relevance between the question and the tool. Make the tool/feature description more specific
active_capabilities (the set of features enabled at that moment) is visible in the Inputs of the Phase 2 final_answer Run.
Search ran but the answer is inaccurate?
Knowledge Base search is recorded as its own RG (Retrieval) Run (knowledge_search, or knowledge_fetch_chunks when chunks are re-fetched). Check the hit count, scores, and rerank status in that Run’s Outputs via total_results, top_scores, reranked, and sources.
No RG Run at all → search never ran. Check that a Knowledge Base is connected to the agent
total_results is 0 → zero search results. A KB document is missing, or search settings (Top K, Reranker, etc.) need review
Sources exist but the answer is off → check what was passed in the final_answer (LM) Run’s Inputs and adjust the answer prompt
sources_count and source_names in the final_answer Outputs are the number of source bundles placed in the answer prompt, so they can differ from the search hit count.
Tool execution failed?
Click the TL (Tool) Run marked with a red dot, read the message in the Error area below Outputs first, and then verify the parameters passed in Inputs.
Response too slow?
In the Run tree, the step whose latency is highlighted in red is the bottleneck the product computed. Switching to the Timeline view adds a Slowest extraction step: <name> · time (n%) summary underneath. The final answer step and any post-processing after it are excluded from the bottleneck calculation, since they happen after the answer is delivered.
LM slow → consider switching to a faster model
TL/EM slow → check tool/search settings or external services
GD (guardrail) slow → disable LLM judgment or switch to a faster model
A feature that analyzes trace data with an LLM to automatically identify the root cause of a problem.The top of the detail modal has Copy Trace and Analyze Trace buttons (plus View Report when a previous analysis exists). Copy Trace puts a text version on the clipboard — the trace summary (Trace ID, Chat ID, status, total latency, total tokens) followed by an indented step list (success/failure, step type, name, latency, tokens, model, and a preview of the start of the inputs and outputs). Use it when sharing an issue or attaching context to a support request.
1
Start analysis
Click the Analyze Trace button at the top of the trace detail modal.
Input
Description
Required
Analysis model
LLM model used for analysis
Required
Description
The observed problem (the field shows example guidance text)
Optional
The analysis model list shows base models only. Custom models derived from another model, preset models, and arena models cannot be selected.
2
Review analysis result
The LLM comprehensively analyzes trace data + agent settings + conversation history + KB/DB/guardrail settings + glossary settings + auto-evaluation results to generate a structured report.
Report Section
Content
Summary
2~3 sentence core summary of the analysis
Trace overview
ID, status, latency, tokens, Run count, error count
Root cause analysis
Primary cause + contributing factors
Phase 1 analysis
Whether tool selection was appropriate, available vs. actual tool calls
Phase 2 analysis
Appropriateness of the final answer relative to collected data
Prompt/setting issues
System prompt, model selection issues
KB/RAG issues
Search settings, document quality, filter issues
DB/SQL issues
NL-to-SQL conversion, schema issues
Guardrail issues
Excessive blocking, false positives
Glossary coverage issues
Terms used in the question that the glossary does not cover
Error analysis
Error Run detailed diagnosis
Improvement recommendations
Immediate actions, setting changes, data improvements, architecture suggestions (mid- to long-term)
Sections with no relevant findings are omitted, so not every section appears every time.
Entering a description enables analysis focused on that context. Example: “KB found documents but they weren’t reflected in the answer”The report is written in the same language as the description. Write the description in the language you want the report in — leaving it blank may produce an English report.
3
Save/share report
These buttons live inside the analysis report modal.
Feature
Description
Copy
Copy full text to clipboard
Download
Download as markdown file (.md)
When a previous analysis exists, the View Report button shows the trace analysis report directly without re-analysis.
Reaching the tracing screen requires being an admin or holding evaluation read permission or higher. Users without it cannot open even their own traces in the UI, and without a tracing license the button is not shown at all.
Admin › Settings › Data Retention — set the retention days for Traces to clean up automatically, or click Run Cleanup Now to delete immediately. (recommended)
Developer API — DELETE /api/v1/traces/cleanup?before_timestamp_ms=<timestamp>. Bulk-deletes traces before the given millisecond timestamp; requires monitoring write permission.
Trace deletion is irreversible. Download necessary analysis reports before deleting.
Yes — when message tracing is enabled (default: enabled), all AI requests are auto-recorded. No additional setup needed.
How long is trace data retained?
By default traces are never auto-deleted — they are kept indefinitely. Automatic cleanup is off by default, and the retention days for every data type default to 0 (keep forever). To clean up old traces, go to Admin › Settings › Data Retention, set the Traces retention to 1 day or more and turn automatic cleanup on, or click Run Cleanup Now to delete immediately.
Does tracing affect response speed?
Traces are recorded inline with request processing as each step starts and completes. Every step adds a short save operation, but it is negligible next to LLM call time, so perceived response speed is barely affected.
Do analysis reports also consume tokens?
Yes — trace analysis is a separate LLM call, tracked in usage as trace_analysis. Analysis only runs when manually triggered.
