Monitoring Overview - Cloosphere Guide

Admin Panel › Monitoring · Evaluations

Cloosphere’s operational visibility splits into two axes.

Monitoring — how the system is being used (activity · usage · logs)
Evaluation — how good the AI responses are (feedback · quality)

These are separate menus with different purposes in the admin panel. Pick the axis you want from the tabs below.

Admin > Monitoring
Admin > Evaluations

Admin › Monitoring

The menu for system operations. The Audit Logs tab is the default landing.

Tab	Function
Audit Logs	View user activity records
KMS Audit	Audit log of encryption key management (KMS) operations
Guardrail Logs	View guardrail detection/blocking events
Conversation Logs	View token usage per individual conversation (request)
File Logs	File upload security monitoring (guardrail classification, blocked/flagged file tracking)
Usage	Token usage, cost, statistics dashboard
Dashboard	AI-based panel charts, sharing, HTML export

The Conversation Logs tab lists individual conversations (requests) row by row. See Conversation Logs for details.

The File Logs tab is always shown. Only files that have file guardrail results are recorded, so the list can be empty on instances that don’t use file guardrails.

The Audit Logs · KMS Audit · Dashboard tabs, and the Auto-evaluation · Tracing tabs under Evaluations, appear only when your license includes those features. If it doesn’t, they never show up in the menu. Check your license under Admin › Settings › License.

Admin › Evaluations

The menu for AI response quality.

Tab	Function
Arena	Blind side-by-side comparison of two models’ responses (default tab)
Leaderboard	Elo-based model ranking from Arena results
Feedback	View user response evaluation (win/loss) records
Auto-evaluation	LLM-based response quality auto-measurement results
Tracing	LLM request processing tracking

Evaluation has its own overview — see Evaluation for the measure→diagnose flow and each tab.

Use Cases

Scenario	How
Monthly AI Cost Analysis	Check token consumption and cost by period/organization/model in Usage to compile the monthly cost report
Security Incident Investigation	Filter suspicious activity in audit logs by time range/user to trace incident details
Response Quality Debugging	In tracing, review step-by-step processing for a specific message and generate an LLM analysis report
Guardrail Policy Improvement	Analyze detection patterns in guardrail logs to reduce false positives and strengthen security policy

Cost amounts only appear when the price per 1M tokens has been entered per model under Admin › Settings › Models › Model Pricing. Models with no price are excluded from the cost estimate (token counts are still aggregated).

Recommended Review Cadence

Cadence	Items
Daily	Anomalous usage patterns, guardrail blocking events
Weekly	Usage trends, auto-evaluation score trend review
Monthly	Cost analysis report, audit log summary, guardrail policy review
Quarterly	Overall utilization assessment, model cost-effectiveness analysis

​Use Cases

​Recommended Review Cadence

Use Cases

Recommended Review Cadence