User Guide¶
This guide covers the main features of AXIS and how to use them effectively.
Modules¶
Evaluate¶
Upload evaluation data, run batch evaluations with the Axion engine, and explore results through interactive tree visualizations and multi-chart analytics.
Production¶
Executive overview combining Agent KPIs, AI quality monitoring, and human-in-the-loop signals in a single at-a-glance dashboard with sparkline trends.
Monitoring¶
Deep-dive production observability — time-series score trends, metric breakdowns, latency distributions, classification analysis, and anomaly alerts.
Annotation Studio¶
Human-in-the-loop quality assessment with 3 annotation formats, tag-based critiques, keyboard shortcuts, and CSV export.
CaliberHQ¶
LLM judge calibration with a 3-step workflow — annotate ground truth, configure the judge, and validate alignment with Cohen's Kappa, confusion matrices, EvidencePipeline-powered pattern discovery, and actionable learning insights.
Simulation¶
Synthetic persona-based agent testing with configurable personas, knowledge base upload, and conversation replay.
Memory¶
Decision memory dashboard with rule extraction, hard stops, batch analysis, decision quality metrics, and knowledge graph visualization.
Human Signals¶
Data-driven HITL dashboard showing signal trends, classification distributions, case-level drill-down, and dynamic KPI strips.
Agent Replay¶
Debug and review AI agent execution traces from Langfuse — step through observation trees, inspect inputs/outputs, and submit verdicts for continuous improvement.
Learn¶
Interactive learning modules and guided tutorials for mastering AXIS features, evaluation methodology, and AI quality best practices.
Settings¶
System configuration, database connections, theme customization, and agent registry — all managed from a single page.
Getting Data In¶
AXIS supports two data ingestion paths:
- CSV Upload (default) — Drag and drop files through the UI
- Database Auto-Load — Configure PostgreSQL connections in YAML
See Data Sources for setup details.