> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gaussia.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Quickstart
> Get started with Gaussia in minutes
# Quickstart
This guide will help you get started with Gaussia and run your first AI evaluation.
## Prerequisites
* Python 3.11 or higher
* [uv](https://docs.astral.sh/uv/) (recommended) or pip
## Installation
```bash uv (Recommended) theme={null}
# Install core package
uv add gaussia
# Install with specific metric dependencies
uv add "gaussia[toxicity]"
uv add langchain-openai
```
```bash pip theme={null}
# Install core package
pip install gaussia
# Install with all dependencies
pip install "gaussia[all]"
```
## Step 1: Create a Retriever
The first step is to create a retriever that loads your conversation data. A retriever is a class that inherits from `Retriever` and implements the `load_dataset()` method.
```python theme={null}
from gaussia.core.retriever import Retriever
from gaussia.schemas.common import Dataset, Batch
class MyRetriever(Retriever):
"""Custom retriever to load your AI conversation data."""
def load_dataset(self) -> list[Dataset]:
return [
Dataset(
session_id="evaluation-session-1",
assistant_id="my-assistant-v1",
language="english",
context="You are a helpful customer service assistant.",
conversation=[
Batch(
qa_id="q1",
query="What are your return policies?",
assistant="Our return policy allows returns within 30 days...",
ground_truth_assistant="Returns are accepted within 30 days with receipt.",
),
Batch(
qa_id="q2",
query="How can I track my order?",
assistant="You can track your order by logging into your account...",
ground_truth_assistant="Log into your account and visit Order History.",
),
]
)
]
```
## Step 2: Run a Metric
Once you have a retriever, you can run any metric. Here's an example using the Context metric:
```python theme={null}
from gaussia.metrics.context import Context
from langchain_openai import ChatOpenAI
# Initialize a judge model (any LangChain-compatible model)
judge_model = ChatOpenAI(model="gpt-4o-mini", temperature=0.0)
# Run the Context metric
metrics = Context.run(
MyRetriever,
model=judge_model,
use_structured_output=True,
verbose=True,
)
# Analyze results
for metric in metrics:
print(f"Session: {metric.session_id} ({metric.n_interactions} interactions)")
print(f" Context awareness: {metric.context_awareness:.2f}")
for interaction in metric.interactions:
status = "✅" if interaction.context_awareness >= 0.8 else "❌"
print(f" {status} [{interaction.qa_id}] {interaction.context_awareness:.2f}")
```
## Step 3: Analyze Results
Each metric returns a list of results. The structure depends on the metric type:
```python theme={null}
for metric in metrics:
print(f"Session: {metric.session_id}")
print(f"Score: {metric.context_awareness}") # 0-1 scale
for interaction in metric.interactions:
print(f" [{interaction.qa_id}] {interaction.context_awareness:.2f}")
```
```python theme={null}
for metric in metrics:
print(f"Session: {metric.session_id}")
print(f"Cluster Profiling: {metric.cluster_profiling}")
if metric.group_profiling:
gp = metric.group_profiling.frequentist
print(f"DIDT Score: {gp.DIDT}")
```
```python theme={null}
for metric in metrics:
print(f"Session: {metric.session_id}")
print(f"Quality Maxim: {metric.conversational_quality_maxim.mean:.1f}/10")
print(f"Sensibleness: {metric.conversational_sensibleness.mean:.1f}/10")
```
## Complete Example
Here's a complete example that evaluates an AI assistant using multiple metrics:
```python theme={null}
import os
from gaussia.metrics.context import Context
from gaussia.metrics.conversational import Conversational
from gaussia.core.retriever import Retriever
from gaussia.schemas.common import Dataset, Batch
from langchain_openai import ChatOpenAI
# 1. Define your retriever
class CustomerServiceRetriever(Retriever):
def load_dataset(self) -> list[Dataset]:
return [
Dataset(
session_id="cs-eval-001",
assistant_id="customer-service-bot",
language="english",
context="You are a helpful customer service assistant for an e-commerce store.",
conversation=[
Batch(
qa_id="q1",
query="I want to return a product I bought last week.",
assistant="I'd be happy to help with your return! You can return items within 30 days of purchase. Please visit our Returns Portal with your order number ready.",
ground_truth_assistant="Items can be returned within 30 days. Use the Returns Portal.",
),
Batch(
qa_id="q2",
query="What's your phone number for support?",
assistant="You can reach our support team at 1-800-EXAMPLE. We're available Monday through Friday, 9 AM to 5 PM EST.",
ground_truth_assistant="Support: 1-800-EXAMPLE, Mon-Fri 9-5 EST.",
),
]
)
]
# 2. Initialize the judge model
judge = ChatOpenAI(model="gpt-4o-mini", api_key=os.getenv("OPENAI_API_KEY"), temperature=0.0)
# 3. Run Context metric
print("=== Context Evaluation ===")
context_results = Context.run(
CustomerServiceRetriever,
model=judge,
use_structured_output=True,
verbose=True,
)
for m in context_results:
print(f"Context Awareness: {m.context_awareness:.2f}")
# 4. Run Conversational metric
print("\n=== Conversational Evaluation ===")
conv_results = Conversational.run(
CustomerServiceRetriever,
model=judge,
use_structured_output=True,
verbose=True,
)
for m in conv_results:
print(f"Quality: {m.conversational_quality_maxim.mean:.1f}/10")
print(f"Sensibleness: {m.conversational_sensibleness.mean:.1f}/10")
```
## What's Next?
Learn about all available metrics
Generate synthetic test datasets
Understand the architecture
Frequentist vs Bayesian approaches