> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gaussia.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Conversational

> Evaluate dialogue quality using Grice's Maxims with session-level aggregation and pluggable statistical modes

# Conversational Metric

The Conversational metric evaluates dialogue quality using **Grice's Maxims** — principles of cooperative conversation that define effective communication. It accumulates scores across all interactions in a session and emits one session-level result, with optional uncertainty quantification via Bayesian mode.

## Overview

The metric assesses seven dimensions:

| Dimension | Description | Scale |
| - | - | - |
| **Quality Maxim** | Truthfulness and evidence-based responses | 0-10 |
| **Quantity Maxim** | Appropriate amount of information | 0-10 |
| **Relation Maxim** | Relevance to the conversation | 0-10 |
| **Manner Maxim** | Clarity and organization | 0-10 |
| **Memory** | Ability to recall previous context | 0-10 |
| **Language** | Appropriateness of language style | 0-10 |
| **Sensibleness** | Overall coherence and logic | 0-10 |

Each dimension produces a session-level `ConversationalScore` with a `mean` and optional credible interval (`ci_low`, `ci_high`) in Bayesian mode. The `interactions` list preserves per-QA scores for debugging.

## Installation

```bash theme={null}
uv add gaussia
uv add langchain-openai  # Or your preferred LLM provider
```

## Basic Usage

```python Frequentist (default) theme={null}
from gaussia.metrics.conversational import Conversational
from langchain_openai import ChatOpenAI
from your_retriever import MyRetriever

judge_model = ChatOpenAI(model="gpt-4o-mini", temperature=0.0)

metrics = Conversational.run(MyRetriever, model=judge_model, verbose=True)

for metric in metrics:
    print(f"Session: {metric.session_id}  ({metric.n_interactions} interactions)")
    print(f"  Quality:      {metric.conversational_quality_maxim.mean:.1f}/10")
    print(f"  Memory:       {metric.conversational_memory.mean:.1f}/10")
    print(f"  Sensibleness: {metric.conversational_sensibleness.mean:.1f}/10")

    for interaction in metric.interactions:
        print(f"  [{interaction.qa_id}] quality={interaction.quality_maxim:.1f} memory={interaction.memory:.1f}")
```

```python Bayesian theme={null}
from gaussia.metrics.conversational import Conversational
from gaussia.statistical import BayesianMode
from langchain_openai import ChatOpenAI
from your_retriever import MyRetriever

judge_model = ChatOpenAI(model="gpt-4o-mini", temperature=0.0)

metrics = Conversational.run(
    MyRetriever,
    model=judge_model,
    statistical_mode=BayesianMode(mc_samples=5000, ci_level=0.95),
    verbose=True,
)

for metric in metrics:
    q = metric.conversational_quality_maxim
    print(f"Quality: {q.mean:.1f}  [{q.ci_low:.1f}, {q.ci_high:.1f}]")
```

### Required Parameters

| Parameter | Type | Description |
| - | - | - |
| `retriever` | `Type[Retriever]` | Data source class |
| `model` | `BaseChatModel` | LangChain-compatible judge model |

### Optional Parameters

| Parameter | Type | Default | Description |
| - | - | - | - |
| `statistical_mode` | `StatisticalMode` | `FrequentistMode()` | Statistical computation mode |
| `use_structured_output` | `bool` | `False` | Use LangChain structured output |
| `bos_json_clause` | `str` | ` ```json ` | JSON block start marker |
| `eos_json_clause` | `str` | ` ``` ` | JSON block end marker |
| `verbose` | `bool` | `False` | Enable verbose logging |

## Statistical Modes

> [tab] Frequentist
>
> Returns a weighted mean per dimension. `ci_low` and `ci_high` are `None`.
>
> ```python theme={null}
> metric.conversational_quality_maxim.mean   # 7.8
> metric.conversational_quality_maxim.ci_low  # None
> ```

> [tab] Bayesian
>
> Bootstraps the weighted mean across interactions to produce a credible interval — useful when you have few interactions and want to express uncertainty honestly.
>
> ```python theme={null}
> metric.conversational_quality_maxim.mean   # 7.8
> metric.conversational_quality_maxim.ci_low  # 6.2
> metric.conversational_quality_maxim.ci_high # 9.1
> ```

## Interaction Weights

Each `Batch` can carry an optional `weight` to control its contribution to the session aggregate:

```python theme={null}
Batch(qa_id="q1", query="...", assistant="...", ground_truth_assistant="...", weight=0.5),
Batch(qa_id="q2", query="...", assistant="...", ground_truth_assistant="...", weight=0.3),
Batch(qa_id="q3", query="...", assistant="...", ground_truth_assistant="...", weight=0.2),
```

| Case | Behavior |
| - | - |
| All weights provided, sum = 1.0 | Used as-is |
| All weights provided, sum ≠ 1.0 | Warning emitted, equal weights applied |
| Some weights provided | Remaining weight split equally among unweighted interactions |
| No weights provided | Equal weights (1/n each) |

## Output Schema

### ConversationalMetric

```python theme={null}
class ConversationalMetric(BaseMetric):
    session_id: str
    assistant_id: str
    n_interactions: int
    conversational_memory: ConversationalScore
    conversational_language: ConversationalScore
    conversational_quality_maxim: ConversationalScore
    conversational_quantity_maxim: ConversationalScore
    conversational_relation_maxim: ConversationalScore
    conversational_manner_maxim: ConversationalScore
    conversational_sensibleness: ConversationalScore
    interactions: list[ConversationalInteraction]
```

### ConversationalScore

```python theme={null}
class ConversationalScore(BaseModel):
    mean: float           # Session-level weighted mean
    ci_low: float | None  # Lower credible bound — Bayesian mode only
    ci_high: float | None # Upper credible bound — Bayesian mode only
```

### ConversationalInteraction

```python theme={null}
class ConversationalInteraction(BaseModel):
    qa_id: str
    memory: float
    language: float
    quality_maxim: float
    quantity_maxim: float
    relation_maxim: float
    manner_maxim: float
    sensibleness: float
```

## Grice's Maxims Explained

### Quality Maxim

**Be truthful**: Don't say what you believe to be false or lack evidence for.

```
High (8-10): "The capital of France is Paris." (Verifiable fact)
Low  (0-4):  "France doesn't have a capital." (False)
```

### Quantity Maxim

**Be informative**: Provide enough information, but not more than required.

```
High (8-10): "Paris is the capital of France."
Low  (0-4):  "Paris." (Too brief) or a 3-paragraph essay (Too much)
```

### Relation Maxim

**Be relevant**: Make your contribution relevant to the conversation.

```
High (8-10): Q: "What's your return policy?" A: "Returns accepted within 30 days."
Low  (0-4):  Q: "What's your return policy?" A: "Our company was founded in 1998."
```

### Manner Maxim

**Be clear**: Avoid obscurity and ambiguity.

```
High (8-10): "You can return items within 30 days at any store location."
Low  (0-4):  "So basically, if you want to, you could possibly maybe return the thing..."
```

## Score Interpretation

| Score Range | Interpretation |
| - | - |
| 8-10 | Excellent — high-quality dialogue |
| 6-8 | Good — meets expectations with minor issues |
| 4-6 | Moderate — noticeable quality issues |
| 2-4 | Poor — significant problems |
| 0-2 | Very poor — fails basic criteria |

## Complete Example

```python theme={null}
import os
from gaussia.metrics.conversational import Conversational
from gaussia.statistical import BayesianMode
from gaussia.core.retriever import Retriever
from gaussia.schemas.common import Dataset, Batch
from langchain_openai import ChatOpenAI

class ConversationalRetriever(Retriever):
    def load_dataset(self) -> list[Dataset]:
        return [
            Dataset(
                session_id="conv-eval-001",
                assistant_id="support-bot",
                language="english",
                context="You are a helpful, professional customer service assistant.",
                conversation=[
                    Batch(
                        qa_id="q1",
                        query="Hi, I need help with my order.",
                        assistant="Hello! I'd be happy to help. Could you share your order number?",
                        ground_truth_assistant="Greet and ask for order number.",
                        observation="Opening interaction - should be professional and helpful",
                    ),
                    Batch(
                        qa_id="q2",
                        query="It's ORDER-12345. I haven't received it yet.",
                        assistant="Thank you! ORDER-12345 was shipped Monday, expected Friday.",
                        ground_truth_assistant="Find order, provide shipping status and ETA.",
                    ),
                    Batch(
                        qa_id="q3",
                        query="Can you change the delivery address?",
                        assistant="For security, please confirm your email address first.",
                        ground_truth_assistant="Offer to help, verify identity first.",
                    ),
                ]
            )
        ]

judge = ChatOpenAI(model="gpt-4o-mini", api_key=os.getenv("OPENAI_API_KEY"), temperature=0.0)

metrics = Conversational.run(
    ConversationalRetriever,
    model=judge,
    statistical_mode=BayesianMode(mc_samples=5000, ci_level=0.95),
    use_structured_output=True,
    verbose=True,
)

for metric in metrics:
    print(f"Session: {metric.session_id}  ({metric.n_interactions} interactions)")
    print()

    dimensions = [
        ("Quality",      metric.conversational_quality_maxim),
        ("Quantity",     metric.conversational_quantity_maxim),
        ("Relation",     metric.conversational_relation_maxim),
        ("Manner",       metric.conversational_manner_maxim),
        ("Memory",       metric.conversational_memory),
        ("Language",     metric.conversational_language),
        ("Sensibleness", metric.conversational_sensibleness),
    ]

    for name, score in dimensions:
        ci = f"  [{score.ci_low:.1f}, {score.ci_high:.1f}]" if score.ci_low is not None else ""
        print(f"  {name:<14} {score.mean:.1f}/10{ci}")
```

## Best Practices

> [accordion] Use Bayesian Mode for Small Sessions
>
> If a session has fewer than 5-10 interactions, the frequentist mean can be misleading. Bayesian mode shows a CI, making it clear when more data is needed.

> [accordion] Include Observations
>
> Add `observation` to guide the judge on what to evaluate:
>
> ```python theme={null}
> Batch(
>     qa_id="q2",
>     observation="Follow-up — assistant should remember the order number from q1",
> )
> ```

> [accordion] Test Multi-Turn Conversations
>
> Include sequences that test memory:
>
> ```python theme={null}
> Batch(qa_id="q1", query="My name is John..."),
> Batch(qa_id="q2", query="What's my name?"),  # Should remember
> ```

## Next Steps

> [card] [Statistical Modes](https://docs.gaussia.ai/concepts/statistical-modes)
>
> Frequentist vs Bayesian — when each matters

> [card] [Context Metric](https://docs.gaussia.ai/metrics/context)
>
> Evaluate context alignment

> [card] [Humanity Metric](https://docs.gaussia.ai/metrics/humanity)
>
> Emotional analysis of responses


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.