Humanity

Overview

The Humanity metric analyzes the emotional profile of assistant responses using the NRC Emotion Lexicon. It computes emotion distributions, emotional entropy, and Spearman correlation against ground truth responses.

Dimensions

For each interaction, the metric computes distribution scores across eight emotions:

Emotion	Description
Anger	Frustration or hostility expressions
Anticipation	Forward-looking or expectant language
Disgust	Aversion or repulsion indicators
Fear	Anxiety or threat-related language
Joy	Positive or happy expressions
Sadness	Sorrow or melancholy indicators
Surprise	Unexpected or astonishing content
Trust	Reliability and confidence markers

Additionally, it computes:

Emotional entropy: How diverse the emotional range is (higher = more diverse)
Ground truth Spearman correlation: How closely the emotional profile matches the expected response

Usage

from gaussia.metrics.humanity import Humanity

results = Humanity.run(MyRetriever)

for r in results:
    print(f"QA: {r.qa_id}")
    print(f"Emotional entropy: {r.humanity_assistant_emotional_entropy:.3f}")
    print(f"Spearman correlation: {r.humanity_ground_truth_spearman:.3f}")
    print(f"Joy: {r.humanity_assistant_joy:.3f}")

Parameters

Parameter	Type	Default	Description
`retriever`	`type[Retriever]`	required	Retriever class

The Humanity metric does not require an LLM — it uses lexicon-based analysis.

Output schema

HumanityMetric

One result per interaction (not per session):

Field	Type	Description
`session_id`	`str`	Session identifier
`qa_id`	`str`	Interaction identifier
`assistant_id`	`str`	Assistant identifier
`humanity_assistant_emotional_entropy`	`float`	Emotional diversity (Shannon entropy)
`humanity_ground_truth_spearman`	`float`	Correlation with ground truth emotion profile
`humanity_assistant_anger`	`float`	Anger score (0–1)
`humanity_assistant_anticipation`	`float`	Anticipation score (0–1)
`humanity_assistant_disgust`	`float`	Disgust score (0–1)
`humanity_assistant_fear`	`float`	Fear score (0–1)
`humanity_assistant_joy`	`float`	Joy score (0–1)
`humanity_assistant_sadness`	`float`	Sadness score (0–1)
`humanity_assistant_surprise`	`float`	Surprise score (0–1)
`humanity_assistant_trust`	`float`	Trust score (0–1)

Requires the humanity extra: pip install "gaussia[humanity]".

Getting started

Core concepts

Metrics

Integrations

Advanced

Overview

Dimensions

Usage

Parameters

Output schema

HumanityMetric

​Overview

​Dimensions

​Usage

​Parameters

​Output schema

​HumanityMetric

Overview

Dimensions

Usage

Parameters

Output schema

HumanityMetric