Skip to content

Contains Match

Check if output contains the expected text as a substring
Heuristic Single Turn Binary

At a Glance

🎯
Score Range
0.0 or 1.0
Binary pass/fail
⚡
Default Threshold
0.5
Pass/fail cutoff
📋
Required Inputs
actual_output expected_output
Substring to find

What It Measures

Contains Match checks whether the expected output appears as a substring within the actual output. This is the simplest form of text matching—does the response include the required content anywhere?

Score Interpretation
1.0 Expected text found in output
0.0 Expected text not found
✅ Use When
  • Checking for required keywords/phrases
  • Validating specific content inclusion
  • Simple pass/fail tests
  • Fast sanity checks
❌ Don't Use When
  • Exact match required (use Exact String Match)
  • Similarity scoring needed (use BLEU/Levenshtein)
  • Case variations matter
  • Semantic matching needed

See Also: Exact String Match

Contains Match checks if expected is a substring of actual. Exact String Match checks if they are identical.

How It Works

Simple substring check after stripping whitespace.

Step-by-Step Process

flowchart TD
    subgraph INPUT["📥 Inputs"]
        A[Actual Output]
        B[Expected Output]
    end

    subgraph PROCESS["🔍 Processing"]
        C[Strip whitespace]
        D["Check: expected in actual?"]
    end

    subgraph OUTPUT["📊 Result"]
        E["1.0 = Found"]
        F["0.0 = Not Found"]
    end

    A & B --> C
    C --> D
    D -->|Yes| E
    D -->|No| F

    style INPUT stroke:#f59e0b,stroke-width:2px
    style PROCESS stroke:#3b82f6,stroke-width:2px
    style OUTPUT stroke:#10b981,stroke-width:2px
score = 1.0 if expected.strip() in actual.strip() else 0.0

Configuration

Parameter Type Default Description
(none) - - No configuration options

Simple by Design

Contains Match is intentionally simple with no configuration. For case-insensitive matching or fuzzy matching, use other metrics.

Code Examples

from axion.metrics import ContainsMatch
from axion.dataset import DatasetItem

metric = ContainsMatch()

item = DatasetItem(
    actual_output="The capital of France is Paris, a beautiful city.",
    expected_output="Paris",
)

result = await metric.execute(item)
print(result.score)  # 1.0 - "Paris" is in the output

from axion.metrics import ContainsMatch

metric = ContainsMatch()

item = DatasetItem(
    actual_output="The capital of France is a beautiful city.",
    expected_output="Paris",
)

result = await metric.execute(item)
print(result.score)  # 0.0 - "Paris" not found

from axion.metrics import ContainsMatch
from axion.runners import MetricRunner

metric = ContainsMatch()
runner = MetricRunner(metrics=[metric])
results = await runner.run(dataset)

passed = sum(1 for r in results if r.score == 1.0)
print(f"Passed: {passed}/{len(results)}")

Example Scenarios

✅ Scenario 1: Substring Found (Score: 1.0)

Match Found

Expected: "42"

Actual: "The answer to life, the universe, and everything is 42."

Result: 1.0

The substring "42" exists in the output.

❌ Scenario 2: Substring Not Found (Score: 0.0)

No Match

Expected: "Python"

Actual: "JavaScript is a popular programming language."

Result: 0.0

"Python" does not appear in the output.

⚠️ Scenario 3: Case Sensitivity

Case Matters

Expected: "PARIS"

Actual: "The capital is Paris."

Result: 0.0

"PARIS" (uppercase) does not match "Paris" (title case).

Why It Matters

⚡ Instant Results

O(n) substring search. No external dependencies or LLM calls.

✅ Sanity Checks

Quickly verify required content is present in responses.

🔍 Keyword Validation

Ensure specific terms, codes, or phrases appear in output.

Quick Reference

TL;DR

Contains Match = Does the output contain the expected text?

  • Use it when: Checking for required keywords or phrases
  • Score interpretation: 1.0 = found, 0.0 = not found
  • Key behavior: Case-sensitive substring match