Skip to content

Length Constraint

Verify response length meets character and sentence constraints
Heuristic Single Turn Binary

At a Glance

📏
Score Range
0.0 or 1.0
Binary pass/fail
⚡
Default Threshold
1.0
Pass/fail cutoff
📋
Required Inputs
actual_output
Text to evaluate

What It Measures

Length Constraint verifies that the response meets character and/or sentence count constraints. This is useful for enforcing output limits in chatbots, summaries, or any application with length requirements.

Score Interpretation
1.0 Passed—within all constraints
0.0 Failed—exceeded character limit or outside sentence range
Use When
  • Enforcing token/character limits for LLM outputs
  • Validating summary length requirements
  • Ensuring concise responses in chatbots
  • Meeting UI display constraints
Don't Use When
  • Quality matters more than length
  • Variable-length outputs are acceptable
  • Sentence structure is irregular (code, lists)
How It Works

Checks character count against max_chars and sentence count against sentence_range.

Step-by-Step Process

flowchart TD
    subgraph INPUT["Input"]
        A[Actual Output]
    end

    subgraph PROCESS["Processing"]
        B[Count characters]
        C[Split into sentences]
        D["Check: chars <= max_chars?"]
        E["Check: sentence_count in range?"]
    end

    subgraph OUTPUT["Result"]
        F["1.0 = All constraints met"]
        G["0.0 = Constraint violated"]
    end

    A --> B
    A --> C
    B --> D
    C --> E
    D & E -->|All pass| F
    D & E -->|Any fail| G

    style INPUT stroke:#f59e0b,stroke-width:2px
    style PROCESS stroke:#3b82f6,stroke-width:2px
    style OUTPUT stroke:#10b981,stroke-width:2px
# Character check
char_passed = len(text) <= max_chars

# Sentence check (splits on .!? followed by space or end)
sentences = [s for s in re.split(r'[.!?]+', text) if s.strip()]
sentence_count = len(sentences)

min_s, max_s = sentence_range
sentence_passed = (min_s is None or sentence_count >= min_s) and \
                  (max_s is None or sentence_count <= max_s)

score = 1.0 if (char_passed and sentence_passed) else 0.0

Configuration

Parameter Type Default Description
max_chars int 2800 Maximum allowed characters. Set to None to disable.
sentence_range tuple None (min, max) sentence bounds. Use None for open ends.
field_mapping dict None Remap actual_output to another field path.

Flexible Sentence Ranges

  • (3, 5) — Between 3 and 5 sentences (inclusive)
  • (None, 5) — At most 5 sentences
  • (3, None) — At least 3 sentences

Code Examples

from axion.metrics import LengthConstraint
from axion.dataset import DatasetItem

# Default: max 2800 characters, no sentence constraint
metric = LengthConstraint()

item = DatasetItem(
    actual_output="This is a short response.",
)

result = await metric.execute(item)
print(result.score)  # 1.0 - within limits
print(result.signals.char_count)  # 26

from axion.metrics import LengthConstraint

# Require 2-4 sentences, max 500 characters
metric = LengthConstraint(
    max_chars=500,
    sentence_range=(2, 4),
)

item = DatasetItem(
    actual_output="First sentence. Second sentence. Third sentence.",
)

result = await metric.execute(item)
print(result.score)  # 1.0 - 3 sentences, 48 chars
print(result.signals.sentence_count)  # 3

from axion.metrics import LengthConstraint

metric = LengthConstraint(
    max_chars=50,
    sentence_range=(None, 2),  # max 2 sentences
)

item = DatasetItem(
    actual_output="First. Second. Third. Fourth. Fifth sentence here.",
)

result = await metric.execute(item)
print(result.score)  # 0.0 - too many sentences
print(result.explanation)
# "FAILED. Sentence count 5 outside (Range: 0-2)."

from axion.metrics import LengthConstraint

# Evaluate a nested field instead of actual_output
metric = LengthConstraint(
    max_chars=1000,
    field_mapping={'actual_output': 'additional_output.summary'},
)

item = DatasetItem(
    additional_output={'summary': 'A brief summary of the document.'},
)

result = await metric.execute(item)
print(result.score)  # 1.0

from axion.metrics import LengthConstraint
from axion.runners import MetricRunner

metric = LengthConstraint(max_chars=1000, sentence_range=(1, 5))
runner = MetricRunner(metrics=[metric])
results = await runner.run(dataset)

passed = sum(1 for r in results if r.score == 1.0)
print(f"Passed length constraints: {passed}/{len(results)}")

Example Scenarios

Scenario 1: Within All Constraints (Score: 1.0)

Passed

Config: max_chars=100, sentence_range=(1, 3)

Actual Output: "Hello there. How can I help you today?"

Result: 1.0

  • Characters: 39 (within 100)
  • Sentences: 2 (within 1-3 range)
Scenario 2: Character Limit Exceeded (Score: 0.0)

Failed - Too Long

Config: max_chars=50

Actual Output: "This response is definitely going to exceed the maximum character limit that was set."

Result: 0.0

Explanation: "FAILED. Exceeded chars (85/50)."

Scenario 3: Too Few Sentences (Score: 0.0)

Failed - Below Minimum

Config: sentence_range=(3, None) (at least 3 sentences)

Actual Output: "Just one sentence."

Result: 0.0

Explanation: "FAILED. Sentence count 1 outside (Range: 3-inf)."

Scenario 4: Too Many Sentences (Score: 0.0)

Failed - Above Maximum

Config: sentence_range=(None, 2) (at most 2 sentences)

Actual Output: "First point. Second point. Third point. And more!"

Result: 0.0

Explanation: "FAILED. Sentence count 4 outside (Range: 0-2)."

Signal Details

The metric returns a LengthResult object with detailed information:

Signal Type Description
char_count int Total characters in the output
max_chars_allowed int Configured maximum (or None if disabled)
sentence_count int Number of sentences detected
sentence_range tuple Configured (min, max) range
passed bool Whether all constraints were met

Why It Matters

⚡ Instant Results

O(n) computation. No LLM calls required.

📱 UI Compliance

Ensure outputs fit display constraints and character limits.

💰 Cost Control

Prevent overly verbose outputs that increase token costs.

🎯 Conciseness

Enforce brevity requirements for summaries and chatbots.

Quick Reference

TL;DR

Length Constraint = Does the output meet character and sentence limits?

  • Use it when: Enforcing length limits for UI, cost, or conciseness
  • Score interpretation: 1.0 = all constraints met, 0.0 = any violation
  • Key behavior: Character counting + regex-based sentence detection