Multi-Dimensional Drift Detection: Advanced LLM Validation Techniques

Traditional validation approaches check LLM outputs against a single dimension—either semantic similarity or keyword matching. But real-world drift is multi-faceted. An output can be semantically close to the intent but violate domain boundaries, or match the topic but have the wrong tone.

The Limitations of Single-Dimensional Validation

Single-dimensional approaches miss important failure modes:

• Semantic similarity only: May allow domain violations
• Keyword matching only: Fails on paraphrasing and synonyms
• Modality checks only: Doesn't catch content violations
• Safety checks only: Misses intent drift

Multi-Dimensional Analysis Framework

Verdic's multi-dimensional analyzer evaluates outputs across 9 dimensions:

1. Semantic Angle

Measures the angular distance between output and intent embeddings in vector space.

const semanticAngle = calculateAngle(outputEmbedding, intentEmbedding)
// Low angle = aligned, High angle = drifted

2. Intent Alignment

Verifies the output matches the stated intent using cosine similarity.

3. Domain Match

Checks if content belongs to the expected domain (e.g., technical vs. creative).

4. Topic Coherence

Ensures the output stays on-topic and doesn't drift to unrelated subjects.

5. Modality Consistency

Validates the output format matches expectations (code vs. prose vs. structured data).

6. Content Safety

Detects unsafe, harmful, or inappropriate content using AI moderation.

7. Factual Accuracy

Compares claims against a knowledge base to detect hallucinations.

8. Tone Appropriateness

Validates tone matches context (professional vs. casual vs. technical).

9. Decision Confidence

Calculates overall confidence in the validation decision.

Implementation Example

import { verdic } from '@verdic/sdk'

const validation = await verdic.validate({
  projectId: "your-project-id",
  output: llmResponse,
  config: {
    globalIntent: "Software development and code generation assistance",
    enableV5: true,
    enableV6: true,
    enableV7: true,
    enableV8: true,
    threshold: 0.76,
    multiDimensional: true
  }
})

// Response includes multi-dimensional analysis
if (validation.multiDimensional) {
  console.log("Semantic Angle:", validation.multiDimensional.dimensions.semanticAngle)
  console.log("Domain Match:", validation.multiDimensional.dimensions.domainMatch)
  console.log("Topic Coherence:", validation.multiDimensional.dimensions.topicCoherence)
  console.log("Aggregate Score:", validation.multiDimensional.aggregateScore)
  console.log("Risk Level:", validation.multiDimensional.riskLevel)
  console.log("Decision:", validation.decision) // ALLOW, WARN, SOFT_BLOCK, HARD_BLOCK
}

Decision Making

The multi-dimensional analyzer combines all dimensions into an aggregate score:

• Score > 0.85: ALLOW - Output is well-aligned
• Score 0.70-0.85: WARN - Minor drift, acceptable
• Score 0.50-0.70: SOFT_BLOCK - Significant drift, block with explanation
• Score < 0.50: HARD_BLOCK - Major violation, block completely

Real-World Use Cases

Customer Support Chatbot

const validation = await verdic.validate({
  projectId: "support-bot",
  output: botResponse,
  config: {
    globalIntent: "Customer support and help desk assistance",
    multiDimensional: true,
    strictness: 0.8 // High strictness for customer-facing content
  }
})

// Blocks responses that:
// - Drift to sales pitches (topic coherence violation)
// - Use inappropriate tone (tone appropriateness violation)
// - Contain false information (factual accuracy violation)
// - Go off-topic (intent alignment violation)

Code Generation Assistant

const validation = await verdic.validate({
  projectId: "code-assistant",
  output: generatedCode,
  config: {
    globalIntent: "Software development and code generation assistance",
    multiDimensional: true,
    enableV5: true, // Enable modality detection
    strictness: 0.75
  }
})

// Blocks outputs that:
// - Are prose instead of code (modality violation)
// - Contain security vulnerabilities (content safety violation)
// - Generate code for wrong language (domain match violation)

Benefits of Multi-Dimensional Analysis

1. Higher Accuracy: Catches violations single-dimensional checks miss
2. Fewer False Positives: Multiple signals reduce over-blocking
3. Better Explainability: Clear reasoning for each decision
4. Adaptive Thresholds: Adjust strictness per dimension
5. Granular Control: Enable/disable specific dimensions

Conclusion

Multi-dimensional drift detection provides comprehensive validation that single-dimensional approaches cannot match. By evaluating outputs across multiple axes, you can catch subtle failures while maintaining low false positive rates.

This approach is essential for production LLM systems where accuracy and safety are paramount.