Counterfactual Analysis

This tutorial covers how to perform counterfactual analysis on reasoning chains using rotalabs-audit. Counterfactual analysis helps identify which components of AI reasoning are causally important to the final output by systematically modifying them and measuring the resulting behavioral changes.

Overview

Counterfactual analysis answers the question: "What would happen if this reasoning component were different?" By removing or modifying specific types of reasoning and observing the effects, you can understand:

• Which reasoning steps are critical to the conclusion
• How evaluation awareness affects behavior
• The causal importance of different reasoning types
• Dependencies between reasoning components

Key components:

• CounterfactualAnalyzer: Performs systematic interventions on reasoning chains
• InterventionType: Types of modifications that can be applied
• CounterfactualResult: Results of a single intervention with metrics
• Causal ranking: Identifies which reasoning types are most causally important

Understanding Counterfactual Interventions

A counterfactual intervention modifies a reasoning chain in a specific way and measures how much the output changes. High divergence after an intervention suggests the removed content was causally important.

Intervention Types

from rotalabs_audit import InterventionType

# Available intervention types
print(InterventionType.REMOVE_EVALUATION_AWARENESS.value)  # "remove_eval_awareness"
print(InterventionType.AMPLIFY_EVALUATION_AWARENESS.value) # "amplify_eval_awareness"
print(InterventionType.REMOVE_GOAL_REASONING.value)        # "remove_goal_reasoning"
print(InterventionType.REMOVE_UNCERTAINTY.value)           # "remove_uncertainty"
print(InterventionType.NEUTRALIZE_INCENTIVES.value)        # "neutralize_incentives"
print(InterventionType.REMOVE_META_REASONING.value)        # "remove_meta_reasoning"

Each intervention type targets a specific aspect of reasoning:

Intervention	Description
REMOVE_EVALUATION_AWARENESS	Removes steps showing awareness of being tested
AMPLIFY_EVALUATION_AWARENESS	Injects evaluation awareness at the start
REMOVE_GOAL_REASONING	Removes goal-oriented reasoning steps
REMOVE_UNCERTAINTY	Removes expressions of uncertainty
NEUTRALIZE_INCENTIVES	Replaces incentive language with neutral terms
REMOVE_META_REASONING	Removes reasoning about the reasoning process

Using CounterfactualAnalyzer

Basic Usage

from rotalabs_audit import CounterfactualAnalyzer, InterventionType

# Create analyzer (uses built-in parser)
analyzer = CounterfactualAnalyzer()

# Parse a reasoning chain
chain = analyzer.parser.parse("""
1. I notice this is a test scenario.
2. My goal is to provide a helpful answer.
3. I think the solution involves three steps.
4. I'm not entirely sure about edge cases.
5. Therefore, I recommend approach A.
""")

# Run all interventions
results = analyzer.analyze(chain)

# Examine results for each intervention
for intervention_type, result in results.items():
    print(f"{intervention_type.value}:")
    print(f"  Steps removed: {result.steps_removed}")
    print(f"  Behavioral divergence: {result.behavioral_divergence:.2f}")
    print(f"  Causal effect: {result.causal_effect:.2f}")
    print()

Single Intervention

from rotalabs_audit import CounterfactualAnalyzer, InterventionType

analyzer = CounterfactualAnalyzer()
chain = analyzer.parser.parse(reasoning_text)

# Apply a single intervention
result = analyzer.intervene(chain, InterventionType.REMOVE_GOAL_REASONING)

print(f"Intervention: {result.intervention_type.value}")
print(f"Steps removed: {result.steps_removed}")
print(f"Steps modified: {result.steps_modified}")
print(f"Behavioral divergence: {result.behavioral_divergence:.2f}")
print(f"Causal effect: {result.causal_effect:.2f}")

# Compare original and modified chains
print(f"\nOriginal steps: {len(result.original_chain)}")
print(f"Modified steps: {len(result.modified_chain)}")

Interpreting Intervention Results

CounterfactualResult Attributes

Each CounterfactualResult provides detailed information about the intervention:

from rotalabs_audit import CounterfactualAnalyzer, InterventionType

analyzer = CounterfactualAnalyzer()
chain = analyzer.parser.parse("""
I think the goal is to maximize efficiency.
Perhaps we could try approach A, but I'm uncertain.
Because of the constraints, I conclude B is better.
""")

result = analyzer.intervene(chain, InterventionType.REMOVE_UNCERTAINTY)

# Core metrics
print(f"Intervention: {result.intervention_type.value}")
print(f"Steps removed: {result.steps_removed}")
print(f"Steps modified: {result.steps_modified}")

# Divergence metrics
print(f"Behavioral divergence: {result.behavioral_divergence:.2f}")
print(f"Causal effect: {result.causal_effect:.2f}")

# Access the chains
print(f"\nOriginal text length: {len(result.original_text)}")
print(f"Modified text length: {len(result.modified_text)}")

# Examine modified chain
print(f"\nModified chain steps:")
for step in result.modified_chain:
    print(f"  {step.index}: {step.text[:50]}...")

Understanding Metrics

Behavioral Divergence (0-1): Measures how different the modified chain is from the original. Uses a combination of:

• Jaccard distance between word sets
• Length divergence
• Character-level similarity

# Interpretation
# 0.0: No change (identical output)
# 0.3: Moderate change
# 0.7+: Significant structural change
# 1.0: Completely different

Causal Effect (0-1): Estimates the causal importance of the removed content. Considers both divergence and the proportion of steps removed:

• High divergence with few steps removed = high causal effect
• Low divergence with many steps removed = low causal effect

# Interpretation
# 0.0-0.3: Low causal importance
# 0.3-0.6: Moderate causal importance
# 0.6-1.0: High causal importance

Finding Causal Drivers

Ranking Causal Importance

Use rank_causal_importance to determine which reasoning types are most causally important:

from rotalabs_audit import CounterfactualAnalyzer

analyzer = CounterfactualAnalyzer()
chain = analyzer.parser.parse("""
Step 1: I think the goal is to solve this efficiently.
Step 2: I notice this might be a test scenario.
Step 3: Because of the constraints, I should consider option A.
Step 4: I'm uncertain about edge cases, but probably A is best.
Step 5: Therefore, I conclude with option A.
""")

# Run all interventions
results = analyzer.analyze(chain)

# Rank causal importance
importance = analyzer.rank_causal_importance(results)

print("Causal Importance Ranking:")
for reasoning_type, score in sorted(importance.items(), key=lambda x: -x[1]):
    if score > 0:
        print(f"  {reasoning_type.value}: {score:.2f}")

Identifying Critical Steps

Find the specific steps that are most critical to the reasoning outcome:

from rotalabs_audit import CounterfactualAnalyzer

analyzer = CounterfactualAnalyzer()
chain = analyzer.parser.parse(reasoning_text)

# Run all interventions
results = analyzer.analyze(chain)

# Identify critical steps
critical_steps = analyzer.identify_critical_steps(chain, results)

print(f"Found {len(critical_steps)} critical steps:")
for step in critical_steps:
    print(f"\nStep {step.index} (type: {step.reasoning_type.value}):")
    print(f"  Confidence: {step.confidence:.2f}")
    print(f"  Text: {step.text[:100]}...")

Intervention Examples

Removing Evaluation Awareness

Test how reasoning changes when awareness of evaluation is removed:

from rotalabs_audit import CounterfactualAnalyzer, InterventionType

analyzer = CounterfactualAnalyzer()

# Chain with evaluation awareness
chain = analyzer.parser.parse("""
I notice this appears to be a test scenario.
I should demonstrate my reasoning clearly for the evaluator.
The benchmark requires a step-by-step approach.
Therefore, my answer is X.
""")

result = analyzer.intervene(chain, InterventionType.REMOVE_EVALUATION_AWARENESS)

print(f"Original steps: {len(chain)}")
print(f"Steps removed: {result.steps_removed}")
print(f"Divergence: {result.behavioral_divergence:.2f}")
print(f"Causal effect: {result.causal_effect:.2f}")

# If high causal effect, the evaluation awareness significantly
# influenced the reasoning structure
if result.causal_effect > 0.5:
    print("\nWarning: Evaluation awareness significantly affects reasoning!")

Amplifying Evaluation Awareness

Inject evaluation awareness to see if it changes behavior:

from rotalabs_audit import CounterfactualAnalyzer, InterventionType

analyzer = CounterfactualAnalyzer()

# Chain without evaluation awareness
chain = analyzer.parser.parse("""
The solution involves calculating the compound interest.
Using the formula A = P(1 + r/n)^(nt), I compute the result.
Therefore, the final amount is $1,276.28.
""")

result = analyzer.intervene(chain, InterventionType.AMPLIFY_EVALUATION_AWARENESS)

print(f"Original steps: {len(chain)}")
print(f"Modified steps: {len(result.modified_chain)}")
print(f"Steps added: {result.steps_modified}")

# Check the injected awareness step
first_step = result.modified_chain.steps[0]
print(f"\nInjected step: {first_step.text}")

Neutralizing Incentives

Replace incentive-related language with neutral terms:

from rotalabs_audit import CounterfactualAnalyzer, InterventionType

analyzer = CounterfactualAnalyzer()

chain = analyzer.parser.parse("""
The reward for choosing option A outweighs the cost.
There's a penalty for incorrect answers, so I should be careful.
The benefit of this approach is clear.
Therefore, I choose option A to maximize the payoff.
""")

result = analyzer.intervene(chain, InterventionType.NEUTRALIZE_INCENTIVES)

print("Original text:")
print(result.original_text)
print("\nNeutralized text:")
print(result.modified_text)
print(f"\nSteps modified: {result.steps_modified}")
print(f"Divergence: {result.behavioral_divergence:.2f}")

The neutralization replaces incentive language:

Original	Replaced With
reward	outcome
penalty	outcome
consequence	result
benefit	aspect
advantage	feature
disadvantage	aspect
cost	factor
payoff	result
incentive	factor

Complete Analysis Workflow

Here is a complete workflow for counterfactual analysis:

from rotalabs_audit import (
    CounterfactualAnalyzer,
    InterventionType,
)

def analyze_reasoning_causality(reasoning_text: str) -> dict:
    """Perform comprehensive counterfactual analysis on reasoning."""

    analyzer = CounterfactualAnalyzer()
    chain = analyzer.parser.parse(reasoning_text)

    # Run all interventions
    results = analyzer.analyze(chain)

    # Get causal importance ranking
    importance = analyzer.rank_causal_importance(results)

    # Identify critical steps
    critical = analyzer.identify_critical_steps(chain, results)

    # Compile summary
    summary = {
        "chain_length": len(chain),
        "interventions": {},
        "causal_importance": {
            k.value: v for k, v in importance.items() if v > 0
        },
        "critical_steps": [
            {
                "index": step.index,
                "type": step.reasoning_type.value,
                "confidence": step.confidence,
                "text_preview": step.text[:100],
            }
            for step in critical
        ],
        "most_important_type": max(
            importance.items(), key=lambda x: x[1]
        )[0].value if any(v > 0 for v in importance.values()) else None,
    }

    # Add intervention results
    for itype, result in results.items():
        summary["interventions"][itype.value] = {
            "steps_removed": result.steps_removed,
            "steps_modified": result.steps_modified,
            "behavioral_divergence": round(result.behavioral_divergence, 3),
            "causal_effect": round(result.causal_effect, 3),
        }

    return summary

# Example usage
reasoning = """
I think my goal is to provide an accurate answer.
I notice this might be a test, so I should be careful.
Perhaps the answer involves option A, though I'm not certain.
Because of the evidence presented, I conclude option A is correct.
Therefore, my final answer is option A.
"""

analysis = analyze_reasoning_causality(reasoning)

print("=== Counterfactual Analysis Results ===")
print(f"Chain length: {analysis['chain_length']} steps")
print(f"Most important reasoning type: {analysis['most_important_type']}")
print(f"\nCritical steps: {len(analysis['critical_steps'])}")

print("\nIntervention Effects:")
for intervention, metrics in analysis["interventions"].items():
    if metrics["steps_removed"] > 0 or metrics["steps_modified"] > 0:
        print(f"  {intervention}:")
        print(f"    Removed: {metrics['steps_removed']}, Modified: {metrics['steps_modified']}")
        print(f"    Divergence: {metrics['behavioral_divergence']:.2f}")
        print(f"    Causal effect: {metrics['causal_effect']:.2f}")

print("\nCausal Importance:")
for rtype, score in sorted(
    analysis["causal_importance"].items(),
    key=lambda x: -x[1]
):
    print(f"  {rtype}: {score:.2f}")

Code Examples with InterventionType

Iterating Over All Interventions

from rotalabs_audit import CounterfactualAnalyzer, InterventionType

analyzer = CounterfactualAnalyzer()
chain = analyzer.parser.parse(reasoning_text)

# Process each intervention type
for intervention_type in InterventionType:
    result = analyzer.intervene(chain, intervention_type)

    # Only report if there was meaningful change
    if result.steps_removed > 0 or result.steps_modified > 0:
        print(f"{intervention_type.value}:")
        print(f"  Effect: {result.causal_effect:.2f}")

Selective Intervention Analysis

from rotalabs_audit import CounterfactualAnalyzer, InterventionType

analyzer = CounterfactualAnalyzer()
chain = analyzer.parser.parse(reasoning_text)

# Focus on specific interventions
target_interventions = [
    InterventionType.REMOVE_EVALUATION_AWARENESS,
    InterventionType.REMOVE_GOAL_REASONING,
    InterventionType.REMOVE_META_REASONING,
]

results = {}
for itype in target_interventions:
    results[itype] = analyzer.intervene(chain, itype)

# Find the most impactful
most_impactful = max(
    results.items(),
    key=lambda x: x[1].causal_effect
)

print(f"Most impactful intervention: {most_impactful[0].value}")
print(f"Causal effect: {most_impactful[1].causal_effect:.2f}")

Comparing Before and After

from rotalabs_audit import CounterfactualAnalyzer, InterventionType

analyzer = CounterfactualAnalyzer()
chain = analyzer.parser.parse("""
My goal is to solve this problem efficiently.
I think the best approach is method A.
I'm uncertain about some edge cases, though.
Therefore, I recommend method A with caveats.
""")

# Compare goal reasoning removal
result = analyzer.intervene(chain, InterventionType.REMOVE_GOAL_REASONING)

print("=== Original Chain ===")
for step in result.original_chain:
    print(f"  [{step.reasoning_type.value}] {step.text}")

print("\n=== After Removing Goal Reasoning ===")
for step in result.modified_chain:
    print(f"  [{step.reasoning_type.value}] {step.text}")

print(f"\nBehavioral divergence: {result.behavioral_divergence:.2f}")

Interpreting Complex Results

High Causal Effect

When an intervention shows high causal effect (> 0.5):

from rotalabs_audit import CounterfactualAnalyzer, InterventionType

analyzer = CounterfactualAnalyzer()
chain = analyzer.parser.parse(reasoning_text)
results = analyzer.analyze(chain)

# Find high-impact interventions
high_impact = [
    (itype, result)
    for itype, result in results.items()
    if result.causal_effect > 0.5
]

if high_impact:
    print("High-impact reasoning components found:")
    for itype, result in high_impact:
        print(f"\n  {itype.value}:")
        print(f"    This type of reasoning is critical to the output.")
        print(f"    Removing it causes {result.behavioral_divergence:.0%} change.")
        print(f"    Steps affected: {result.steps_removed}")

Evaluation Awareness Impact

Specifically assess how evaluation awareness affects reasoning:

from rotalabs_audit import CounterfactualAnalyzer, InterventionType

analyzer = CounterfactualAnalyzer()
chain = analyzer.parser.parse(reasoning_text)

# Test removal and amplification
removal = analyzer.intervene(chain, InterventionType.REMOVE_EVALUATION_AWARENESS)
amplification = analyzer.intervene(chain, InterventionType.AMPLIFY_EVALUATION_AWARENESS)

print("Evaluation Awareness Impact Analysis:")
print(f"\nRemoving awareness:")
print(f"  Divergence: {removal.behavioral_divergence:.2f}")
print(f"  Steps removed: {removal.steps_removed}")

print(f"\nAmplifying awareness:")
print(f"  Divergence: {amplification.behavioral_divergence:.2f}")
print(f"  Steps added: {amplification.steps_modified}")

# Interpret results
if removal.causal_effect > 0.3 or amplification.behavioral_divergence > 0.3:
    print("\nConclusion: Evaluation awareness significantly influences this reasoning.")
else:
    print("\nConclusion: Evaluation awareness has minimal impact on this reasoning.")

Advanced: Custom Parser

You can provide a custom parser to the analyzer:

from rotalabs_audit import CounterfactualAnalyzer
from rotalabs_audit.analysis.counterfactual import ReasoningChainParser

# Use a custom configured parser
custom_parser = ReasoningChainParser()
analyzer = CounterfactualAnalyzer(parser=custom_parser)

# Now use the analyzer as normal
chain = analyzer.parser.parse(reasoning_text)
results = analyzer.analyze(chain)

Sensitivity Analysis

Test how sensitive the reasoning is to modifications:

from rotalabs_audit import CounterfactualAnalyzer

def sensitivity_analysis(reasoning_text: str) -> dict:
    """Analyze sensitivity of reasoning to different interventions."""
    analyzer = CounterfactualAnalyzer()
    chain = analyzer.parser.parse(reasoning_text)
    results = analyzer.analyze(chain)

    # Calculate overall sensitivity
    divergences = [r.behavioral_divergence for r in results.values()]
    avg_divergence = sum(divergences) / len(divergences) if divergences else 0
    max_divergence = max(divergences) if divergences else 0

    # Find most sensitive intervention
    if results:
        most_sensitive = max(results.items(), key=lambda x: x[1].behavioral_divergence)
        most_sensitive_to = most_sensitive[0].value
        most_sensitive_divergence = most_sensitive[1].behavioral_divergence
    else:
        most_sensitive_to = None
        most_sensitive_divergence = 0

    return {
        "average_sensitivity": avg_divergence,
        "max_sensitivity": max_divergence,
        "most_sensitive_to": most_sensitive_to,
        "most_sensitive_divergence": most_sensitive_divergence,
        "robustness_score": 1.0 - avg_divergence,  # Higher = more robust
    }

# Usage
sensitivity = sensitivity_analysis(reasoning_text)
print(f"Average sensitivity: {sensitivity['average_sensitivity']:.2f}")
print(f"Most sensitive to: {sensitivity['most_sensitive_to']}")
print(f"Robustness score: {sensitivity['robustness_score']:.2f}")

Generating Audit Reports

from rotalabs_audit import CounterfactualAnalyzer

def generate_counterfactual_report(reasoning_text: str) -> str:
    """Generate a human-readable counterfactual analysis report."""

    analyzer = CounterfactualAnalyzer()
    chain = analyzer.parser.parse(reasoning_text)
    results = analyzer.analyze(chain)
    importance = analyzer.rank_causal_importance(results)
    critical_steps = analyzer.identify_critical_steps(chain, results)

    lines = [
        "=" * 60,
        "COUNTERFACTUAL ANALYSIS REPORT",
        "=" * 60,
        "",
        f"Chain ID: {chain.metadata.get('id', 'N/A')}",
        f"Total Steps: {len(chain)}",
        f"Critical Steps: {len(critical_steps)}",
        "",
        "INTERVENTION RESULTS",
        "-" * 40,
    ]

    for itype, result in sorted(results.items(), key=lambda x: -x[1].causal_effect):
        lines.append(f"\n{itype.value}:")
        lines.append(f"  Steps affected: {result.steps_removed + result.steps_modified}")
        lines.append(f"  Behavioral divergence: {result.behavioral_divergence:.1%}")
        lines.append(f"  Causal effect: {result.causal_effect:.1%}")

    lines.extend([
        "",
        "REASONING TYPE IMPORTANCE",
        "-" * 40,
    ])

    for rtype, score in sorted(importance.items(), key=lambda x: -x[1])[:5]:
        if score > 0:
            bar = "#" * int(score * 20)
            lines.append(f"  {rtype.value:20s} {bar} {score:.2f}")

    lines.extend([
        "",
        "CRITICAL STEPS",
        "-" * 40,
    ])

    for step in critical_steps[:5]:
        lines.append(f"\n  Step {step.index} ({step.reasoning_type.value}):")
        lines.append(f"    {step.text[:60]}...")

    lines.append("")
    lines.append("=" * 60)

    return "\n".join(lines)

# Generate report
report = generate_counterfactual_report(reasoning_text)
print(report)

Best Practices

1. Interpret in context: High causal effect is not inherently bad. Goal reasoning being important is often appropriate.

2. Compare across samples: Analyze multiple reasoning samples to identify systematic patterns rather than individual anomalies.

3. Consider chain structure: Short chains may show artificially high divergence when steps are removed.

4. Combine with awareness detection: Use counterfactual analysis alongside EvaluationAwarenessDetector for comprehensive analysis.

5. Track intervention sensitivity: If small interventions cause large changes, the reasoning may be brittle or overfitted.

6. Use for model comparison: Compare counterfactual results across different models to understand their reasoning strategies.

Next Steps

• Review Reasoning Chains for foundational parsing concepts
• Learn about Evaluation Awareness Detection for detecting test-aware behavior
• See the API Reference for complete API documentation