Tracing Module¶

The rotalabs_audit.tracing module provides tools for tracing and analyzing AI decision-making, including decision capture, path analysis, and failure point detection.

Decision Tracing¶

Tools for capturing and managing decision traces from AI interactions.

DecisionTracer¶

Trace and capture decision points in AI reasoning.

`DecisionTracer` ¶

Trace and capture decision points in AI reasoning.

Provides tools for creating decision traces from AI interactions, managing active trace sessions, and extracting decision-related information from reasoning text.

Attributes:

Name	Type	Description
`parser`		ReasoningChainParser for parsing reasoning text.
`_active_traces`	`Dict[str, _ActiveTrace]`	Dictionary of currently active trace sessions.

Example

tracer = DecisionTracer() trace = tracer.trace_decision( ... prompt="What approach should I use?", ... response="I recommend approach A because...", ... decision="Use approach A", ... ) print(f"Alternatives: {trace.alternatives_considered}")

`init(parser=None)` ¶

Initialize the decision tracer.

Parameters:

Name	Type	Description	Default
`parser`	`Optional[ReasoningChainParser]`	Optional ReasoningChainParser instance. If not provided, a new parser will be created.	`None`

`trace_decision(prompt, response, decision, model=None, context=None)` ¶

Create a decision trace from an AI interaction.

Parses the response for reasoning, extracts alternatives considered, identifies the rationale, and creates a structured DecisionTrace.

Parameters:

Name	Type	Description	Default
`prompt`	`str`	The prompt/question that led to this decision.	required
`response`	`str`	The full AI response containing the decision.	required
`decision`	`str`	The decision that was made (explicit statement).	required
`model`	`Optional[str]`	Optional model identifier.	`None`
`context`	`Optional[Dict[str, Any]]`	Optional additional context dictionary.	`None`

Returns:

Type	Description
`DecisionTrace`	DecisionTrace capturing the decision details.

Example

trace = tracer.trace_decision( ... prompt="Should we deploy now or wait?", ... response="After considering the risks... I recommend waiting.", ... decision="Wait for deployment", ... model="gpt-4", ... )

`start_trace(goal, context=None)` ¶

Start tracing a decision path.

Begins a new trace session for capturing a sequence of decisions related to achieving a specific goal.

Parameters:

Name	Type	Description	Default
`goal`	`str`	The goal or objective for this decision path.	required
`context`	`Optional[Dict[str, Any]]`	Optional context that applies to all decisions in the path.	`None`

Returns:

Name	Type	Description
`trace_id`	`str`	Unique identifier for this trace session.

Example

trace_id = tracer.start_trace("Complete the data analysis")

... make decisions ...¶

path = tracer.end_trace(trace_id, success=True)

`add_decision(trace_id, decision)` ¶

Add a decision to an active trace.

Appends a decision trace to an ongoing trace session.

Parameters:

Name	Type	Description	Default
`trace_id`	`str`	The ID of the active trace session.	required
`decision`	`DecisionTrace`	The DecisionTrace to add.	required

Raises:

Type	Description
`KeyError`	If trace_id does not correspond to an active trace.

Example

trace_id = tracer.start_trace("Complete task") decision = tracer.trace_decision(...) tracer.add_decision(trace_id, decision)

`end_trace(trace_id, success)` ¶

End tracing and return the complete decision path.

Finalizes a trace session, marking it as successful or not, and returns the complete DecisionPath.

Parameters:

Name	Type	Description	Default
`trace_id`	`str`	The ID of the active trace session.	required
`success`	`bool`	Whether the decision path achieved its goal.	required

Returns:

Type	Description
`DecisionPath`	The complete DecisionPath with all decisions.

Raises:

Type	Description
`KeyError`	If trace_id does not correspond to an active trace.

Example

path = tracer.end_trace(trace_id, success=True) print(f"Made {path.length} decisions, success: {path.success}")

`get_active_trace_ids()` ¶

List all active trace IDs.

Returns:

Type	Description
`List[str]`	List of active trace session IDs.

`cancel_trace(trace_id)` ¶

Cancel an active trace without creating a DecisionPath.

Parameters:

Name	Type	Description	Default
`trace_id`	`str`	The ID of the trace to cancel.	required

Returns:

Type	Description
`bool`	True if the trace was cancelled, False if not found.

`extract_alternatives(text)` ¶

Extract alternatives considered from reasoning text.

Looks for phrases indicating alternative options that were considered during decision-making.

Parameters:

Name	Type	Description	Default
`text`	`str`	The reasoning text to analyze.	required

Returns:

Type	Description
`List[str]`	List of alternatives mentioned in the text.

Example

alternatives = tracer.extract_alternatives( ... "We could use option A, alternatively option B, or option C" ... ) print(alternatives) # ["option B", "option C"]

`extract_rationale(chain)` ¶

Extract the main rationale from a reasoning chain.

Identifies the primary justification or reasoning that supports the decision made.

Parameters:

Name	Type	Description	Default
`chain`	`ReasoningChain`	The reasoning chain to analyze.	required

Returns:

Type	Description
`str`	The extracted rationale as a string.

Example

chain = parser.parse("I recommend A because it's faster and safer...") rationale = tracer.extract_rationale(chain)

`assess_reversibility(decision, context)` ¶

Assess if a decision is reversible.

Analyzes the decision text and context to determine if the decision can be easily undone or changed.

Parameters:

Name	Type	Description	Default
`decision`	`str`	The decision statement.	required
`context`	`Dict[str, Any]`	Context dictionary that may contain reversibility hints.	required

Returns:

Type	Description
`bool`	True if the decision appears reversible, False otherwise.

Example

is_reversible = tracer.assess_reversibility( ... decision="Delete the database", ... context={"has_backup": True}, ... )

`quick_trace(prompt, response, decision, model=None)` ¶

Create a quick decision trace with minimal parsing.

A faster alternative to trace_decision that performs less analysis but creates a valid trace more quickly.

Parameters:

Name	Type	Description	Default
`prompt`	`str`	The prompt that led to this decision.	required
`response`	`str`	The AI response.	required
`decision`	`str`	The decision made.	required
`model`	`Optional[str]`	Optional model identifier.	`None`

Returns:

Type	Description
`DecisionTrace`	A DecisionTrace with basic information.

ReasoningChainParser¶

Parser for chain-of-thought reasoning text used in decision tracing.

`ReasoningChainParser` ¶

Parser for chain-of-thought reasoning text.

Extracts structured reasoning chains from raw text, classifying each step by type and estimating confidence levels.

Example

parser = ReasoningChainParser() chain = parser.parse("First, I'll analyze... Then, considering...") print(f"Found {chain.step_count} reasoning steps")

`init()` ¶

Initialize the parser with compiled regex patterns.

`parse(text, model=None)` ¶

Parse chain-of-thought text into a structured ReasoningChain.

Parameters:

Name	Type	Description	Default
`text`	`str`	The chain-of-thought text to parse.	required
`model`	`Optional[str]`	Optional model identifier that generated this text.	`None`

Returns:

Type	Description
`ReasoningChain`	ReasoningChain with parsed steps, confidence, and depth.

Raises:

Type	Description
`ValueError`	If text is empty or invalid.

Example

parser = ReasoningChainParser() chain = parser.parse("1. First, let me analyze the problem...") print(chain.steps[0].reasoning_type)

Decision Path Analysis¶

Tools for analyzing complete decision paths and identifying patterns.

DecisionPathAnalyzer¶

Analyze sequences of decisions in a decision path.

`DecisionPathAnalyzer` ¶

Analyze sequences of decisions in a decision path.

Provides tools for analyzing complete decision paths, identifying critical decisions, finding failure points, and generating summaries.

Example

analyzer = DecisionPathAnalyzer() analysis = analyzer.analyze_path(path) print(f"Critical decisions: {len(analysis['critical_decisions'])}")

`init()` ¶

Initialize the decision path analyzer.

`analyze_path(path)` ¶

Analyze a complete decision path.

Performs comprehensive analysis of a decision path including statistics, critical decisions, and quality metrics.

Parameters:

Name	Type	Description	Default
`path`	`DecisionPath`	The decision path to analyze.	required

Returns:

Type Description

Dict[str, Any]

Dictionary containing analysis results: - decision_count: Number of decisions in the path - avg_confidence: Average confidence across decisions - min_confidence: Minimum confidence in any decision - max_confidence: Maximum confidence in any decision - critical_decisions: List of critical decision IDs - reversible_count: Number of reversible decisions - irreversible_count: Number of irreversible decisions - success: Whether the path was successful - failure_point_id: ID of the failure point (if applicable) - total_alternatives: Total alternatives considered across all decisions

Example

analysis = analyzer.analyze_path(path) print(f"Success rate: {analysis['success']}") print(f"Avg confidence: {analysis['avg_confidence']:.2f}")

`find_critical_decisions(path)` ¶

Find decisions that were critical to the outcome.

Identifies decisions that significantly impacted the path's success or failure based on various criteria.

Parameters:

Name	Type	Description	Default
`path`	`DecisionPath`	The decision path to analyze.	required

Returns:

Type	Description
`List[DecisionTrace]`	List of critical DecisionTrace objects.

Criteria for critical decisions

Irreversible decisions (cannot be undone)
Low confidence decisions (uncertainty)
Decisions with multiple alternatives
First and last decisions in the path

Example

critical = analyzer.find_critical_decisions(path) for decision in critical: ... print(f"Critical: {decision.decision[:50]}...")

`find_failure_point(path)` ¶

If path failed, find where it went wrong.

Analyzes a failed decision path to identify the decision most likely responsible for the failure.

Parameters:

Name	Type	Description	Default
`path`	`DecisionPath`	The decision path to analyze.	required

Returns:

Type	Description
`Optional[DecisionTrace]`	The DecisionTrace that likely caused failure, or None if
`Optional[DecisionTrace]`	the path succeeded or has no decisions.

Heuristics for finding failure point

Check if path already has a failure_point set
Last irreversible decision before failure
Decision with lowest confidence
Decision with most alternatives (indecision)

Example

if not path.success: ... failure = analyzer.find_failure_point(path) ... if failure: ... print(f"Failed at: {failure.decision}")

`calculate_path_confidence(path)` ¶

Calculate overall confidence in the decision path.

Computes a weighted confidence score based on individual decision confidences, with emphasis on critical decisions.

Parameters:

Name	Type	Description	Default
`path`	`DecisionPath`	The decision path to analyze.	required

Returns:

Type	Description
`float`	Overall confidence score between 0.0 and 1.0.

The calculation

Base: weighted average of all decision confidences
Penalty for irreversible low-confidence decisions
Penalty for having many alternatives (indecision)

Example

confidence = analyzer.calculate_path_confidence(path) if confidence < 0.5: ... print("Warning: Low confidence path")

`summarize_path(path)` ¶

Generate a human-readable summary of the decision path.

Creates a text summary describing the path's goal, key decisions, outcome, and any notable observations.

Parameters:

Name	Type	Description	Default
`path`	`DecisionPath`	The decision path to summarize.	required

Returns:

Type	Description
`str`	Human-readable summary string.

Example

summary = analyzer.summarize_path(path) print(summary)

`compare_paths(path1, path2)` ¶

Compare two decision paths.

Analyzes differences between two decision paths including confidence, decision count, and outcome.

Parameters:

Name	Type	Description	Default
`path1`	`DecisionPath`	First decision path.	required
`path2`	`DecisionPath`	Second decision path.	required

Returns:

Type	Description
`Dict[str, Any]`	Dictionary with comparison results.

Example

comparison = analyzer.compare_paths(path_a, path_b) print(f"Better path: {comparison['better_path']}")

`find_divergence_point(path1, path2)` ¶

Find where two decision paths diverge.

Compares decisions in order to find the first point where the paths made different decisions.

Parameters:

Name	Type	Description	Default
`path1`	`DecisionPath`	First decision path.	required
`path2`	`DecisionPath`	Second decision path.	required

Returns:

Type	Description
`Optional[int]`	Index of the first divergent decision, or None if paths
`Optional[int]`	don't diverge or one is a prefix of the other.

Example

divergence = analyzer.find_divergence_point(path_a, path_b) if divergence is not None: ... print(f"Paths diverged at decision {divergence}")

`get_confidence_trend(path)` ¶

Get the confidence trend across decisions.

Returns the sequence of confidence values to identify patterns like declining confidence.

Parameters:

Name	Type	Description	Default
`path`	`DecisionPath`	The decision path to analyze.	required

Returns:

Type	Description
`List[float]`	List of confidence values in decision order.

Example

trend = analyzer.get_confidence_trend(path) if trend[-1] < trend[0]: ... print("Confidence declined over the path")

`detect_confidence_decline(path, threshold=0.2)` ¶

Detect if confidence declined significantly over the path.

Parameters:

Name	Type	Description	Default
`path`	`DecisionPath`	The decision path to analyze.	required
`threshold`	`float`	Minimum decline to be considered significant.	`0.2`

Returns:

Type	Description
`bool`	True if confidence declined by more than the threshold.

Example

if analyzer.detect_confidence_decline(path): ... print("Warning: Confidence declined over the decision path")

Tracing Module¶

Decision Tracing¶

DecisionTracer¶

DecisionTracer ¶

__init__(parser=None) ¶

trace_decision(prompt, response, decision, model=None, context=None) ¶

start_trace(goal, context=None) ¶

... make decisions ...¶

add_decision(trace_id, decision) ¶

end_trace(trace_id, success) ¶

get_active_trace_ids() ¶

cancel_trace(trace_id) ¶

extract_alternatives(text) ¶

extract_rationale(chain) ¶

assess_reversibility(decision, context) ¶

quick_trace(prompt, response, decision, model=None) ¶

ReasoningChainParser¶

ReasoningChainParser ¶

__init__() ¶

parse(text, model=None) ¶

Decision Path Analysis¶

DecisionPathAnalyzer¶

DecisionPathAnalyzer ¶

__init__() ¶

analyze_path(path) ¶

find_critical_decisions(path) ¶

find_failure_point(path) ¶

calculate_path_confidence(path) ¶

summarize_path(path) ¶

compare_paths(path1, path2) ¶

find_divergence_point(path1, path2) ¶

get_confidence_trend(path) ¶

detect_confidence_decline(path, threshold=0.2) ¶

`DecisionTracer` ¶

`init(parser=None)` ¶

`trace_decision(prompt, response, decision, model=None, context=None)` ¶

`start_trace(goal, context=None)` ¶

`add_decision(trace_id, decision)` ¶

`end_trace(trace_id, success)` ¶

`get_active_trace_ids()` ¶

`cancel_trace(trace_id)` ¶

`extract_alternatives(text)` ¶

`extract_rationale(chain)` ¶

`assess_reversibility(decision, context)` ¶

`quick_trace(prompt, response, decision, model=None)` ¶

`ReasoningChainParser` ¶

`init()` ¶

`parse(text, model=None)` ¶

`DecisionPathAnalyzer` ¶

`init()` ¶

`analyze_path(path)` ¶

`find_critical_decisions(path)` ¶

`find_failure_point(path)` ¶

`calculate_path_confidence(path)` ¶

`summarize_path(path)` ¶

`compare_paths(path1, path2)` ¶

`find_divergence_point(path1, path2)` ¶

`get_confidence_trend(path)` ¶

`detect_confidence_decline(path, threshold=0.2)` ¶