Explainability Reports¶
Explainability reports provide detailed insight into Vulcan rule evaluation, the facts used in decisions, and the reasoning behind AI responses.
Reports are important for debugging complex rule sets, understanding system behavior, and meeting compliance requirements in regulated environments.
Generating Reports¶
To generate an explainability report, first enable auditing during rule evaluation with engine.evaluate(audit=True):
After rule evaluation completes, you can generate a YAML report as follows:
| Python | |
|---|---|
report:
iterations:
- id: 0
timestamp: '2025-07-14T13:21:00.204306Z'
elapsed: 0.0
matches:
- rule: 7ec4bdbc:Calculate total cost
timestamp: '2025-07-14T13:21:00.204318Z'
elapsed: 0.0
evaluation: True = (CartItem.quantity|3| > 0)
consequences:
TotalCost.amount: 7.5
The above shows the rule evaluated CartItem.quantity > 0 as True (since quantity was 3), and the consequence of it evaluating True was to set TotalCost.amount to 7.5.
Reporting Scope
Explainability data is reset on every call to engine.evaluate(). If you wish to retain a report, be sure to copy it before making further evaluations.
Why AI Explainability is Important¶
Understanding subtle behaviors exhibited by LLMs is often difficult. Their generated outputs are predictive in nature, utilizing large mathematical models, which make their decision-making opaque and non-deterministic. An LLM's output may vary greatly for similar inputs due to input settings (temperature, top-k, etc.), model snapshot version, seed values, and subtle prompt variations such as extra whitespace.
Explainable AI is not only important for reliability, but is also important for compliance and accountability in regulated industries like finance, healthcare, and law.
Vulcan addresses these limitations by promoting micro-prompting techniques combined with computed (instead of predicted) logical reasoning. Complex prompts are broken down into smaller, easier-to-explain decisions, allowing deeper AI traceability through an explicit chain of rule evaluations, fact changes, and logical conditions.
Report Structure¶
Iterations, Rule Matching, and Ordering¶
Vulcan evaluates rules iteratively and in batches. During each iteration, Vulcan selects rules that reference facts that have changed, evaluates those rules, and then applies any resulting actions to modify other facts. This cycle continues until no further rules match changed facts, or until an iteration limit is reached.
A visual representation of this process is shown below:
sequenceDiagram
participant User
participant Engine as RuleEngine
participant WM as Working Memory
participant Rules as Rule Definitions
User->>Engine: evaluate()
rect rgb(240, 248, 255)
Note over Engine, Rules: Iteration Loop
WM->>Engine: Find Fact/Rule matches
Engine->>Rules: Evaluate conditions with Facts
Rules->>Engine: Condition result (True/False)
Engine->>WM: Apply Rule actions
end
Engine->>Engine: Repeat iteration loop until<br/> no more facts change
Engine->>User: Evaluation complete
Each iteration in the report contains:
- id: Sequential iteration number starting from 0
- timestamp: When the iteration started (ISO 8601 format)
- elapsed: Total time spent processing this iteration (in seconds)
- matches: Array of rules that were evaluated during this iteration
Let's see this in action with the following example:
report:
iterations:
- id: 0
timestamp: '2025-07-14T13:21:00.266903Z'
elapsed: 0.0
matches:
- rule: b0078915:Calculate subtotal
timestamp: '2025-07-14T13:21:00.266916Z'
elapsed: 0.0
evaluation: True = (CartItem.quantity|5| > 0)
consequences:
Subtotal.amount: 12.5
- id: 1
timestamp: '2025-07-14T13:21:00.267024Z'
elapsed: 0.0
matches:
- rule: 1bf7272a:Large order discount
timestamp: '2025-07-14T13:21:00.267030Z'
elapsed: 0.0
evaluation: True = (Subtotal.amount|12.5| > 10.0)
consequences:
Discount.percentage: 0.1
- id: 2
timestamp: '2025-07-14T13:21:00.267129Z'
elapsed: 0.0
matches:
- rule: a9f89063:Calculate final total
timestamp: '2025-07-14T13:21:00.267135Z'
elapsed: 0.0
evaluation: True = (Subtotal.amount|12.5| > 0 and Discount.percentage|0.1| >= 0)
consequences:
FinalTotal.amount: 11.25
The above rules depend on one another for evaluation. As a result, Vulcan will process them in dependency order, based on when their dependent fact information becomes available in the working memory. We see this in the report as three iterations containing the rules and fact values.
Rule Evaluation¶
For each rule that matches Facts in an iteration, the evaluation information contains the condition expression, fact information, and the logical result.
The format is {result} = {expression}, where:
- result: The boolean outcome of the rule's
whenstatement - expression: The condition(s) with the evaluated values wrapped by
|characters
Let's explore the evaluation information with a simple example:
report:
iterations:
- id: 0
timestamp: '2025-07-14T13:21:00.286678Z'
elapsed: 0.0
matches:
- rule: 34f26b5a:Special handling for large perishable orders
timestamp: '2025-07-14T13:21:00.286690Z'
elapsed: 0.0
evaluation: False = (CartItem.quantity|5| > 10 and CartItem.is_perishable|True|)
consequences: None
The evaluation line shows:
- The rule's
whenclause evaluated toFalse - During evaluation, the value of
CartItem.quantitywas5 - During evaluation, the value of
CartItem.is_perishablewasTrue - Since
5 > 10is false, the entireandexpression is false - The
consequencesfield showsNonebecause thewhencondition was not met
Lambda vs Decorated Functions¶
Depending on whether lambda: conditions or decorated functions are used in a rule's when statement, the report notation will vary slightly to indicate the difference. Lambda conditions are always surrounded by parentheses, while decorated functions are shown as a function call.
Below is an example using a decorated function condition:
report:
iterations:
- id: 0
timestamp: '2025-07-14T13:21:00.324568Z'
elapsed: 0.0
matches:
- rule: c2b0c3b8:Apply volume discount
timestamp: '2025-07-14T13:21:00.324579Z'
elapsed: 0.0
evaluation: False = qualifies_for_volume_discount()|False|
consequences:
Discount.percentage: 0
- id: 1
timestamp: '2025-07-14T13:21:00.324693Z'
elapsed: 0.0
matches:
- rule: c527fc87:Apply loyalty discount
timestamp: '2025-07-14T13:21:00.324699Z'
elapsed: 0.0
evaluation: False = (Discount.percentage|0| < 0 and Customer.is_loyalty_member|True|)
consequences: None
The above example demonstrates that:
- Lambda conditions: Show the full expression with evaluated values
- Decorated functions: Show the function name and result:
qualifies_for_discount()|True|
Short-Circuit Evaluation¶
Vulcan uses short-circuit evaluation in logical expressions. This means that if a condition evaluates to False, subsequent conditions in an or or and expression are not evaluated. This is useful for performance and avoiding unnecessary computations.
Below is an example of how this is shown in the report:
report:
iterations:
- id: 0
timestamp: '2025-07-14T13:21:00.357357Z'
elapsed: 0.0
matches:
- rule: 6cce3261:Lambda short circuit evaluation
timestamp: '2025-07-14T13:21:00.357369Z'
elapsed: 0.0
evaluation: False = (Customer.age|18| >= 21 and TotalCost.amount|5.0| > 100.0)
consequences: None
- rule: 2f43c0ec:Compound condition short circuit evaluation
timestamp: '2025-07-14T13:21:00.357495Z'
elapsed: 0.0
evaluation: True = (TotalCost.amount|5.0| < 10.0) or loyalty_check()|-|
consequences:
Checkout.special_handling_required: true
In lambda: expressions, the report will display all the Fact values that would have been used, even though evaluation would not have considered those Facts due to short-circuiting. As no computation is needed to resolve the values, Vulcan includes them in the report.
However, for compound conditions, the result is not known until after evaluation. As Vulcan does not evaluate unnecessary parts of compound conditions, it will show the unknown values as |-| in the report. In the above example, this can be seen as loyalty_check()|-|, indicating that the decorated condition was not evaluated because of the first condition.
Contextual Information¶
Reports provide evaluation values inline for easy reference. However, for long strings and Similarity Fact attributes, including them can make the report difficult to read. To address this, reports may include a context section that provides the complete value of long strings and retrieved similarity searches, while retaining only the Fact reference in the evaluation for improved readability.
Below is an example of how this is shown in reports:
report:
iterations:
- id: 0
timestamp: '2025-07-14T13:21:00.376838Z'
elapsed: 0.0
matches:
- rule: 424d225c:Vocal customer discount
timestamp: '2025-07-14T13:21:00.376869Z'
elapsed: 0.0
evaluation: False = (len(Customer.review_history) > 100)
consequences: None
context:
- Customer.review_history: A long string that represents customer reviews
In the above report, we can see that the long string value was not displayed inline, but instead left as references with their values appearing in the context section.
Rule Consequences¶
For each rule match, the report will show the result of actions that were taken (if any). Fact attributes that were modified by the rule will appear in the consequences section. This is the same for both then actions (when the condition is True) or inverse actions (when the condition is False).
The consequences format is FactName.attribute: value, indicating which Fact attributes were modified.
Below is a scenario involving rules that demonstrate inverse and no consequences:
- Only if the rule's dependent Facts are present in the working memory will the rule apply the
thenaction if the condition evaluates toTrue, or theinverseaction if the condition evaluates toFalse.
report:
iterations:
- id: 0
timestamp: '2025-07-14T13:21:00.399432Z'
elapsed: 0.0
matches:
- rule: 5c314431:Age verification
timestamp: '2025-07-14T13:21:00.399444Z'
elapsed: 0.0
evaluation: False = (Customer.age|19| >= 21)
consequences:
Checkout.age_verification_required: true
- rule: 7a81de31:VIP processing
timestamp: '2025-07-14T13:21:00.399534Z'
elapsed: 0.0
evaluation: False = (TotalCost.amount|45| > 200)
consequences: None
Reviewing the consequences section, we can see the following actions occurred during rule evaluation:
- Age verification rule: Since the condition was
False(19 is not >= 21), theinverseaction fired, settingCheckout.age_verification_required: true - VIP processing rule: Since the condition was
False(45 is not > 200) and noinverseaction was defined, the consequences showNonesince no action was taken.
AI Rationale¶
A powerful feature of Vulcan is visibility into an LLM's reasoning process when evaluating rules. This is captured in the rationale section and populated only for AI conditions.
report:
iterations:
- id: 0
timestamp: '2025-07-14T13:21:01.854969Z'
elapsed: 1.214
matches:
- rule: edfcae78:Customer retention discount
timestamp: '2025-07-14T13:21:01.855002Z'
elapsed: 1.214
evaluation: True = Does the {Customer.review_history} overall sentiment indicate an unhappy customer?
consequences:
Discount.percentage: 0.15
context:
- Customer.review_history: 'January: Worst company ever.;Feburary: I got a good deal with a coupon!;March: Got broken product.'
rationale: The review history includes both negative and positive sentiments, but the negative reviews outweigh the positive ones, indicating overall unhappiness.
AI Rationale Accuracy
As the rationale is generated by the LLM when it evaluated the condition, it may not always be accurate or complete. While it does provide some insight into the LLM's reasoning process, it should not be considered definitive proof of the decision-making logic used by the LLM. Furthermore, the accuracy of the rationale depends on the specific LLM configuration and model used.
For improved reasoning and explainability, use short, focused questions in LLM queries. This maximizes the likelihood that the LLM will respond correctly to the query and provide an accurate explanation. By combining the responses of several small LLM queries using Vulcan's computed rule capabilities, you can achieve improved accuracy and overall traceability of the decision-making process.
Rule Warnings¶
During audited rule evaluations, Vulcan detects potential issues and reports them as warnings.
Below is an example that will trigger various warnings in the report:
report:
iterations:
- id: 0
timestamp: '2025-07-14T13:21:03.090673Z'
elapsed: 0.0
matches:
- rule: 9a5b47a0:Approve payment for loyalty members
timestamp: '2025-07-14T13:21:03.090684Z'
elapsed: 0.0
evaluation: True = (Customer.is_loyalty_member|True|)
consequences:
Checkout.payment_approved: true
- rule: 533ce3b0:For demonstration, negate the previous rule
timestamp: '2025-07-14T13:21:03.090790Z'
elapsed: 0.0
evaluation: True = (Customer.is_loyalty_member|True|)
consequences:
Checkout.payment_approved: false
warnings:
- Rule Ordering | Rule:9a5b47a0 consequence (Checkout.payment_approved|True|) was overridden by Rule:533ce3b0 (Checkout.payment_approved|False|) within the same iteration
- rule: c8ca9130:Deny payment for minors
timestamp: '2025-07-14T13:21:03.090872Z'
elapsed: 0.0
evaluation: True = (Customer.age|17| <= 18)
consequences:
Checkout.payment_approved: false
Checkout.special_handling_required: false
Checkout.age_verification_required: false
warnings:
- Fact Replacement | Rule:c8ca9130 consequence replaces (Checkout), potentially altering unintended attributes. Consider using a partial update to ensure only intended changes.
Vulcan detects and reports the following warnings:
- Rule Ordering: When multiple rules modify the same fact attribute within the same iteration, it may lead to unexpected behavior depending on rule ordering. In the above example, both
VIP processingandAge verificationrules modified theCheckout.age_verification_requiredattribute in the same iteration. - Fact Replacement: When a rule updates an entire fact instead of using partial updates, it may cause unintended side effects if the Fact declares default values. In the above example, the
VIP processingrule replaced the entireCheckoutfact, which will replace all attributes with their defaults if not specified in the rule's action.
Best Practices for Explainability¶
When designing rules for optimal explainability reporting, consider these best practices:
Use Descriptive Rule Names¶
Clear names make reports easier to understand.
Descriptive Rule Names
| Python | |
|---|---|
Vague or Missing Rule Names
Compose Complex Conditions¶
Single-responsibility conditions are easier to trace. When assigned to a variable prior to rule definition, they can make rules more readable and the conditions can be reused across rules.
Composed Conditions
Long Complex Conditions
Bringing it All Together¶
Let's create a comprehensive grocery store checkout system that demonstrates all the reporting features we've covered. This system will handle item scanning, discounts, loyalty programs, and payment processing with full explainability.
| Python | |
|---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 | |
report:
iterations:
- id: 0
timestamp: \'2025-07-14T13:21:03.270933Z\'
elapsed: 0.0
matches:
- rule: fca7c562:Calculate initial subtotal
timestamp: \'2025-07-14T13:21:03.270948Z\'
elapsed: 0.0
evaluation: True = (CartItem.quantity|12| > 0)
consequences:
Pricing.subtotal: 51.0
- rule: 3e25ed9e:Special handling for perishables
timestamp: \'2025-07-14T13:21:03.271093Z\'
elapsed: 0.0
evaluation: True = (CartItem.is_perishable|True| and CartItem.quantity|12| > 5)
consequences:
Checkout.special_handling_required: true
- rule: 15ee8273:Age verification for restricted items
timestamp: \'2025-07-14T13:21:03.271212Z\'
elapsed: 0.0
evaluation: True = (CartItem.category|alcohol| == "alcohol")
consequences:
Checkout.age_verification_required: true
- id: 1
timestamp: \'2025-07-14T13:21:03.271328Z\'
elapsed: 1.518
matches:
- rule: 1e49556c:Bulk discount eligibility
timestamp: \'2025-07-14T13:21:03.271333Z\'
elapsed: 0.0
evaluation: True = (CartItem.quantity|12| >= 10 and Pricing.subtotal|51.0| > 30.0)
consequences:
Promotions.bulk_discount_applied: true
- rule: a4ebc897:Customer retention discount
timestamp: \'2025-07-14T13:21:03.271392Z\'
elapsed: 1.517
evaluation: True = (Pricing.subtotal|51.0| > 30.0) and Does the {Customer.review_history} overall sentiment indicate an unhappy customer? and (Customer.is_loyalty_member|True|)
consequences:
Promotions.loyalty_discount_applied: true
context:
- Customer.review_history: January - Worst company ever.;February - I got a good deal with a coupon!;March - Got broken product.
rationale: The review history contains both negative and positive sentiments, but the negative reviews (\'Worst company ever.\' and \'Got broken product.\') outweigh the positive one (\'I got a good deal with a coupon!\'). Therefore, the overall sentiment indicates an unhappy customer.
- rule: d93118d3:Payment approval
timestamp: \'2025-07-14T13:21:04.788919Z\'
elapsed: 0.0
evaluation: False = (Customer.age|28| >= 21 or not Checkout.age_verification_required|True|) and (Pricing.total_amount|0.0| > 0)
consequences: None
- id: 2
timestamp: \'2025-07-14T13:21:04.788990Z\'
elapsed: 0.0
matches:
- rule: 15e3ae4b:Apply bulk discount amount
timestamp: \'2025-07-14T13:21:04.788996Z\'
elapsed: 0.0
evaluation: True = (Promotions.bulk_discount_applied|True|)
consequences:
Pricing.discount_amount: 7.65
Promotions.applied: true
- rule: 58f0ac02:Apply loyalty discount amount
timestamp: \'2025-07-14T13:21:04.789100Z\'
elapsed: 0.0
evaluation: False = (Promotions.loyalty_discount_applied|True| and not Promotions.bulk_discount_applied|True|)
consequences: None
- id: 3
timestamp: \'2025-07-14T13:21:04.789132Z\'
elapsed: 0.0
matches:
- rule: 3233f1a3:Calculate final total with tax
timestamp: \'2025-07-14T13:21:04.789138Z\'
elapsed: 0.0
evaluation: True = (Promotions.applied|True| and Pricing.subtotal|51.0| > 0)
consequences:
Pricing.total_amount: 46.82
- id: 4
timestamp: \'2025-07-14T13:21:04.789206Z\'
elapsed: 0.0
matches:
- rule: d93118d3:Payment approval
timestamp: \'2025-07-14T13:21:04.789213Z\'
elapsed: 0.0
evaluation: True = (Customer.age|28| >= 21 or not Checkout.age_verification_required|True|) and (Pricing.total_amount|46.82| > 0)
consequences:
Checkout.payment_approved: true
- rule: 29287609:High-value purchase alert
timestamp: \'2025-07-14T13:21:04.789296Z\'
elapsed: 0.0
evaluation: False = (Pricing.total_amount|46.82| > 100.0)
consequences: None