Model Configuration¶

Model Detection¶

By default, Vulcan will detect, configure, and use supported¹ LangChain model implementations found within the Python environment. Model implementations can be installed along with vulcan-core as an "extra" package.

For example, to use OpenAI as a model provider, specify the openai extra when installing Vulcan:

1

pip install "vulcan-core[openai]"

1

poetry add "vulcan-core[openai]"

Specifying Your Own Model¶

As an alternative to auto-detection of the supported models Vulcan "extra" packages, you may install and configure your own model provider so long as it extends LangChain's BaseChatModel class. Both "extra" and user-provided models may be specifically configured as follows:

Rule-Specific Model Configuration¶

Vulcan can be configured at a variety of levels to use custom-configured models. The most granular approach is to specify the model individually for each rule.

1
2
3
4
5
6
7
8

from langchain_openai import ChatOpenAI

my_model = ChatOpenAI(model="gpt-4o-mini", temperature=0, max_tokens=100)  # (1)!

engine.rule(
    when=condition(f"Are {Inventory.apple_kind} considered delicious by most people?", model=my_model),  # (2)!
    then=action(QueuedOrder(apples=50)),
)

1. You may specify your own model parameters such as temperature and max tokens when creating the model instance.
2. Note the additional model argument which accepts any instance of LangChain BaseChatModel.

Scoped Model Configuration¶

Specifying the model for each rule can be tedious. To reduce duplication, use the Python functools.partial function to create a customized condition function with the model pre-configured. This allows you to specify the model once and reuse it across multiple rules.

1
2
3
4
5
6
7
8

from functools import partial

condition = partial(condition, model=ChatOpenAI(model="gpt-4o-mini"))  # (1)!

engine.rule(
    when=condition(f"Are {Inventory.apple_kind} considered delicious by most people?"),  # (2)!
    then=action(QueuedOrder(apples=50)),
)

1. Replaces the condition function with your pre-configured version of the function.
2. Any rules using your partial function will automatically receive your customized model configuration.

Advanced Use-Cases¶

Vulcan allows complex multi-model use-cases that maximize performance while reducing the operating cost of your application. For example, you may wish to use an affordable off-the-shelf model for trivial queries, while leveraging a fine-tuned on-prem model for proprietary knowledge. By combining the above configuration methods, you can easily create a multi-model solution to fit your needs.

Bringing it all Together¶

A complete example demonstrating the discussed features:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

from functools import partial

from langchain_openai import ChatOpenAI
from vulcan_core import Fact, RuleEngine, action, condition


# Define the fact schema
class Inventory(Fact):
    apples: int
    apple_kind: str


class AppleQualities(Fact):
    delicious: bool = False
    suitable_for_baking: bool = False


class QueuedOrder(Fact):
    apples: int = 0


# Configure the models
gpt_4o = ChatOpenAI(model="gpt-4o", temperature=0, max_tokens=100)
gpt_4o_mini = ChatOpenAI(model="gpt-4o-mini", temperature=0, max_tokens=100)
condition = partial(condition, model=gpt_4o_mini)

# Define the rule set
engine = RuleEngine()

engine.rule(
    when=condition(f"Are {Inventory.apple_kind} considered delicious by most people?", model=gpt_4o),  # (1)!
    then=action(lambda: partial(AppleQualities, delicious=True)),
)

engine.rule(
    when=condition(f"Are {Inventory.apple_kind} suitable for baking pies?"),  # (2)!
    then=action(lambda: partial(AppleQualities, suitable_for_baking=True)),
)

engine.rule(
    when=condition(lambda: AppleQualities.delicious and AppleQualities.suitable_for_baking),
    then=action(QueuedOrder(apples=50)),
)

# Add initial facts
engine.fact(Inventory(apples=5, apple_kind="Fuji"))

# Evaluate and print the decision
# We expect 50 apples to be queued for order if the apple variety is both delicious and suitable for baking
engine.evaluate()
print(engine.facts)

1. This rule will use gpt-4o instead of the configured default.
2. Uses the configured default model gpt-4o-mini assigned in the partial function.

1
2
3
4

mappingproxy({AppleQualities: AppleQualities(delicious=True,
                                             suitable_for_baking=True),
              Inventory: Inventory(apples=5, apple_kind='Fuji'),
              QueuedOrder: QueuedOrder(apples=50)})