Base Classes

Base class for chat providers.

class smarter.apps.prompt.providers.base_classes.ChatProviderBase(provider, base_url, api_key, default_model, default_system_role, default_temperature, default_max_tokens, valid_chat_completion_models, add_built_in_tools, *args, **kwargs)

Bases: ProviderDbMixin

Base class for all chat providers.

__init__(provider, base_url, api_key, default_model, default_system_role, default_temperature, default_max_tokens, valid_chat_completion_models, add_built_in_tools, *args, **kwargs)

Constructor method for the ProviderDbMixin class.

property api_key: str | None

Get the API key of the chat provider. This property returns the unmasked internal _api_key attribute.

Returns:

The unmasked API key of the chat provider.

Return type:

Optional[str]

append_error_response(error_message)

Append a message indicating that an error occurred.

Parameters:

error_message (str) – The error message to append.

Returns:

None

Return type:

None

append_message(role, content, message=None)

Append a message to the internal message list. This method validates the role and content before appending the message.

Parameters:

• role (str) – The role of the message (e.g., “user”, “assistant”, “system”).

• content (Optional[Union[dict[str, Any], list, str]]) – The content of the message. Can be a string, dict[str, Any], or list.

• message (Optional[dict[str, Any]]) – An optional message dictionary to append. If provided, it will be used instead of creating a new message.

Raises:

SmarterValueError – If the role is invalid or if both content and message are empty.

Returns:

None

Return type:

None

append_message_plugin_selected(plugin)

Append a message indicating that a plugin was selected.

Parameters:

plugin (str) – The name of the selected plugin.

Returns:

None

Return type:

None

append_message_tool_called(tool_call)

Append a message indicating that a tool was called.

Parameters:

tool_call (ChatCompletionMessageToolCallUnion) – The tool call object containing function name and arguments.

Returns:

None

Return type:

None

available_functions: dict[str, Any]

property base_url: str | None

Get the base URL of the chat provider. This property returns the internal _base_url attribute.

Returns:

The base URL of the chat provider.

Return type:

Optional[str]

completion_tokens: Optional[int]

data: Optional[dict[str, Any]]

property default_max_tokens: int | None

Get the default max completion tokens of the chat provider. This property returns the internal _default_max_completion_tokens attribute.

Returns:

The default max completion tokens of the chat provider.

Return type:

Optional[int]

property default_model: str | None

Get the default model of the chat provider. This property returns the internal _default_model attribute.

Returns:

The default model of the chat provider.

Return type:

Optional[str]

property default_system_role: str | None

Get the default system role of the chat provider. This property returns the internal _default_system_role attribute.

Returns:

The default system role of the chat provider.

Return type:

Optional[str]

property default_temperature: float | None

Get the default temperature of the chat provider. This property returns the internal _default_temperature attribute.

Returns:

The default temperature of the chat provider.

Return type:

Optional[float]

first_iteration: dict[str, Any]

first_response: Optional[ChatCompletion]

property formatted_class_name: str

Returns the class name in a formatted string along with the name of this mixin.

Returns:

The formatted class name.

Return type:

str

functions: Optional[List[str]]

get_input_text_prompt(data)

Extract the input text prompt from the incoming data. This method validates that the input text is present and is a string.

Raises:

SmarterValueError – If the input text is missing or invalid.

Parameters:

data (dict[str, Any]) – The incoming data containing the input text.

Returns:

The input text prompt.

Return type:

str

get_message_thread(data)

Initialize a new message thread with a system prompt and the incoming data. This method ensures that the system role is present in the message thread.

Raises:

SmarterValueError – If the request body is invalid.

Parameters:

data (dict[str, Any]) – The incoming data containing the message thread.

Returns:

The initialized message thread.

Return type:

List[Dict[str, str]]

input_text: Optional[str]

iteration: int

max_completion_tokens: Optional[int]

property messages: List[Dict[str, str]] | None

Get the list of messages in the chat. This property returns the internal _messages attribute.

Returns:

The list of messages.

Return type:

Optional[List[Dict[str, str]]]

messages_set_is_new(messages, is_new=False)

Set the is_new flag for all messages in the message thread. This is used to track which messages are new and which have already been processed.

This affects the treatment of messages in the Reactapp component, where new messages are styled differently.

Parameters:

messages (list[dict[str, Any]]) – The list of messages to set the is_new flag for.

Return type:

list[dict[str, Any]]

model: Optional[str]

plugins: List[PluginBase]

prompt_tokens: Optional[int]

property provider: str | None

Get the name of the chat provider. This property returns the internal _provider attribute.

Returns:

The name of the chat provider.

Return type:

Optional[str]

prune_empty_values(data)

Remove empty values from a dictionary. Some LLM providers, including MetaAI and GoogleAI will break if empty values are present in the completion request body.

Return type:

Optional[dict[str, Any]]

property ready: bool

Check if the chat provider is ready to process requests.

Returns:

True if the chat provider is ready, False otherwise.

Return type:

bool

reference: Optional[str]

request_meta_data: dict[str, Any]

second_iteration: Optional[dict[str, Any]]

second_response: Optional[ChatCompletion]

serialized_tool_calls: Optional[list[dict[str, Any]]]

temperature: Optional[float]

tools: Optional[list[dict[str, Any]]]

total_tokens: Optional[int]

property url: str | None

Get the full URL for chat completions. This property constructs the URL by appending the chat completions endpoint to the base URL.

Examples:

If the base URL is “https://api.some-llm-company.com”, the full URL will be “https://api.some-llm-company.com/v1/chat/completions”.

Returns:

The full URL for chat completions.

Return type:

Optional[str]

property valid_chat_completion_models: list[str] | None

Get the list of valid chat completion models for the chat provider. This property returns the internal _valid_chat_completion_models attribute.

A valid chat completion model is one that is supported by the chat provider’s API for chat completions.

Returns:

The list of valid chat completion models.

Return type:

Optional[list[str]]

validate()

Validate that all required properties are set. Raise SmarterValueError if any required property is missing.

Raises:

SmarterValueError – If any required property is missing.

Returns:

None

Return type:

None

class smarter.apps.prompt.providers.base_classes.HandlerProtocol(*args, **kwargs)

Bases: Protocol

A fixed Protocol for all chat provider handler functions. Ensures that all handler functions have exactly the same signature.

Parameters:

• user (User) – The user making the request.

• chat (Chat) – The chat object.

• data (Union[dict[str, Any], list]) – The request data.

• plugins (Optional[List[PluginBase]]) – Optional list of plugins to use.

• functions (Optional[list[str]]) – Optional list of function names to use.

Returns:

The response data.

Return type:

Union[dict[str, Any], list]

__init__(*args, **kwargs)

class smarter.apps.prompt.providers.base_classes.InternalKeys

Bases: object

Internal dict keys used in the chat provider.

API_KEY = 'api_key'

API_URL = 'api_url'

MAX_COMPLETION_TOKENS_KEY = 'max_completion_tokens'

MESSAGES_KEY = 'messages'

MODEL_KEY = 'model'

PLUGINS_KEY = 'plugins'

REQUEST_KEY = 'request'

RESPONSE_KEY = 'response'

SMARTER_IS_NEW = 'smarter_is_new'

SMARTER_PLUGIN_KEY = 'smarter_plugin'

TEMPERATURE_KEY = 'temperature'

TOOLS_KEY = 'tools'

TOOL_CHOICE = 'tool_choice'

class smarter.apps.prompt.providers.base_classes.OpenAICompatibleChatProvider(provider, base_url, api_key, default_model, default_system_role, default_temperature, default_max_tokens, valid_chat_completion_models, add_built_in_tools, *args, **kwargs)

Bases: ChatProviderBase

A chat provider that works with any vendor provider that is fully compatible with OpenAI’s text completion API.

append_openai_error_response(response, e=None)

Append an error message to the internal message list based on the OpenAI response.

Parameters:

response (ChatCompletion) – The OpenAI chat completion response containing the error.

Returns:

None

Return type:

None

append_openai_response(response)

Append the OpenAI-compatible response message to the internal message list. 2025-06-20: updated to use model_dump_json() to ensure compatibility with Pydantic v2. 2025-10-02: updated to validate that the response message is indeed a ChatCompletionMessage.

Parameters:

response (ChatCompletion) – The OpenAI-compatible chat completion response.

Returns:

None

Return type:

None

available_functions: dict[str, Any]

completion_tokens: Optional[int]

data: Optional[dict[str, Any]]

first_iteration: dict[str, Any]

first_response: Optional[ChatCompletion]

functions: Optional[List[str]]

handle_completion()

Handle chat completion response. This method formats the final response to be returned to the client.

Returns:

A dictionary representing the final chat completion response.

Return type:

dict

handle_function_provided(function)

Handle a function being provided.

Parameters:

function (str) – The name of the function that was provided.

Returns:

None

Return type:

None

handle_plugin_called(plugin)

handle a plugin tool call. example: SqlPlugin, ApiPlugin, StaticPlugin etc.

Parameters:

plugin (PluginBase) – The plugin instance that was called.

Returns:

None

Return type:

None

handle_plugin_selected(plugin)

Handle a plugin being selected.

does the prompt have anything to do with any of the search terms defined in a plugin? TODO: need to decide on how to resolve which of many plugin values sets to use for model, temperature, max_completion_tokens 2025-10-02: updated to validate that messages and tools are lists. 2025-10-02: updated to use plugin.plugin_meta.name for the plugin name.

Parameters:

plugin (PluginBase) – The plugin instance that was selected.

Returns:

None

Return type:

None

handle_response()

handle internal billing, and append messages to the response for prompt completion and the billing summary

Returns:

None

Return type:

None

handle_tool_called(function_name, function_args)

handle a built-in tool call. example: get_current_weather()

Parameters:

• function_name (str) – The name of the tool function called.

• function_args (str) – The arguments passed to the tool function.

Returns:

None

Return type:

None

handler(user, chat, data, plugins=None, functions=None)

Process a chat prompt request and invoke the appropriate OpenAI-compatible API endpoint.

This method orchestrates the entire chat completion workflow, including:

• Validating input and internal state.

• Initializing or updating the message thread.

• Selecting and configuring plugins and/or functions (collectively, tool calls) for the LLM.

• Preparing and sending requests to the OpenAI API (or compatible provider).

• Handling tool calls and plugin responses.

• Managing billing, logging, and signal dispatch.

• Returning a formatted HTTP response with the LLM’s output and relevant metadata.

Parameters:

• user (User) – The user instance making the request.

• chat (Chat) – The chat session instance associated with this request.

• data (Union[dict[str, Any], list]) – The request payload, typically containing a session key and a list of message dictionaries.

• plugins (Optional[list[PluginBase]]) – A list of plugin instances to be considered for selection and presentation to the LLM.

• functions (Optional[list[str]]) – A list of predefined function definitions for tool calls.

Returns:

An HTTP response dictionary (or list) containing the LLM’s output, tool call results, and metadata.

Return type:

dict or list

Raises:

• SmarterValueError – If required parameters are missing or invalid.

• SmarterConfigurationError – If there are configuration issues with the provider or plugins.

• SmarterIlligalInvocationError – If the method is invoked in an invalid state.

Note

This method manages both the initial and any required follow-up LLM requests (e.g., for tool calls). It also handles plugin selection logic and ensures that all required signals and billing events are triggered.

See also

InternalKeys OpenAIMessageKeys PluginBase ChatCompletion ChatCompletionMessageToolCall

Example usage:

response = provider.handler(
    chat=chat_instance,
    data=request_data,
    plugins=[plugin1, plugin2],
    functions=[function_definition_1, function_definition_2],
    user=current_user
)

input_text: Optional[str]

iteration: int

max_completion_tokens: Optional[int]

model: Optional[str]

property new_messages: list[dict[str, Any]]

Return a list of messages that are marked as new. This property filters the internal message list to return only those messages that have the ‘smarter_is_new’ flag set to True.

Returns:

A list of new messages.

Return type:

list[dict[str, Any]]

property openai_messages: list[dict[str, Any]]

Return a sanitized list of messages compatible with OpenAI’s chat completion API.

This property processes the internal message list, removing Smarter-specific annotations (such as metadata about tool calls and interim completion token charges) to ensure that only valid OpenAI message fields are included. This is essential for avoiding API errors related to unexpected or extraneous fields.

Returns:

A list of dictionaries representing chat messages, formatted for OpenAI’s API.

Return type:

list[dict[str, Any]]

Raises:

SmarterValueError – If the internal message list is not a list.

Example:

[
    {
        "role": "assistant",
        "content": "Welcome to Smarter!",
        "tool_calls": [
            {
                "id": "call_ABC123",
                "type": "function",
                "function": {
                    "name": "smarter_plugin_0000000045",
                    "arguments": "{"description":"AI"}"
                }
            }
        ]
    },
    {
        "role": "tool",
        "name": "smarter_plugin_0000000045",
        "content": "SqlPlugin stackademy_sql response: ...",
        "tool_call_id": "call_ABC123"
    }
]

Important

• OpenAI expects that every assistant message with a tool_calls field is immediately followed by a corresponding tool message for each tool_call_id. Failure to do so will result in an API error.

• If you include Smarter-specific fields (such as smarter_is_new) in the message list, OpenAI’s API may reject the request.

• On the first iteration, tool call responses are excluded from the message list to comply with OpenAI’s requirements.

See also

• https://platform.openai.com/docs/api-reference/chat/create

• OpenAIMessageKeys

plugins: List[PluginBase]

prep_first_request()

Prepare the first request for the chat completion. This is called at the beginning of the chat completion process.

Raises:

SmarterValueError – If the messages are not a list, or if tool definitions are invalid.

Returns:

None

Return type:

None

prep_second_request()

Prepare the second request for the chat completion. This is called in response to a tool call that requires a second request to the LLM.

Returns:

None

Return type:

None

process_tool_call(tool_call)

Process a tool call from the LLM. This method handles both built-in tool calls and plugin tool calls.

Parameters:

tool_call (ChatCompletionMessageToolCallUnion) – The tool call data from the LLM.

Returns:

None

Return type:

None

prompt_tokens: Optional[int]

reference: Optional[str]

request_meta_data: dict[str, Any]

request_meta_data_factory()

Return a dictionary of request meta data. This includes the model, temperature, max_completion_tokens, and input_text.

Returns:

A dictionary of request meta data.

Return type:

dict

second_iteration: Optional[dict[str, Any]]

second_response: Optional[ChatCompletion]

serialized_tool_calls: Optional[list[dict[str, Any]]]

temperature: Optional[float]

tools: Optional[list[dict[str, Any]]]

total_tokens: Optional[int]

smarter.apps.prompt.providers.base_classes.should_log(level)

Check if logging should be done based on the waffle switch.