Providers

A chat provider convenience class. This class is a collection of all the chat providers and their handlers. It also provides a default provider and handler.

There are a few objectives of this class: 1. To provide a single point of access to all the chat providers. 2. To hide complexity introduced into the provider classes due to Pydantic models. 3. To provide a default provider and handler.

smarter.apps.provider.services.text_completion.providers.CACHE_TIMEOUT = 10

Cache timeout in seconds for chat providers. This is to allow for short-term caching of provider instances. For now, this only benefits inner-process calls that lead to multiple instantiatiaons of the same provider within a short time frame.

However, the objective to to increase the cache timeout as observed usage patterns emerge that are confirmed to be cache safe.

class smarter.apps.provider.services.text_completion.providers.ClientTypeEnum(*values)

Bases: SmarterEnumAbstract

Client type distinguishes between the kind of handler we want from the provider.

PASSTHROUGH = 'OpenAIPassthroughClient'

SMARTER = 'OpenAISmarterClient'

class smarter.apps.provider.services.text_completion.providers.OpenAICompatibleClientFactory(client_type=ClientTypeEnum.SMARTER)

Bases: SmarterHelperMixin

A newer version of the OpenAICompatiblePassthroughChatProviders class.

__init__(client_type=ClientTypeEnum.SMARTER)

property all: List[str]

Returns a list of all provider names.

property client_type: ClientTypeEnum

default_handler(request, **kwargs)

A convenience method to get the default handler.

Return type:

Union[SmarterChatHandlerProtocol, OpenAICompatiblePassthroughProtocol]

property default_handler_name: str

Returns the name of the platform-wide default provider. If no default provider is found, it raises a SmarterValueError.

get_client_orm_by_provider_name_and_user(provider_name, user)

Return type:

Provider

get_openai_client_for_provider(provider_name, user)

Return type:

OpenAIPassthroughClient

get_passthrough_handler(request, provider_name=None, **kwargs)

Instantiates a OpenAIPassthroughClient for the given provider name and returns its passthrough handler. The key thing is that whatever handler we use here must implement the OpenAICompatiblePassthroughProtocol.

Return type:

OpenAICompatiblePassthroughProtocol

get_smarter_handler(request, provider_name=None, **kwargs)

A convenience method to get a handler by provider name.

Return type:

SmarterChatHandlerProtocol

handler(request, provider_name=None, **kwargs)

A convenience method to get a handler by provider name.

Return type:

Union[SmarterChatHandlerProtocol, OpenAICompatiblePassthroughProtocol]

smarter.apps.provider.services.text_completion.providers.should_log(level)

Check if logging should be done based on the waffle switch.

smarter.apps.provider.services.text_completion.providers.should_log_caching(level)

Check if logging should be done based on the waffle switch.