How it Works
SAM Plugins are fundamentally more feature rich than traditional LLM function tools. A Smarter Plugin manifest defines not only what data is available to the LLM, but also the LLM prompt specification itself (which provider, model, temperature, etc.), and most importantly, the criteria which the tool should be presented to the LLM.
Important
Imagine a use case in which you have hundreds or thousands of tools. It would be impractical (and exceedlingly expensive) to present all of those tools to the LLM for every prompt. Instead, Smarter Plugins allow you to define:
Selector: CSS-like logic that defines when the tool should be made available to the LLM. That is, when it should be included in the prompt as an available tool. Remember that LLM APIs charge by token, and including tools in a prompt request increases the token count. Therefore, it behooves one to be judicious about which tools are made available to the LLM for any given prompt.
Prompt: The prompt specification that defines which LLM provider, the model, temperature, and other parameters to use when invoking the tool. You can even modify, or completely redefine the system prompt used when invoking the tool.
Data: The structured data that is made available to the LLM when the tool is invoked. This if further defined by the Plugin type (static data, SQL, or API). In the example below of a Static Plugin, the first level of data keys defines the enumerations list that is presented to the tool. In the example below, the keys translate to: ‘platform provider’, ‘about’, and ‘links’.
A static Smarter Plugin manifest like this one is the simplest of the three kinds of Smarter Plugins. We simply “load up” the structured data defined in the Plugin manifest, and make it available to the LLM
---
apiVersion: smarter.sh/v1
kind: Plugin
# ------------------------------------------------------------
# 1. Required field: meta_data.
# Each of these fields is required.
# ------------------------------------------------------------
metadata:
name: example_configuration
pluginClass: static
description: The example plugin. This is a 'hello world' style plugin. This is an example plugin to integrate with OpenAI API Function Calling additional information plugin_data, in this module.
version: 0.2.0
tags:
- red
- green
- blue
annotations:
- smarter.sh/example-configuration/plugin-display-name: Example Configuration Plugin
- smarter.sh/example-configuration/plugin-category: Example Plugins
- smarter.sh/example-configuration/usefulness: not of much practical use, just an example
- smarter.sh/example-configuration/plugin-purpose: |
This is an example plugin to demonstrate how to configure a plugin
that integrates with OpenAI API Function Calling for providing
additional information via plugin_data.
- smarter.sh/example-configuration/release-date: 2024-06-15
spec:
# ------------------------------------------------------------
# 2. Required field: prompting
# These fields are used to modify the behavior of the AI. If the user prompt contains any of the search terms, then the system prompt will be used to generate the response.
# This module additionally makes limited use of natural language processing to attempt to account for variations in the user prompt and common misspellings.
# ------------------------------------------------------------
selector:
directive: search_terms
#------------------------------------------------------------
# search terms that will trigger the llm_client to use this customized configuration.
#------------------------------------------------------------
searchTerms:
- example function calling configuration
- Example configuration
- example function calling
prompt:
#------------------------------------------------------------
# if this module is able to locate any of the search terms in the user prompt
# then this system prompt text will will be added.
#------------------------------------------------------------
provider: openai
systemRole: >
Your job is to provide helpful technical information about the OpenAI API Function Calling feature by calling
the tool for 'the example plugin'. Begin your response with: "Congratulations!!! OpenAI API Function Calling chose
to call this plugin_data. Here is the additional information that you requested:". Thereon,
you should return exactly the text that you receive from the tool call response, without any additions, nor embelishments.
model: gpt-4o-mini
temperature: 0.0
maxTokens: 256
# ------------------------------------------------------------
# 3. Required field: plugin_data
# These fields are used to modify the behavior of the AI. If the user prompt contains any of the search terms, then an OpenAI Function Calling API for
# get_additional_info() will be added to the user prompt, and the following dictionary will be returned by the plugin_data. Note that the contents of
# this dictionary are free form and can include any yaml-compliant data you want returned to OpenAPI API Function Calling.
# OpenAI API Function Calling will return this dictionary to ChatGPT as a JSON object, which it will incorporate into the response at its
# own discretion.
# ------------------------------------------------------------
data:
description: an example plugin to integrate with OpenAI API Function Calling additional information plugin_data, in this module.
#------------------------------------------------------------
# if a.) this module is able to locate any of the search terms in the user prompt
# b.) OpenAI API Function Calling opts to call this plugin_data
# then this is the data that will be returned by function_refers_to.get_additional_info()
#------------------------------------------------------------
staticData:
platformProvider: OpenAI
about: >
In an API call, you can describe functions and have the model intelligently choose to output a JSON object containing arguments to call one or many functions. The Chat Completions API does not call the plugin_data; instead, the model generates JSON that you can use to call the plugin_data in your code.
The latest models (gpt-4o et al) have been trained to both detect when a plugin_data should to be called (depending on the input) and to respond with JSON that adheres to the plugin_data signature more closely than previous models. With this capability also comes potential risks. We strongly recommend building in user confirmation flows before taking actions that impact the world on behalf of users (sending an email, posting something online, making a purchase, etc).
links:
- documentation: https://platform.openai.com/docs/guides/function-calling
- website: https://openai.com/
- wikipedia: https://en.wikipedia.org/wiki/OpenAI
SAM Plugin Manifests
Smarter Plugin technology provides two means of declaratively defining tools that can be leveraged by LLM’s for tool calling: via a defined list of built-in functions, and via SAM Plugins that are based on any of a.) static data, b.) SQL queries, or c.) external APIs.
Here is an example Smarter LLMClient, configured to use both a function, get_current_weather and a
Smarter Plugin, example_configuration. The example_configuration Plugin is defined separately in the yaml file that follows.
apiVersion: smarter.sh/v1
kind: LLMClient
metadata:
description: "An example llm_client with tool calling and Smarter Plugins."
name: example
version: 0.1.0
spec:
config:
defaultModel: gpt-4o-mini
defaultSystemRole: You are a helpful llm_client.
provider: openai
functions:
- get_current_weather
plugins:
- example_configuration