Smarter Plugin

Overview

Plugins provide a declarative yaml manifest alternative to programming in Python in order to extend LLM tool functionality. Smarter Application Manifests (SAM) are used to define Smarter Plugins, which can be used to provide three powerful kinds of enterprise data integrations, two of which require a Connection resource as well as a Secret resource to store authentication credentials:

Plugins Types

  • Static Plugin: These plugins provide structured data that is part of the SAM itself.

  • SQL Plugin: These plugins allow you to run docs/build/html/adr.htmlSQL queries against a connected database.

  • API Plugin: These plugins allow you to connect to external APIs.

Connection Types

Plugins are fundamentally more feature rich than traditional LLM function tools. A Smarter Plugin manifest defines not only what proprietary data is being made available to the LLM, but also the LLM prompt specification itself (which provider, model, temperature, etc.), and most importantly, the criteria which the tool should be presented to the LLM.

Important

Imagine a use case in which you have hundreds or thousands of tools. It would be impractical (and exceedlingly expensive) to present all of those tools to the LLM for every prompt. Instead, Smarter Plugins allow you to define:

  • Selector: CSS-like logic that defines when the tool should be made available to the LLM. That is, when it should be included in the prompt as an available tool. Remember that LLM APIs charge by token, and including tools in a prompt request increases the token count. Therefore, it behooves one to be judicious about which tools are made available to the LLM for any given prompt.

  • Prompt: The prompt specification that defines which LLM provider, the model, temperature, and other parameters to use when invoking the tool. You can even modify, or completely redefine the system prompt used when invoking the tool.

  • Data: The structured data that is made available to the LLM when the tool is invoked. This if further defined by the Plugin type (static data, SQL, or API). In the example below of a Static Plugin, the first level of data keys defines the enumerations list that is presented to the tool. In the example below, the keys translate to: ‘platform provider’, ‘about’, and ‘links’.

Live Demo

Smarter Prompt Engineering Workbench Demo

See also

Usage

apiVersion: smarter.sh/v1
kind: LLMClient
metadata:
  name: stackademy_sql
  description: Stackademy University course catalogue inquiries using the Stackademy SQL plugin.
  version: 1.0.0
spec:
  config:
    provider: openai
    defaultModel: gpt-4-turbo
    defaultSystemRole: You are a helpful assistant.
  plugins:
    - stackademy_sql

Example Manifest

Example SQL Plugin Manifest
apiVersion: smarter.sh/v1
kind: LLMClient
metadata:
  name: stackademy_sql
  description: Stackademy University course catalogue inquiries using the Stackademy SQL plugin.
  version: 1.0.0
  tags:
    - stackademy
    - sql
    - llm_client
  annotations:
    - smarter.sh/llm-client/stackademy-sql/creator: QA Team
    - smarter.sh/llm-client/stackademy-sql/purpose: llm_client for Stackademy course catalogue inquiries
    - smarter.sh/llm-client/stackademy-sql/release-date: 2024-06-15
    - smarter.sh/llm-client/stackademy-sql/backward-compatibility: true
    - smarter.sh/llm-client/stackademy-sql/note: |
        Uses SQL plugin to answer course catalogue questions.
        Very fast and accurate when the database is well-structured.
        Leverages cache for improved performance.
spec:
  config:
    deployed: false
    provider: openai
    defaultModel: gpt-4o-mini
    defaultSystemRole: >
      You are a helpful assistant. When given the opportunity to utilize
      function calling, you should always do so. This will allow you to
      provide the best possible responses to the user. DO NOT GUESS. IF
        YOU DON'T KNOW THE ANSWER, RESPOND THAT YOU DON'T KNOW.
    defaultTemperature: 0.5
    defaultMaxTokens: 1024
    appName: Stackademy
    appAssistant: Stanley
    appWelcomeMessage: Welcome to Stackademy! How can I help you today?
    appExamplePrompts:
      - "Do you offer any courses on AI?"
      - "My budget is $1,000. What courses can I take?"
      - "I want to study programming. What do you suggest?"
    appPlaceholder: "Ask me anything about Stackademy courses..."
    appInfoUrl: https://stackademy.edu/online-courses
  plugins:
    - stackademy_sql
  functions: []

Technical Reference