Smarter LLMTool Plugin

Overview

LLMTool Plugins (“plugins”) provide a declarative yaml manifest alternative to programming in Python in order to extend LLM tool functionality. Every kind of plugin extends the LLM tool specification, so in these docs “plugin” always means an LLMTool Plugin. Plugins are implemented in the smarter.apps.llmtool Django app. Smarter Application Manifests (SAM) are used to define LLMTool Plugins, which can be used to provide several powerful kinds of enterprise data integrations, two of which require a Connection resource as well as a Secret resource to store authentication credentials:

Plugin Types

Connection Types

Plugins are fundamentally more feature rich than traditional LLM function tools. An LLMTool Plugin manifest defines not only what proprietary data is being made available to the LLM, but also the LLM prompt specification itself (which provider, model, temperature, etc.), and most importantly, the criteria which the tool should be presented to the LLM.

Important

Imagine a use case in which you have hundreds or thousands of tools. It would be impractical (and exceedlingly expensive) to present all of those tools to the LLM for every prompt. Instead, LLMTool Plugins allow you to define:

  • Selector: CSS-like logic that defines when the tool should be made available to the LLM. That is, when it should be included in the prompt as an available tool. Remember that LLM APIs charge by token, and including tools in a prompt request increases the token count. Therefore, it behooves one to be judicious about which tools are made available to the LLM for any given prompt.

  • Prompt: The prompt specification that defines which LLM provider, the model, temperature, and other parameters to use when invoking the tool. You can even modify, or completely redefine the system prompt used when invoking the tool.

  • Data: The structured data that is made available to the LLM when the tool is invoked. This is further defined by the plugin type (static data, SQL, or API). In the example below of a Static Plugin, the first level of data keys defines the enumerations list that is presented to the tool. In the example below, the keys translate to: ‘platform provider’, ‘about’, and ‘links’.

Live Demo

Smarter Prompt Engineering Workbench Demo

See also

Usage

apiVersion: smarter.sh/v1
kind: LLMClient
metadata:
  name: stackademy_sql
  description: Stackademy University course catalogue inquiries using the Stackademy SQL plugin.
  version: 1.0.0
spec:
  config:
    prompt:
      provider: openai
      defaultModel: gpt-4-turbo
      defaultSystemRole: You are a helpful assistant.
  plugins:
    - stackademy_sql

Example Manifest

Example SQL Plugin Manifest
apiVersion: smarter.sh/v1
kind: LLMClient
metadata:
  name: stackademy_sql
  description: Stackademy University course catalogue inquiries using the Stackademy SQL plugin.
  version: 1.0.0
  tags:
    - stackademy
    - sql
    - llmclient
  annotations:
    - smarter.sh/llm-client/stackademy-sql/creator: QA Team
    - smarter.sh/llm-client/stackademy-sql/purpose: llmclient for Stackademy course catalogue inquiries
    - smarter.sh/llm-client/stackademy-sql/release-date: 2024-06-15
    - smarter.sh/llm-client/stackademy-sql/backward-compatibility: true
    - smarter.sh/llm-client/stackademy-sql/note: |
        Uses SQL plugin to answer course catalogue questions.
        Very fast and accurate when the database is well-structured.
        Leverages cache for improved performance.
spec:
  config:
    prompt:
      deployed: false
      provider: openai
      defaultModel: gpt-6-luna
      defaultSystemRole: >
        You are a helpful assistant. When given the opportunity to utilize
        function calling, you should always do so. This will allow you to
        provide the best possible responses to the user. DO NOT GUESS. IF
          YOU DON'T KNOW THE ANSWER, RESPOND THAT YOU DON'T KNOW.
      defaultTemperature: 1.0
      defaultMaxTokens: 1024
      appName: Stackademy
      appAssistant: Stanley
      appWelcomeMessage: Welcome to Stackademy! How can I help you today?
      appExamplePrompts:
        - "Do you offer any courses on AI?"
        - "My budget is $1,000. What courses can I take?"
        - "I want to study programming. What do you suggest?"
      appPlaceholder: "Ask me anything about Stackademy courses..."
      appInfoUrl: https://stackademy.edu/online-courses
    styles:  # Stackademy's colors
      # appLogoUrl: https://cdn.smarter.sh/images/llmclients/stackademy/logo.png  # coming soon: upload, then uncomment
      # appBackgroundImageUrl: https://cdn.smarter.sh/images/llmclients/stackademy/background.png  # coming soon: upload, then uncomment
      # assistantAvatarUrl: https://cdn.smarter.sh/images/llmclients/stackademy/assistant.png  # coming soon: upload, then uncomment
      googleFont: Nunito
      fontSize: 15px
      borderRadius: 16px
      conversationHeader:
        backgroundColor: '#14213d'
        color: '#ffffff'
        infoColor: '#fca311'
      messageList:
        backgroundColor: rgba(255, 255, 255, 0.85)
        color: '#14213d'
      incomingMessage:
        backgroundColor: '#e5e5e5'
        color: '#14213d'
        linkColor: '#0b57d0'
      outgoingMessage:
        backgroundColor: '#fca311'
        color: '#14213d'
      messageInput:
        backgroundColor: '#ffffff'
        editorBackgroundColor: '#f1f1f1'
        placeholderColor: '#6c757d'
      buttons:
        color: '#fca311'
      typingIndicator:
        color: '#fca311'
  plugins:
    - stackademy_sql
  functions: []

Technical Reference