Smarter LLMClient
==================

.. attention::

   The term 'LLMClient' is used interchangeably with 'Agent' and 'Workflow Unit' throughout
   this documentation.

Overview
--------

Smarter LLMClients are highly advanced conversational agents designed to provide
intelligent and context-aware interactions with human users as well as
fully automated workflows. They leverage the vanguard
of generative AI text completion technology to deliver personalized and efficient
responses. Namely, these llm_clients leverage the Smarter Plugin architecture, which
provides extensible tool integration capabilities, including secure access to
to private, secure data sources and external APIs.

LLMClients are recognized by the yaml-based :doc:`Smarter API Manifest (SAM) <../smarter-framework/smarter-api>` architecture,
facilitating both behavioral and visual customization. This allows developers to
tailor the llm_client's functionality and appearance to meet specific use cases and
user preferences.

LLMClients are managed with the :doc:`Smarter command-line interface (CLI) <../smarter-platform/cli>`.

.. seealso::

    - :doc:`Smarter API Manifest (SAM) <../smarter-framework/smarter-api>`
    - :doc:`Smarter Plugin <../smarter-resources/smarter-plugin>`
    - :doc:`Smarter Prompt <../smarter-resources/smarter-prompt>`
    - :doc:`Smarter CLI <../smarter-framework/smarter-cli>`
    - :doc:`Smarter Chat <../smarter-framework/developer-reference/react-integration/smarter-chat>`

Usage
--------

.. code-block:: bash

  # Smarter Prompt Engineer Workbench
  curl -X POST http://localhost:9357/api/v1/llm-clients/9/chat/?session_key=e5c0368d6d7201b60f4f20c470f4b5ba36faf45e80ddbe8b04b6cf20f33167a7

  # Deployed Smarter LLMClient - Production
  curl -X GET https://stackademy.3141-5926-5359.api.example.com/chat/?session_key=<SESSION-KEY>

Example Manifest
----------------

.. literalinclude:: ../../../smarter/smarter/apps/plugin/data/stackademy/stackademy-llm_client-sql.yaml
   :language: yaml
   :caption: Example Smarter Manifest



Technical Reference
-------------------

.. toctree::
   :maxdepth: 1

   llm_clients/api
   llm_clients/models
   llm_clients/sam
   llm_clients/serializers
   llm_clients/react-ui
   llm_clients/helper
   llm_clients/kubernetes-ingress
   llm_clients/management-commands
   llm_clients/middleware
   llm_clients/tasks
   llm_clients/signals
   llm_clients/urls
   llm_clients/utils


Sandbox Mode
------------

Smarter LLMClients can be operated in a 'Sandbox Mode', which restricts their
capabilities to ensure safe experimentation and testing. In this mode, llm_clients
are only addressable using URL schemes that authenticate with Django sessions.
That is, they cannot be accessed via API keys nor will they function using
URL schemas such as `stackademy.1234-5678-9012.api.example.com`.

An example sandbox mode url:

  ``https://platform.smarter.sh/workbench/stackademy-sql/chat/``



Deploying
---------

Deploy a Smarter LLMClient using the Smarter CLI. For example:

.. code-block:: bash

  smarter deploy llm_client stackademy-sql

.. code-block:: bash

  smarter deploy llm_client -h
  Deploys a LLMClient:

  smarter deploy llm_client <name> [flags]

  The Smarter API will deploy the LLMClient.

  Usage:
    smarter deploy llm_client <name> [flags]

  Flags:
    -h, --help   help for llm_client

  Global Flags:
        --api_key string         Smarter API key to use
        --config string          config file (default is $HOME/.smarter/config.yaml)
        --environment string     environment to use: local, alpha, beta, next, prod. Default is prod
    -o, --output_format string   output format: json, yaml (default "json")
    -v, --verbose                verbose output



Updating
--------

Update a Smarter LLMClient using the Smarter CLI. For example:

.. code-block:: bash

  smarter apply -f path/to/llm_client-manifest.yaml


Deleting
--------

Delete a Smarter LLMClient using the Smarter CLI. For example:

.. code-block:: bash

  smarter delete llm_client stackademy-sql


Testing
-------

Test your Smarter LLMClient using the Smarter Workbench while in Sandbox Mode.

.. raw:: html

   <div style="text-align: center;">
     <video src="https://cdn.smarter.sh/videos/read-the-docs2.mp4"
            autoplay loop muted playsinline
            style="width: 100%; height: auto; display: block; margin: 0; border-radius: 0;">
       Sorry, your browser doesn't support embedded videos.
     </video>
     <div style="font-size: 0.95em; color: #666; margin-top: 0.5em;">
       <em>Smarter Prompt Engineering Workbench Demo</em>
     </div>
   </div>
   <br/>


Monitoring
----------

Use ad hoc Sql queries to monitor your Smarter LLMClient's production performance and usage.

See:

 - :py:class:`smarter.apps.llm_client.models.LLMClientRequests`
 - :py:class:`smarter.apps.plugin.models.PluginSelectorHistory`
 - :py:class:`smarter.lib.journal.models.SAMJournal`

Scaling
-------

If you use Kubernetes Smarter LLMClients will scale seamlessly with demand.
See `https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/ <https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/>`__ for more information.

Trouble Shooting
-----------------

Beyond the Django models listed above, you should also check the smarter pod logs
for any errors. Use the following command to view the logs:

.. code-block:: bash

  kubectl logs -n <namespace> <smarter-pod-name>

for example,

.. code-block:: bash

  kubectl logs -n smarter-platform-prod smarter-68f445c866-59lmp