Infrastructure Services

Overview

Smarter creates and manages cloud infrastructure on behalf of its users. Deploying an LLMClient creates a DNS record for its host, and a Kubernetes Ingress whose TLS certificate is issued automatically. Registering a custom domain creates a DNS zone and a TLS certificate. A self-hosted vectorstore runs in Kubernetes, with a persistent volume for its data. And the platform sends email, for example to activate an account or to reset a password, and posts to Slack, for example the outcome of each deployment. A user’s data export, from Download My Data in the user menu, is saved to private storage, and shared with a link that expires.

All of this is done through one layer of Python code, the infrastructure services, in the Django app smarter.apps.infrastructure. The rest of the platform never calls a cloud provider’s SDK, such as AWS’s boto3. It asks the infrastructure services for what it needs, in terms that do not depend on the cloud: a DNS zone, a DNS record, a TLS certificate, a Kubernetes manifest, an email, a Slack message, a private file.

from smarter.apps.infrastructure.services import infrastructure

zone, created = infrastructure.dns.get_or_create_zone("example.com")
certificate_id, _ = infrastructure.certificates.get_or_create_certificate("example.com")
infrastructure.certificates.create_validation_records(certificate_id)
infrastructure.kubernetes.apply_manifest(manifest)
infrastructure.email.send_email(subject="Welcome!", body="<p>Hello</p>", to="user@example.com", html=True)
infrastructure.slack.post_message("smarter", "*Deployed*")
url = infrastructure.storage.presigned_url("data-exports/42/7/export.zip", expires_in=86400)

Smarter runs on Amazon Web Services today. Because the platform depends only on the infrastructure services, supporting another cloud, such as Microsoft Azure, Google Cloud or DigitalOcean, means adding a cloud provider, not changing the platform.

The Services

infrastructure gives the platform seven services:

  • dns: DNS zones and records. A zone, for example an AWS Route53 hosted zone, holds the records of a domain and its subdomains. Zones and records are described by the provider-independent DNSZone and DNSRecord.

  • certificates: TLS certificates, issued by the cloud provider, for a domain and its subdomains. The provider issues a certificate once DNS records prove that the domain is under Smarter’s control, so the certificate service creates those validation records with the DNS service.

  • storage: private object storage, for files that belong to one person, such as the zip file of a user’s data export. The environment’s public bucket serves the web console and the chat widget through the CDN, so nothing private can go there. An object is shared with a presigned url, which works without credentials until it expires, at most seven days later. On AWS, it is S3, in the environment’s private exports.<environment bucket> bucket, smarter_settings.aws_s3_exports_bucket_name, which smarter-infrastructure creates: it blocks all public access, encrypts its objects, refuses requests without TLS, and deletes data exports a day after they were saved.

  • kubernetes: the resources of the platform’s Kubernetes cluster, through kubectl.

  • email: the platform’s outgoing email, through SMTP. See SMTP Email Support.

  • slack: the platform’s messages to Slack, through the Slack Web API, as the bot of SMARTER_SLACK_BOT_TOKEN. It posts messages, and uploads files into a channel or a message’s thread. The Helm chart’s init Job and nightly database backup post their outcome and log with it (see Smarter on Kubernetes). Like email, it never raises: a failure is logged, and returned as None or False.

  • logs: the last 72 hours of the platform’s pod logs, from the Loki that the Helm chart runs, at SMARTER_LOKI_URL. A query is structured, not LogQL: the service escapes every value, so that a caller, such as an LLM’s tool call, cannot inject a query of its own. The MCPServer’s staff-only tools read it.

  • provider: the cloud provider itself, for example its account identity, and a description of the Kubernetes cluster for the status API.

Each service has a ready property, which is True once it is authenticated and connected. Celery tasks that create infrastructure check it first, so that a Smarter installation without cloud credentials, such as a developer’s laptop, skips them rather than failing.

Cloud Providers

Some services depend on a cloud, and some do not:

  • DNS, TLS certificates and private storage are implemented by each cloud provider. On AWS, DNS is Route53, certificates come from AWS Certificate Manager (ACM), and private storage is S3.

  • Kubernetes does not depend on a cloud: only the cluster’s credentials do. kubectl works the same with any cluster, so the Kubernetes service is implemented once, and the cloud provider contributes only the kubeconfig, for example with aws eks update-kubeconfig.

  • Email does not depend on a cloud either. Smarter sends it with SMTP, to any SMTP server, for example AWS Simple Email Service’s.

  • Slack does not depend on a cloud: it is Slack’s own Web API.

SMARTER_CLOUD_PROVIDER, which is smarter_settings.cloud_provider, selects the provider. It is aws by default. memory selects a provider that keeps DNS zones, records, certificates and private objects in memory, for local development without a cloud account, and for the unit tests.

SMARTER_CLOUD_PROVIDER=aws

A provider implements only a few primitives of the DNS and certificate services, for example “find a zone”, “create a zone”, “list a zone’s records”. The operations that the platform uses, for example copying the environment’s A record to a new LLMClient’s host, are built on those primitives once, in the services, so they behave the same in every cloud.

Adding a Cloud

To add a cloud, for example Azure:

  1. Add a package, smarter.apps.infrastructure.providers.azure, with a subclass of CloudProvider, and subclasses of DNSService and CertificateService that implement their abstract primitives with the cloud’s SDK. Translate the SDK’s errors into the infrastructure exceptions, with the services’ operation() context manager.

  2. Register the provider in smarter.apps.infrastructure.providers, under its CloudProviders name.

  3. Set SMARTER_CLOUD_PROVIDER=azure.

The AWS provider, smarter.apps.infrastructure.providers.aws, and the in-memory provider, smarter.apps.infrastructure.providers.memory, are examples.

Credentials

Smarter needs two kinds of credentials: the cloud’s, for DNS, certificates, node groups and storage, and the Kubernetes cluster’s, for kubectl. Where it runs decides where they come from, and in each case they carry only the permissions that the platform uses, for one environment.

In a Kubernetes pod, as the Helm chart deploys it, Smarter stores no credentials at all:

  • The app, the Celery workers, beat and the init job run as the chart’s smarter ServiceAccount. kubectl authenticates as it, with the cluster address and token that Kubernetes mounts in every pod (in_cluster_server()), so Smarter has exactly the permissions of the chart’s Role in its own namespace, plus a read-only ClusterRole for nodes and persistent volumes.

  • On AWS, the ServiceAccount is annotated with an IAM role (the chart’s serviceAccount.awsRoleArn), and the EKS pod identity webhook gives the pods short-lived credentials for it (IAM Roles for Service Accounts). The MariaDB backup CronJob has a role of its own, which can only upload backups. Static keys in the pod’s environment would win over the role, so the deploy leaves them out when a role is set.

In the Docker containers of a developer’s laptop, there is neither a pod nor a role, so Smarter uses the access key in .env (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, or AWS_PROFILE), and authenticates to the cluster with a token that it creates from that key. smarter-infrastructure creates an IAM user for this, in its local environment, with the same permissions as a deployed environment’s pods, scoped to the local environment and its namespace. Don’t use a deploy key, or an administrator’s key, in .env.

The workflows that build and deploy an environment use a third identity: a deploy user per environment, whose key is in the GitHub environment’s secrets. It can push the environment’s image, and is an administrator of the environment’s namespace only. The platform never uses it.

Signals

The infrastructure services send Django signals, whichever provider implements them, so that the platform can observe its infrastructure without knowing which cloud it runs on. Each signal has the arguments service, for example dns, and provider, for example aws.

  • infrastructure_authenticated and infrastructure_authentication_failed: a provider authenticated with its cloud, or could not. They are sent when the state changes, rather than on every check.

  • infrastructure_connected and infrastructure_connection_failed: a service connected to its backend, for example the Kubernetes cluster, or could not.

  • billable_resource_creating, billable_resource_created, billable_resource_destroying and billable_resource_destroyed: a resource that the cloud bills for, for example a DNS zone, a persistent volume, or a load balancer, is about to be, or was, created or destroyed.

  • resource_created and resource_destroyed: a resource that is not billed on its own, for example a DNS record or a free TLS certificate, was created, updated or destroyed.

  • resource_applied: a Kubernetes manifest was applied.

  • infrastructure_operation_failed: a service’s operation failed.

  • email_sent and email_failed.

Connect a receiver to observe them:

from django.dispatch import receiver
from smarter.apps.infrastructure.signals import billable_resource_created

@receiver(billable_resource_created)
def meter(sender, service, provider, resource_type, resource_name, resource_id, **kwargs):
    ...

The Resource Ledger

Smarter’s own receivers log every infrastructure signal, and record each resource that is created or destroyed in a ledger, the Django model InfrastructureResource: its provider, service, type, name, the provider’s id for it, whether it is billable, and whether it still exists. The ledger answers “what has Smarter provisioned in our cloud account, and what is it costing us?” without asking each cloud.

Much of the infrastructure that Smarter runs on isn’t created by the platform: the Kubernetes cluster, its add-ons and its node groups are provisioned with Terraform, nodes come and go with the cluster’s autoscaler, and cert-manager issues the TLS certificates of the ingresses. The inventory, smarter.apps.infrastructure.services.inventory, discovers them, and reconciles the ledger with what exists. The cluster is shared by every environment, so besides the cluster itself, that is, the cluster, its add-ons, node groups and nodes, it discovers only the resources of the environment’s namespace, smarter_settings.environment_namespace. It records these resource types:

  • kubernetes.cluster: the managed cluster, for example AWS EKS, which is billed by the hour.

  • kubernetes.addon: the cluster’s managed add-ons, for example the VPC CNI, CoreDNS and the EBS CSI driver.

  • kubernetes.nodegroup: the managed node groups. Their nodes are billed, not the groups.

  • kubernetes.node: the compute nodes, for example EC2 instances. Billable.

  • kubernetes.persistentvolume: the block storage volumes, for example EBS volumes, that are bound to claims in the environment’s namespace. Billable.

  • kubernetes.loadbalancer: the environment’s Services of type LoadBalancer, which provision cloud load balancers. Billable.

  • kubernetes.ingress and kubernetes.certificate: the environment’s Ingresses and cert-manager Certificates.

A resource that the ledger doesn’t know is recorded as created, and an active one that no longer exists as destroyed, with the same signals as the services send, so the ledger is still written only by its receivers. A resource type that can’t be listed, for example because the cluster is unavailable, is left as it is. The inventory only reads the cloud: it never creates or destroys anything. Celery Beat runs it every 15 minutes, with the task smarter.apps.infrastructure.tasks.sync_infrastructure_inventory, on the infrastructure queue, and manage.py initialize_platform runs it once at the end, so a new platform’s ledger isn’t empty until the first scheduled run. To run it on demand:

python manage.py sync_infrastructure_inventory

Superusers see the ledger in the web console, under Settings, Infrastructure Resources: a summary of the active, billable and destroyed resources, and a list that can be filtered by status, by provider, by whether a resource is billable, and by text. The list is read-only: the platform writes the ledger as it creates, destroys and discovers resources. It is also in the Django admin.

Testing Safely

The Smarter Docker containers hold real cloud credentials, and a real kubeconfig, which can create billable resources. So the real services refuse to reach their backends from the unit tests: the AWS provider is never ready, kubectl never runs, SMTP sends nothing, and Slack posts nothing. Tests install fakes instead:

from smarter.apps.infrastructure.providers import configure_provider
from smarter.apps.infrastructure.providers.memory import InMemoryProvider
from smarter.apps.infrastructure.services import (
    InMemoryEmailService,
    InMemorySlackService,
    configure_email,
    configure_slack,
)

provider = InMemoryProvider()
configure_provider(lambda: provider)
configure_email(InMemoryEmailService)
configure_slack(InMemorySlackService)
...
configure_provider(None)
configure_email(None)
configure_slack(None)

Tests of code that uses infrastructure can also patch it where it is used, for example patch("smarter.apps.llmclient.tasks.verify_custom_domain.infrastructure"). Tests that must use real infrastructure are tagged infrastructure, are skipped by default, and allow it explicitly, for example with AWSProvider(allow_in_tests=True).

Technical Reference