Infrastructure Services
Overview
Smarter creates and manages cloud infrastructure on behalf of its users. Deploying an LLMClient creates a DNS record for its host, and a Kubernetes Ingress whose TLS certificate is issued automatically. Registering a custom domain creates a DNS zone and a TLS certificate. A self-hosted vectorstore runs in Kubernetes, with a persistent volume for its data. And the platform sends email, for example to activate an account or to reset a password, and posts to Slack, for example the outcome of each deployment. A user’s data export, from Download My Data in the user menu, is saved to private storage, and shared with a link that expires.
All of this is done through one layer of Python code, the infrastructure services, in the
Django app smarter.apps.infrastructure. The rest of the platform never calls a cloud
provider’s SDK, such as AWS’s boto3. It asks the infrastructure services for what it needs, in
terms that do not depend on the cloud: a DNS zone, a DNS record, a TLS certificate, a
Kubernetes manifest, an email, a Slack message, a private file.
from smarter.apps.infrastructure.services import infrastructure
zone, created = infrastructure.dns.get_or_create_zone("example.com")
certificate_id, _ = infrastructure.certificates.get_or_create_certificate("example.com")
infrastructure.certificates.create_validation_records(certificate_id)
infrastructure.kubernetes.apply_manifest(manifest)
infrastructure.email.send_email(subject="Welcome!", body="<p>Hello</p>", to="user@example.com", html=True)
infrastructure.slack.post_message("smarter", "*Deployed*")
url = infrastructure.storage.presigned_url("data-exports/42/7/export.zip", expires_in=86400)
Smarter runs on Amazon Web Services today. Because the platform depends only on the infrastructure services, supporting another cloud, such as Microsoft Azure, Google Cloud or DigitalOcean, means adding a cloud provider, not changing the platform.
The Services
infrastructure gives the platform seven services:
dns: DNS zones and records. A zone, for example an AWS Route53 hosted zone, holds the records of a domain and its subdomains. Zones and records are described by the provider-independentDNSZoneandDNSRecord.certificates: TLS certificates, issued by the cloud provider, for a domain and its subdomains. The provider issues a certificate once DNS records prove that the domain is under Smarter’s control, so the certificate service creates those validation records with the DNS service.storage: private object storage, for files that belong to one person, such as the zip file of a user’s data export. The environment’s public bucket serves the web console and the chat widget through the CDN, so nothing private can go there. An object is shared with a presigned url, which works without credentials until it expires, at most seven days later. On AWS, it is S3, in the environment’s privateexports.<environment bucket>bucket,smarter_settings.aws_s3_exports_bucket_name, which smarter-infrastructure creates: it blocks all public access, encrypts its objects, refuses requests without TLS, and deletes data exports a day after they were saved.kubernetes: the resources of the platform’s Kubernetes cluster, through kubectl.email: the platform’s outgoing email, through SMTP. See SMTP Email Support.slack: the platform’s messages to Slack, through the Slack Web API, as the bot ofSMARTER_SLACK_BOT_TOKEN. It posts messages, and uploads files into a channel or a message’s thread. The Helm chart’s init Job and nightly database backup post their outcome and log with it (see Smarter on Kubernetes). Like email, it never raises: a failure is logged, and returned asNoneorFalse.logs: the last 72 hours of the platform’s pod logs, from the Loki that the Helm chart runs, atSMARTER_LOKI_URL. A query is structured, not LogQL: the service escapes every value, so that a caller, such as an LLM’s tool call, cannot inject a query of its own. The MCPServer’s staff-only tools read it.provider: the cloud provider itself, for example its account identity, and a description of the Kubernetes cluster for thestatusAPI.
Each service has a ready property, which is True once it is authenticated and
connected. Celery tasks that create infrastructure check it first, so that a Smarter
installation without cloud credentials, such as a developer’s laptop, skips them rather than
failing.
Cloud Providers
Some services depend on a cloud, and some do not:
DNS, TLS certificates and private storage are implemented by each cloud provider. On AWS, DNS is Route53, certificates come from AWS Certificate Manager (ACM), and private storage is S3.
Kubernetes does not depend on a cloud: only the cluster’s credentials do. kubectl works the same with any cluster, so the Kubernetes service is implemented once, and the cloud provider contributes only the kubeconfig, for example with
aws eks update-kubeconfig.Email does not depend on a cloud either. Smarter sends it with SMTP, to any SMTP server, for example AWS Simple Email Service’s.
Slack does not depend on a cloud: it is Slack’s own Web API.
SMARTER_CLOUD_PROVIDER, which is smarter_settings.cloud_provider, selects the provider.
It is aws by default. memory selects a provider that keeps DNS zones, records,
certificates and private objects in memory, for local development without a cloud account, and for the unit tests.
SMARTER_CLOUD_PROVIDER=aws
A provider implements only a few primitives of the DNS and certificate services, for example “find a zone”, “create a zone”, “list a zone’s records”. The operations that the platform uses, for example copying the environment’s A record to a new LLMClient’s host, are built on those primitives once, in the services, so they behave the same in every cloud.
Adding a Cloud
To add a cloud, for example Azure:
Add a package,
smarter.apps.infrastructure.providers.azure, with a subclass ofCloudProvider, and subclasses ofDNSServiceandCertificateServicethat implement their abstract primitives with the cloud’s SDK. Translate the SDK’s errors into the infrastructure exceptions, with the services’operation()context manager.Register the provider in
smarter.apps.infrastructure.providers, under itsCloudProvidersname.Set
SMARTER_CLOUD_PROVIDER=azure.
The AWS provider, smarter.apps.infrastructure.providers.aws, and the in-memory provider,
smarter.apps.infrastructure.providers.memory, are examples.
Credentials
Smarter needs two kinds of credentials: the cloud’s, for DNS, certificates, node groups and storage, and the Kubernetes cluster’s, for kubectl. Where it runs decides where they come from, and in each case they carry only the permissions that the platform uses, for one environment.
In a Kubernetes pod, as the Helm chart deploys it, Smarter stores no credentials at all:
The app, the Celery workers, beat and the init job run as the chart’s
smarterServiceAccount. kubectl authenticates as it, with the cluster address and token that Kubernetes mounts in every pod (in_cluster_server()), so Smarter has exactly the permissions of the chart’s Role in its own namespace, plus a read-only ClusterRole for nodes and persistent volumes.On AWS, the ServiceAccount is annotated with an IAM role (the chart’s
serviceAccount.awsRoleArn), and the EKS pod identity webhook gives the pods short-lived credentials for it (IAM Roles for Service Accounts). The MariaDB backup CronJob has a role of its own, which can only upload backups. Static keys in the pod’s environment would win over the role, so the deploy leaves them out when a role is set.
In the Docker containers of a developer’s laptop, there is neither a pod nor a role, so
Smarter uses the access key in .env (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY,
or AWS_PROFILE), and authenticates to the cluster with a token that it creates from that
key. smarter-infrastructure creates an IAM user for this, in its local environment, with the
same permissions as a deployed environment’s pods, scoped to the local environment and its
namespace. Don’t use a deploy key, or an administrator’s key, in .env.
The workflows that build and deploy an environment use a third identity: a deploy user per environment, whose key is in the GitHub environment’s secrets. It can push the environment’s image, and is an administrator of the environment’s namespace only. The platform never uses it.
Signals
The infrastructure services send Django signals, whichever provider implements them, so that
the platform can observe its infrastructure without knowing which cloud it runs on. Each signal
has the arguments service, for example dns, and provider, for example aws.
infrastructure_authenticatedandinfrastructure_authentication_failed: a provider authenticated with its cloud, or could not. They are sent when the state changes, rather than on every check.infrastructure_connectedandinfrastructure_connection_failed: a service connected to its backend, for example the Kubernetes cluster, or could not.billable_resource_creating,billable_resource_created,billable_resource_destroyingandbillable_resource_destroyed: a resource that the cloud bills for, for example a DNS zone, a persistent volume, or a load balancer, is about to be, or was, created or destroyed.resource_createdandresource_destroyed: a resource that is not billed on its own, for example a DNS record or a free TLS certificate, was created, updated or destroyed.resource_applied: a Kubernetes manifest was applied.infrastructure_operation_failed: a service’s operation failed.email_sentandemail_failed.
Connect a receiver to observe them:
from django.dispatch import receiver
from smarter.apps.infrastructure.signals import billable_resource_created
@receiver(billable_resource_created)
def meter(sender, service, provider, resource_type, resource_name, resource_id, **kwargs):
...
The Resource Ledger
Smarter’s own receivers log every infrastructure signal, and record each resource that is
created or destroyed in a ledger, the Django model
InfrastructureResource: its provider, service,
type, name, the provider’s id for it, whether it is billable, and whether it still exists. The
ledger answers “what has Smarter provisioned in our cloud account, and what is it costing us?”
without asking each cloud.
Much of the infrastructure that Smarter runs on isn’t created by the platform: the Kubernetes
cluster, its add-ons and its node groups are provisioned with Terraform, nodes come and go with
the cluster’s autoscaler, and cert-manager issues the TLS certificates of the ingresses. The
inventory, smarter.apps.infrastructure.services.inventory, discovers them, and reconciles
the ledger with what exists. The cluster is shared by every environment, so besides the cluster
itself, that is, the cluster, its add-ons, node groups and nodes, it discovers only the
resources of the environment’s namespace, smarter_settings.environment_namespace. It records these resource types:
kubernetes.cluster: the managed cluster, for example AWS EKS, which is billed by the hour.kubernetes.addon: the cluster’s managed add-ons, for example the VPC CNI, CoreDNS and the EBS CSI driver.kubernetes.nodegroup: the managed node groups. Their nodes are billed, not the groups.kubernetes.node: the compute nodes, for example EC2 instances. Billable.kubernetes.persistentvolume: the block storage volumes, for example EBS volumes, that are bound to claims in the environment’s namespace. Billable.kubernetes.loadbalancer: the environment’s Services of type LoadBalancer, which provision cloud load balancers. Billable.kubernetes.ingressandkubernetes.certificate: the environment’s Ingresses and cert-manager Certificates.
A resource that the ledger doesn’t know is recorded as created, and an active one that no
longer exists as destroyed, with the same signals as the services send, so the ledger is still
written only by its receivers. A resource type that can’t be listed, for example because the
cluster is unavailable, is left as it is. The inventory only reads the cloud: it never creates
or destroys anything. Celery Beat runs it every 15 minutes, with the task
smarter.apps.infrastructure.tasks.sync_infrastructure_inventory, on the infrastructure queue,
and manage.py initialize_platform runs it once at the end, so a new platform’s ledger isn’t
empty until the first scheduled run. To run it on demand:
python manage.py sync_infrastructure_inventory
Superusers see the ledger in the web console, under Settings, Infrastructure Resources: a summary of the active, billable and destroyed resources, and a list that can be filtered by status, by provider, by whether a resource is billable, and by text. The list is read-only: the platform writes the ledger as it creates, destroys and discovers resources. It is also in the Django admin.
Testing Safely
The Smarter Docker containers hold real cloud credentials, and a real kubeconfig, which can create billable resources. So the real services refuse to reach their backends from the unit tests: the AWS provider is never ready, kubectl never runs, SMTP sends nothing, and Slack posts nothing. Tests install fakes instead:
from smarter.apps.infrastructure.providers import configure_provider
from smarter.apps.infrastructure.providers.memory import InMemoryProvider
from smarter.apps.infrastructure.services import (
InMemoryEmailService,
InMemorySlackService,
configure_email,
configure_slack,
)
provider = InMemoryProvider()
configure_provider(lambda: provider)
configure_email(InMemoryEmailService)
configure_slack(InMemorySlackService)
...
configure_provider(None)
configure_email(None)
configure_slack(None)
Tests of code that uses infrastructure can also patch it where it is used, for example
patch("smarter.apps.llmclient.tasks.verify_custom_domain.infrastructure"). Tests that must
use real infrastructure are tagged infrastructure, are skipped by default, and allow it
explicitly, for example with AWSProvider(allow_in_tests=True).