Infrastructure Services

Overview

Smarter creates and manages cloud infrastructure on behalf of its users. Deploying an LLMClient creates a DNS record for its host, and a Kubernetes Ingress whose TLS certificate is issued automatically. Registering a custom domain creates a DNS zone and a TLS certificate. A self-hosted vectorstore runs in Kubernetes, with a persistent volume for its data. And the platform sends email, for example to activate an account or to reset a password.

All of this is done through one layer of Python code, the infrastructure services, in the Django app smarter.apps.infrastructure. The rest of the platform never calls a cloud provider’s SDK, such as AWS’s boto3. It asks the infrastructure services for what it needs, in terms that do not depend on the cloud: a DNS zone, a DNS record, a TLS certificate, a Kubernetes manifest, an email.

from smarter.apps.infrastructure.services import infrastructure

zone, created = infrastructure.dns.get_or_create_zone("example.com")
certificate_id, _ = infrastructure.certificates.get_or_create_certificate("example.com")
infrastructure.certificates.create_validation_records(certificate_id)
infrastructure.kubernetes.apply_manifest(manifest)
infrastructure.email.send_email(subject="Welcome!", body="<p>Hello</p>", to="user@example.com", html=True)

Smarter runs on Amazon Web Services today. Because the platform depends only on the infrastructure services, supporting another cloud, such as Microsoft Azure, Google Cloud or DigitalOcean, means adding a cloud provider, not changing the platform.

The Services

infrastructure gives the platform five services:

  • dns: DNS zones and records. A zone, for example an AWS Route53 hosted zone, holds the records of a domain and its subdomains. Zones and records are described by the provider-independent DNSZone and DNSRecord.

  • certificates: TLS certificates, issued by the cloud provider, for a domain and its subdomains. The provider issues a certificate once DNS records prove that the domain is under Smarter’s control, so the certificate service creates those validation records with the DNS service.

  • kubernetes: the resources of the platform’s Kubernetes cluster, through kubectl.

  • email: the platform’s outgoing email, through SMTP. See SMTP Email Support.

  • provider: the cloud provider itself, for example its account identity, and a description of the Kubernetes cluster for the status API.

Each service has a ready property, which is True once it is authenticated and connected. Celery tasks that create infrastructure check it first, so that a Smarter installation without cloud credentials, such as a developer’s laptop, skips them rather than failing.

Cloud Providers

Some services depend on a cloud, and some do not:

  • DNS and TLS certificates are implemented by each cloud provider. On AWS, DNS is Route53, and certificates come from AWS Certificate Manager (ACM).

  • Kubernetes does not depend on a cloud: only the cluster’s credentials do. kubectl works the same with any cluster, so the Kubernetes service is implemented once, and the cloud provider contributes only the kubeconfig, for example with aws eks update-kubeconfig.

  • Email does not depend on a cloud either. Smarter sends it with SMTP, to any SMTP server, for example AWS Simple Email Service’s.

SMARTER_CLOUD_PROVIDER, which is smarter_settings.cloud_provider, selects the provider. It is aws by default. memory selects a provider that keeps DNS zones, records and certificates in memory, for local development without a cloud account, and for the unit tests.

SMARTER_CLOUD_PROVIDER=aws

A provider implements only a few primitives of the DNS and certificate services, for example “find a zone”, “create a zone”, “list a zone’s records”. The operations that the platform uses, for example copying the environment’s A record to a new LLMClient’s host, are built on those primitives once, in the services, so they behave the same in every cloud.

Adding a Cloud

To add a cloud, for example Azure:

  1. Add a package, smarter.apps.infrastructure.providers.azure, with a subclass of CloudProvider, and subclasses of DNSService and CertificateService that implement their abstract primitives with the cloud’s SDK. Translate the SDK’s errors into the infrastructure exceptions, with the services’ operation() context manager.

  2. Register the provider in smarter.apps.infrastructure.providers, under its CloudProviders name.

  3. Set SMARTER_CLOUD_PROVIDER=azure.

The AWS provider, smarter.apps.infrastructure.providers.aws, and the in-memory provider, smarter.apps.infrastructure.providers.memory, are examples.

Signals

The infrastructure services send Django signals, whichever provider implements them, so that the platform can observe its infrastructure without knowing which cloud it runs on. Each signal has the arguments service, for example dns, and provider, for example aws.

  • infrastructure_authenticated and infrastructure_authentication_failed: a provider authenticated with its cloud, or could not. They are sent when the state changes, rather than on every check.

  • infrastructure_connected and infrastructure_connection_failed: a service connected to its backend, for example the Kubernetes cluster, or could not.

  • billable_resource_creating, billable_resource_created, billable_resource_destroying and billable_resource_destroyed: a resource that the cloud bills for, for example a DNS zone, a persistent volume, or a load balancer, is about to be, or was, created or destroyed.

  • resource_created and resource_destroyed: a resource that is not billed on its own, for example a DNS record or a free TLS certificate, was created, updated or destroyed.

  • resource_applied: a Kubernetes manifest was applied.

  • infrastructure_operation_failed: a service’s operation failed.

  • email_sent and email_failed.

Connect a receiver to observe them:

from django.dispatch import receiver
from smarter.apps.infrastructure.signals import billable_resource_created

@receiver(billable_resource_created)
def meter(sender, service, provider, resource_type, resource_name, resource_id, **kwargs):
    ...

The Resource Ledger

Smarter’s own receivers log every infrastructure signal, and record each resource that is created or destroyed in a ledger, the Django model InfrastructureResource: its provider, service, type, name, the provider’s id for it, whether it is billable, and whether it still exists. The ledger answers “what has Smarter provisioned in our cloud account, and what is it costing us?” without asking each cloud.

Superusers see the ledger in the web console, under Settings, Infrastructure Resources: a summary of the active, billable and destroyed resources, and a list that can be filtered by status, by provider, by whether a resource is billable, and by text. The list is read-only: the platform writes the ledger as it creates and destroys resources. It is also in the Django admin.

Testing Safely

The Smarter Docker containers hold real cloud credentials, and a real kubeconfig, which can create billable resources. So the real services refuse to reach their backends from the unit tests: the AWS provider is never ready, kubectl never runs, and SMTP sends nothing. Tests install fakes instead:

from smarter.apps.infrastructure.providers import configure_provider
from smarter.apps.infrastructure.providers.memory import InMemoryProvider
from smarter.apps.infrastructure.services import InMemoryEmailService, configure_email

provider = InMemoryProvider()
configure_provider(lambda: provider)
configure_email(InMemoryEmailService)
...
configure_provider(None)
configure_email(None)

Tests of code that uses infrastructure can also patch it where it is used, for example patch("smarter.apps.llmclient.tasks.verify_custom_domain.infrastructure"). Tests that must use real infrastructure are tagged infrastructure, are skipped by default, and allow it explicitly, for example with AWSProvider(allow_in_tests=True).

Technical Reference