Kubernetes for Running Applications at Scale

Why Smarter. Kubernetes courses usually deploy a sample app that has none of the parts that make Kubernetes worth learning: background workers, a database, a cache, secrets, autoscaling and network isolation. Smarter has all of them, and its Helm chart is the one that runs it in production. The chart deploys the same images that students ran in Docker Compose, so the course can start from something familiar and add only what Kubernetes brings:

  • The same image, as four Deployments: the web server, the worker, the infrastructure worker and the scheduler, with liveness, readiness and startup probes on /healthz/ and /readiness/.

  • Horizontal scaling: autoscalers that scale the web server from 3 to 25 pods at 75% CPU, and the worker on its own.

  • Lifecycle Jobs: a post-install init Job that creates the database, migrates and initializes the platform, and a CronJob that backs the database up to S3.

  • Configuration that is checked: values validated by a JSON Schema, and Bitnami subcharts for MySQL or MariaDB, Redis, Grafana, Prometheus and Loki, each switched on or off.

  • A production cluster to compare with: in smarter-infrastructure, an EKS cluster on Spot node groups with the Cluster Autoscaler, Traefik and cert-manager for ingress, and Calico network policies with default-deny ingress.

Smarter also manages Kubernetes itself, which is a good example of an application that talks to the cluster’s API. Deploying an LLMClient creates its own ingress, host name and certificate, and serving a self-hosted model with LLMHost renders a PersistentVolumeClaim, a Deployment, a Service and an Ingress, and creates an EKS node group of GPU nodes on demand.

What the course covers. Why Kubernetes, and what Compose cannot do; the control plane, the nodes and the reconciliation loop; Pods, Deployments, ReplicaSets, Services and Jobs; Helm charts, templates, values, releases and subcharts, through Smarter’s chart; ConfigMaps, Secrets, service accounts and RBAC; requests, limits, probes and the Horizontal Pod Autoscaler; Services, Ingress, cluster DNS and certificates; network policies, and blocking the cloud metadata address; operations: logs, events, upgrades, backups and monitoring; applications that manage the cluster themselves; and managed clusters, node groups and their cost.

Look at.

Each course plan lists every file that it uses under Smarter in the Source Code.

Read. Smarter on Kubernetes, Kubernetes, Cloud Infrastructure, ADR 003 on Kubernetes.

Exercises.

  • Install Smarter with the Helm chart on a local cluster, such as kind, minikube or the one built into Docker Desktop, and compare each Kubernetes object with its Docker Compose service.

  • Load test the web service, and watch the autoscaler add pods.

  • Write a network policy that stops a pod from reaching the cloud provider’s metadata address, and compare it with the Calico policies in smarter-infrastructure.

  • Read the LLMHost renderer, and list every Kubernetes object that a self-hosted model would create, and why each one is needed.

What it costs. A class that runs Smarter on a managed Kubernetes cluster in the cloud should expect a baseline infrastructure cost of around $350 USD a month, for the cluster, its nodes, the load balancer, the database and the other services that Smarter needs. This grows very slightly with each student: the variable cloud infrastructure cost per student is significantly less than a dollar a month. Model provider charges are separate, and are controlled with Budgets. A local cluster, such as kind or minikube, costs nothing and is enough for most of the exercises above.

Course plans. Three complete, ready-to-teach plans, each with a schedule, graded assignments and rubrics, two capstone projects, recommended texts, further study and technical prerequisites. See Course Formats and Grading for what the formats have in common.

See the Smarter Instructor Guide to install Smarter and set up a class.