Self-Hosting Open Models
Why Smarter. LLMHost serves freely downloadable models, such as Llama, Qwen, Gemma, Mistral and gpt-oss, on a Kubernetes cluster with vLLM, SGLang, TGI, llama.cpp or Ollama, and adds and removes the GPU or CPU nodes that they need. An LLMClient uses a self-hosted model in exactly the same way as a commercial one, so students can compare them side by side. This suits an MLOps or machine learning systems course.
Look at. smarter/smarter/apps/llmhost/.
Read. Smarter LLM Host, Smarter on Kubernetes.
Exercises.
Serve a small open model with two different inference servers, and compare their throughput and memory use.
Put a self-hosted model and a commercial model behind the same LLMClient, and compare quality, latency and cost per thousand requests, including the cost of the GPU nodes.
Discuss when an organization should host its own model: privacy, cost at scale, availability, and the work of keeping it running.
Course plans. Three complete, ready-to-teach plans, each with a schedule, graded assignments and rubrics, two capstone projects, recommended texts, further study and technical prerequisites. See Course Formats and Grading for what the formats have in common.
See the Smarter Instructor Guide to install Smarter and set up a class.