Prompt Engineering: Two-Week Online Course

A two-week online course in prompt engineering, for students who need no programming experience. Students build, test and brand an AI assistant by editing a YAML manifest. Everything students build is a Smarter LLMClient manifest, the same YAML that runs in production, and they start from the nineteen built-in LLMClients that ship with every installation: the one-line simple assistant, and customer_support, a complete support agent with a persona, refusals, a skill, a tool and four guardrails. The prompt engineering workbench shows every request sent to the model, every tool call and every plugin selected, beside the chat, so students see the effect of each edit at once. There are six 30-minute recorded lectures and a 90-minute live session each week. See Course Formats and Grading for the format, the grading scale and the course policies, and Prompt Engineering for an overview of the subject.

At a Glance

Students

Anyone: business, communications, education, and the humanities as well as computing.

Deployment

A shared Smarter instance run by the instructor. Students need only a web browser.

Workload

3 hours of instruction and 4 to 6 hours of assignments each week.

Cost

Expect less than $5 USD of model charges per student, capped by a per-student Budget.

Learning Objectives

By the end of the course, students can:

  1. Explain how a model turns a prompt into an answer, and what temperature and token limits do.

  2. Write system prompts with a persona, a task, constraints, an output format and refusals.

  3. Test an assistant systematically, and improve it from the evidence.

  4. Choose a model and settings for a job by comparing quality, tokens and cost.

  5. Ground an assistant with knowledge and tool plugins, and protect it with guardrails.

Technical Prerequisites

  • Required: a computer with a current web browser and a reliable connection for video calls, and experience of using a chat assistant such as ChatGPT, Claude or Gemini.

  • Required: basic YAML: key-value pairs, indentation, lists and multi-line strings. Lecture 1 reviews it, and Learn YAML in Y minutes covers it.

  • Helpful: Visual Studio Code with a YAML extension, and a GitHub account for submitting work.

  • Not needed: programming or the command line.

Schedule

The files that the lectures refer to are listed in full under Smarter in the Source Code.

Week

Item

Content

1

Lecture 1 (30 min)

How a language model reads a prompt. Tokens, next-token prediction, the context window, and system and user messages. A YAML refresher, and a tour of the web console and the prompt engineering workbench.

1

Lecture 2 (30 min)

Anatomy of a good system prompt. Persona, audience, task, constraints, output format and refusals. Examples and delimiters. A line-by-line reading of the built-in customer_support LLMClient’s system prompt.

1

Lecture 3 (30 min)

Testing an assistant. Writing test prompts with expected behavior, including edge cases and adversarial ones. Keeping a test sheet, and changing one thing at a time.

1

Live session 1 (90 min)

Live build of an assistant from the simple LLMClient, with questions. In pairs, students critique each other’s system prompts with the checklist from Lecture 2. Start Assignment 1 together.

2

Lecture 4 (30 min)

Settings, models and cost. defaultTemperature, defaultMaxTokens, provider and defaultModel. Comparing models fairly. How Smarter records the tokens and cost of every request.

2

Lecture 5 (30 min)

Knowledge and tools. Grounding with Static and Skill plugins, and acting with Websearch and API plugins. Smarter offers a plugin only when the prompt matches one of its searchTerms, and the model then decides, from the plugin’s description, whether to call it.

2

Lecture 6 (30 min)

When prompts are not enough. Prompt injection and jailbreaks. Why a Guardrail is enforced while a system prompt is only a request. Branding with styles, and deploying an assistant.

2

Live session 2 (90 min)

Capstone presentations (about 5 minutes each, with 2 minutes of questions), then a class discussion of what worked across projects.

Assignments 1 and 2 are due before Live session 2. Assignment 3 and the capstone are due at the start of Live session 2.

Smarter in the Source Code

Every lecture, lab and assignment in this course works with the manifests below. They are the built-in examples that ship with every Smarter installation, so students can open them in the web console, or read them on the main branch of github.com/smarter-sh/smarter. No programming is needed to read them. The Python files are listed for instructors and curious students. Python paths are under smarter/smarter/.

Prompts and Settings

Topic

What Smarter does

Where to look

A minimal assistant

The simple LLMClient: a one-line system prompt, a small model, temperature 1.0 and a 4,096-token limit. Every setting a prompt engineer changes sits under spec.config.prompt.

llmclient-simple.yaml

A production-quality system prompt

The customer_support LLMClient: a persona (Sam), an audience, a narrow job, an output format, refusals, a skill for on-brand replies, a plugin that reads web pages, and four input guardrails.

llmclient-customer-support.yaml

Seventeen more examples

Built-in LLMClients for weather, mathematics, dates, data analysis, research, ticket triage, workplace writing, programming and others, each a worked example of a different prompting job.

llmclient/data/llm-clients/

Every setting, defined

The Pydantic model that defines and validates provider, defaultModel, defaultSystemRole, defaultTemperature, defaultMaxTokens and the app... settings, each with a description. A misspelled key is rejected, with an error that says why.

llmclient/manifest/models/llmclient/spec.py

Branding

spec.config.styles: logo, background, avatars, fonts and colors for each part of the chat, validated as plain CSS. Styling Your LLMClient’s Chat.

llmclient/manifest/models/llmclient/styles.py

Knowledge and Tools

Topic

What Smarter does

Where to look

Two decisions, not one

A plugin is offered to the model only when Smarter selects it, because the prompt contains one of its searchTerms exactly, with a typo, or as a synonym, or because its selector directive is ALWAYS. Only then does the model decide, from the plugin’s description, whether to call it. Natural Language Processing (NLP).

plugin/plugin/base.py (selected()), plugin/nlp.py

Static knowledge

A Static Plugin returns fixed facts, here about a fictional Willy Wonka product, so the model quotes them instead of making them up.

everlasting-gobstopper.yaml

Skills as knowledge

SkillPlugins for brand voice, meeting notes, code review and SQL analysis, and three of Anthropic’s published skills.

skill-brand-voice.yaml, sample-plugins/

Tools that act

Websearch Plugins for research, news, local search and reading a page, and an ImageSearchPlugin that lets the model show real pictures instead of inventing urls.

websearch-research-assistant.yaml, image-search.yaml

Built-in functions

A calculator, a date calculator and a weather function that any LLMClient can list under spec.functions.

prompt/functions/

Safety, Cost and Evidence

Topic

What Smarter does

Where to look

Guardrails

Eighteen built-in guardrails, each a short manifest. off_topic_input asks a small model whether a message is in scope, and pii_redaction_input removes personal data before the model sees it.

off-topic-input.yaml, guardrail/data/guardrails/

The workbench Console

Beside the chat, the Console shows the LLMClient’s configuration, every request sent to the model, every tool call, which plugins were selected, and the server’s logs.

smarter-chat: src/components/Console/

Tokens and cost

Every request is recorded as a charge, with its tokens, provider and model, and priced per model. Cost Accounting.

account/models/charge.py, account/models/llm_prices.py

A record of every prompt

Each prompt, answer and tool call is kept, so an assignment can ask for the conversation that produced a result. Smarter Journal.

prompt/models/

A per-student allowance

A Budget that caps what each student can spend in a month. Smarter Budget.

student-monthly-allowance.yaml

Graded Assignments

Assignment 1: A Focused Assistant with a Test Sheet

Students turn the general-purpose simple LLMClient into an assistant with one narrow job, and test it with fifteen prompts: ten that it should answer and five that it should decline.

Turn in: the manifest, and the test sheet with expected behavior and results.

Criterion

Pts

Exemplary

Developing

Manifest

4

Valid, applies cleanly, unique name, sensible welcome message and example prompts.

Applies only with help, or keeps defaults from simple.

System prompt

6

Clear persona, audience, job and refusals.

Persona present, but the job or the refusals are vague.

Test sheet

6

Varied prompts, each with a checkable expected behavior written in advance.

Easy prompts only, or expected behavior written after the fact.

Reporting

4

Results are honest, and failures are explained.

Every result is a pass, including ones that are not.

Assignment 2: Revise from Evidence

Students revise their system prompt twice, each time to fix failures from the test sheet, and re-run every test after each revision. A short note explains each change and its effect.

Turn in: the three versions of the manifest (or a Git history with three commits), the updated test sheet, and a one-page change log.

Criterion

Pts

Exemplary

Developing

Targeted changes

6

Each revision addresses specific, observed failures.

Changes are general rewrites unconnected to the test results.

Regression checking

5

All tests are re-run after each revision, and regressions are noted.

Only the failing tests are re-run.

Technique

5

Uses at least two techniques from the lectures, such as examples, delimiters or an output format.

Only adds more instructions.

Change log

4

Concise and specific: what changed, why, and what happened.

Missing, or a list of changes without reasons.

Assignment 3: Model, Settings and Tools

Students run five of their test prompts at temperature 0 and 1 and on a second model, then add a plugin and a guardrail, and show each one working.

Turn in: a comparison table of quality, tokens and cost, the plugin and guardrail manifests, Console screenshots of a tool call and of a guardrail intervention, and a recommendation of one paragraph.

Criterion

Pts

Exemplary

Developing

Controlled comparison

5

One setting changed at a time, same prompts, figures from Smarter’s records.

Several settings changed at once, or estimated figures.

Plugin

5

Called when it should be, and its result is used in the answer.

Applied but never called, without explanation.

Guardrail

5

Blocks or redacts what it should, and lets normal prompts through.

Blocks normal prompts too, or never triggers.

Recommendation

5

A configuration chosen for the assistant’s job, with reasons from the data.

Missing, or unsupported by the data.

Capstone Projects

Students choose one, and present it in Live session 2.

Capstone A: An Assistant for a Real Client

Learning objective. Turn a real person’s needs into a tested, branded and governed AI assistant, and justify its design.

Students interview a real client, such as a campus office, a club, a nonprofit or a local business, and build an LLMClient for them with at least one knowledge plugin, one tool plugin, two guardrails and custom styles.

What students turn in:

  • A Git repository with every manifest, and a README.md that explains how to apply them.

  • A two-page design brief: the client, the audience, the job, the risks, and the reason for each plugin and guardrail.

  • A test sheet of at least twenty-five prompts, with results from the final version.

  • A five-minute presentation with a live demonstration.

How it is graded (100 points):

Criterion

Pts

Exemplary

Developing

Fit to the client

20

The brief reflects a real interview, and the assistant meets the client’s needs.

The client is named, but the assistant is generic.

System prompt

20

Clear, consistent and robust to ambiguous and adversarial prompts.

Works for easy prompts only.

Plugins and guardrails

25

Each is justified, configured correctly and shown working.

Present but unused, misconfigured or unexplained.

Testing

20

Varied tests, honest results, and failures addressed or explained.

Few or easy tests, or results that do not match the demonstration.

Presentation

15

Clear, on time, and shows the design decisions as well as the result.

Overruns, or shows the chat without explaining the design.

Capstone B: A Prompt Pattern Library

Learning objective. Generalize from individual prompts to reusable patterns, and demonstrate with evidence when each pattern helps.

Students build a small library of five prompt patterns, for example persona with refusals, few-shot examples, structured output, step-by-step reasoning and grounded answer with citations. For each pattern they build a before-and-after pair of LLMClients that differ only in that pattern, and measure the difference on ten inputs.

What students turn in:

  • A Git repository with the ten LLMClient manifests and the inputs.

  • For each pattern, a one-page card: what it is, when to use it, a template, the measured effect, and a known failure.

  • A five-minute presentation of the most surprising result.

How it is graded (100 points):

Criterion

Pts

Exemplary

Developing

Pattern selection

15

Five distinct patterns, each useful in practice.

Patterns overlap, or two are the same idea.

Controlled pairs

25

Each pair differs only in the pattern, and is tested on the same inputs.

Pairs differ in several ways.

Evidence

25

Measured effect on quality, tokens and cost, with scoring criteria stated.

Claims without measurements.

Pattern cards

20

Templates are reusable, and failures are honest.

Cards describe the example rather than the pattern.

Presentation

15

Clear, on time, and teaches classmates something new.

Overruns, or reads the cards aloud.

Further Study

  • Online guides: Anthropic’s prompt engineering documentation, OpenAI’s prompt engineering guide, and the community Prompt Engineering Guide from DAIR.AI, which catalogs techniques with links to the research papers behind them.

  • YouTube: Andrej Karpathy’s Intro to Large Language Models and Deep Dive into LLMs like ChatGPT on his channel, the clearest explanations of what a model actually does with a prompt. The neural networks series on 3Blue1Brown, for the mathematics, visually.

  • Podcasts: Latent Space, for practitioners building with models, and Practical AI, for a broader audience.

  • Writing: Simon Willison’s weblog, which tests new models and prompting techniques as they appear, and Ethan Mollick’s One Useful Thing, on using AI well at work and in teaching.

  • Practice: Gandalf, a game in which you prompt a model into revealing a password, is a memorable lesson in why instructions alone do not control a model.