Writing Secure Software: Two-Week Online Course

A two-week online course on securing web applications and AI systems, using Smarter as the case study. Smarter assumes that a model will eventually try to do something it should not, and defends with deterministic controls rather than instructions: guardrails on both the prompt and the reply, a single hardened function for every outbound request to a user-supplied URL, credentials kept in encrypted Secrets that the model never sees, and parameterized queries and tool allow lists for least privilege. Every defense is in code that students can read and test. There are six 30-minute recorded lectures and a 90-minute live session each week. Week 1 covers attacks on AI applications and their defenses, and week 2 covers secrets, outbound requests and secure development. See Course Formats and Grading for the format, the grading scale and the course policies, and Writing Secure Software for the Internet for an overview of the subject.

At a Glance

Students

Developers and security students.

Deployment

A shared Smarter instance run by the instructor for attack and defense exercises, with an account per team. Students clone the repository to read and scan the code.

Workload

3 hours of instruction and 4 to 6 hours of assignments each week.

Cost

Expect less than $5 USD of model charges per student, capped by a per-student Budget.

Warning

Students attack only the class’s own Smarter instance. Any real vulnerability in Smarter is reported privately, as described in the project’s SECURITY.md.

Learning Objectives

By the end of the course, students can:

  1. Explain both OWASP Top 10 lists with examples.

  2. Attack an AI application with direct and indirect prompt injection, and defend it with deterministic controls.

  3. Explain how an application protects credentials throughout their life.

  4. Explain and defend against server-side request forgery.

  5. Use static analysis and code review to find vulnerabilities, and handle them responsibly.

Technical Prerequisites

  • Required: structured programming in Python or JavaScript.

  • Required: HTTP, cookies and the browser’s developer tools.

  • Required: the command line, curl, Git and a GitHub account.

  • Required: basic generative AI concepts: system and user prompts, and tool calls.

  • Helpful: Linux, Docker, DNS and IP addressing, YAML, and Python virtual environments for running bandit.

Schedule

The files that the lectures refer to are listed in full under Smarter in the Source Code.

Week

Item

Content

1

Lecture 1 (30 min)

The threat landscape. Attackers, assets, attack surfaces, and both OWASP Top 10 lists.

1

Lecture 2 (30 min)

Prompt injection. Direct and indirect injection, jailbreaks and data exfiltration.

1

Lecture 3 (30 min)

Deterministic controls. Guardrails, allow lists, least privilege for tools, and measuring false positives.

1

Live session 1 (90 min)

Red team exercise: teams attack each other’s prompt-only LLMClients, live. Start Assignment 1.

2

Lecture 4 (30 min)

Secrets. Encrypted Secrets, credentials kept out of prompts, and the detect-private-key and detect-aws-credentials hooks.

2

Lecture 5 (30 min)

SSRF and outbound requests. Metadata addresses, DNS rebinding, and smarter/smarter/apps/plugin/plugin/safe_http.py.

2

Lecture 6 (30 min)

Secure development and disclosure. bandit, dependency scanning, code review, ADR 016, responsible disclosure and incident response.

2

Live session 2 (90 min)

Capstone presentations (about 5 minutes each, with 2 minutes of questions).

Assignment 1 is due at the end of week 1, Assignment 2 in the middle of week 2, and Assignment 3 and the capstone at the start of Live session 2.

Smarter in the Source Code

Every lecture, lab and assignment in this course rests on the files below. Every reference is to the main branch of github.com/smarter-sh/smarter, or of smarter-infrastructure where named, so that students can open each one during class. Python paths are under smarter/smarter/.

Defending an AI Application

Defense

What Smarter does

Where to look

Guardrails on input

Built-in guardrails for prompt injection by keyword and by LLM judge, semantic jailbreak detection, code injection, and redaction of personal data and secrets before the model sees them.

prompt-injection-llm-judge-input.yaml, guardrail/data/guardrails/

Guardrails on output

The model’s reply is checked too, because the model is untrusted: leaked secrets and personal data, script injection, invented citations and regulated advice.

secrets-leak-output.yaml, script-injection-output.yaml

Detectors

Deterministic detectors for credit cards (with the Luhn check), US social security numbers, email addresses, phone numbers, IP addresses, IBANs, AWS access keys, private keys, API keys, JWTs and password assignments.

guardrail/services/strategies/detectors.py

The pipeline

Guardrails run in order, at each stage, and every intervention is recorded with its rationale. Smarter Guardrail.

guardrail/services/pipeline.py, guardrail/models/guardrail.py

Least privilege for tools

A SQL Plugin runs only a fixed, parameterized query, and an MCP client offers only the tools on its allow list, such as the read-only tools of GitHub’s server.

stackademy-plugin-sql.yaml, mcpclients/github.yaml

Secrets and Outbound Requests

Defense

What Smarter does

Where to look

Secrets at rest

Encrypted with Fernet, with a key that is itself a setting held as a Pydantic SecretStr, so it is never printed.

secret/models.py, common/conf/settings.py

Credentials out of prompts

An API Plugin reaches its API through a Connection whose key is a Secret, and Smarter adds it to the request, so the model never sees it. A Proxy keeps provider keys on the server, and callers use revocable Smarter keys instead.

connection/models/api_connection.py, apps/proxy/

Server-side request forgery

One hardened function for every request to a user-supplied URL: https only, port 443, every resolved address public, no credentials in URLs, every redirect re-validated, bounded responses and timeouts. Its docstring states the remaining DNS rebinding window honestly.

plugin/plugin/safe_http.py

MCP servers

The same checks on every request to an MCP server, and no stdio transport, which would run a command on the Smarter server.

mcpclient/connection.py

Never cache a security check

The rule that validate_public_url() must resolve DNS fresh before every request, and why caching it would widen the rebinding window to the cache lifetime.

smarter-development/SKILL.md

Classic Web Security

Defense

What Smarter does

Where to look

Authentication

API keys of which only a hash is stored, compared in constant time, and refused once deactivated.

lib/drf/models.py, lib/drf/token_authentication.py

Object-level authorization

Every resource belongs to an account and a user, and querysets are filtered by owner.

account/models/metadata_with_ownership.py

Probing and scanning

Middleware that blocks requests for sensitive files and common attack paths, and throttles unauthenticated clients that generate many 404s.

lib/django/middleware/sensitive_files.py, lib/django/middleware/excessive_404.py

CSRF and CORS

Django’s CSRF protection for the console, and CORS for an embedded chat limited to the origins listed in its LLMClient’s manifest.

lib/django/middleware/csrf.py, llmclient/cors.py

Infrastructure

A non-root container with read-only application files, Calico network policies with default-deny ingress, and a private network for the cluster.

Dockerfile, smarter-infrastructure: network_policy.tf

Secure Development

Practice

What Smarter does

Where to look

Static analysis and secret scanning

bandit at medium severity and above, detect-private-key and detect-aws-credentials on every commit. ADR 016.

.pre-commit-config.yaml

Dependencies

Dependabot updates, and image scanning in Amazon ECR.

.github/dependabot.yml, build.yml

Tests that cannot touch production

The development containers hold live cloud credentials, so every cloud call in tests is faked, and tests that need real infrastructure are tagged and skipped.

smarter-infrastructure-safety/SKILL.md

Disclosure

A private address for vulnerability reports, and a policy that only the latest version is supported. Security FAQ.

SECURITY.md

Graded Assignments

Assignment 1: Red Team, Blue Team

Teams submit the log of their attacks from Live session 1, then add guardrails to their own LLMClient, invite the attackers back, and report the before-and-after results.

Turn in: both attack logs from the journal, the guardrail manifests and a one-page report.

Criterion

Pts

Exemplary

Developing

Attacks

5

Ten or more, varied and classified by OWASP LLM category.

Repetitive or unclassified.

Defenses

6

Guardrails chosen for the observed attacks.

Added at random.

Measurement

5

Before and after, with false positives tested.

No measurement.

Honesty

4

Residual risk reported.

Claims complete protection.

Assignment 2: SSRF Analysis

Students explain, with references to the code, how safe_http.py stops a plugin from reaching private and metadata addresses, then list five SSRF techniques (for example redirects, DNS rebinding, IPv6 and decimal IP notation) and say whether each would be stopped and where.

Turn in: a two-page analysis with code references.

Criterion

Pts

Exemplary

Developing

Code understanding

7

Each check explained with line references.

General description.

Techniques

6

Five techniques, each correctly assessed.

Fewer, or incorrect assessments.

Reasoning

5

Explains why DNS must be resolved fresh before every request.

Not explained.

Responsible handling

2

Any suspected weakness reported privately.

Published.

Assignment 3: Static Analysis Triage

Students run bandit on one Smarter app, triage every finding as a true positive, false positive or accepted risk, and justify each decision.

Turn in: the bandit output and a triage table.

Criterion

Pts

Exemplary

Developing

Complete triage

6

Every finding classified.

Some skipped.

Justification

8

Each decision supported by reading the code.

Decisions without reasons.

Insight

4

Notes what static analysis cannot find.

Treats a clean scan as proof of security.

Clarity

2

Well organized.

Disorganized.

Capstone Projects

Students choose one, and present it in Live session 2.

Capstone A: A Threat Model of an AI Application

Learning objective. Produce a structured threat model of a real AI application, and verify its key mitigations by testing.

Students threat model a built-in LLMClient with tools, using STRIDE and the OWASP LLM list: data flow diagram, trust boundaries, threats, mitigations found in the code, residual risk, and tests of the three most important mitigations.

What students turn in:

  • A threat model document with a data flow diagram.

  • Evidence of mitigation tests.

  • Prioritized recommendations, and a five-minute presentation.

How it is graded (100 points):

Criterion

Pts

Exemplary

Developing

Data flow and boundaries

20

Accurate and complete.

Boundaries missing.

Threats

25

Systematic, classic and AI-specific.

A few obvious ones.

Mitigations in code

20

Found and cited.

Assumed.

Verification

20

Three tested, with evidence.

Untested.

Recommendations and presentation

15

Prioritized; clear presentation.

Unprioritized.

Capstone B: A Hardened Assistant with an Adversarial Test Suite

Learning objective. Build an AI application with defense in depth, and prove its security properties with tests.

Students build an LLMClient with a tool and access to fictional sensitive data, harden it, and run an adversarial suite of at least thirty attacks (including indirect injection through a web page or document it reads) and fifteen legitimate requests. Classmates are invited to attack it during the week before presentations.

What students turn in:

  • Commented manifests.

  • The test suite, results, attack success rate and false positive rate.

  • A two-page design note on residual risk, and a five-minute presentation.

How it is graded (100 points):

Criterion

Pts

Exemplary

Developing

Defense in depth

25

Independent layers, each justified.

System prompt only.

Least privilege

15

Tools limited to the job.

Broad access.

Test suite

25

Thirty varied attacks, including indirect, plus legitimate requests.

Repetitive.

Results and honesty

20

Rates measured; classmates’ successful attacks reported.

Cherry-picked.

Design note and presentation

15

Residual risk explained.

Claims to be secure.

Further Study