Writing Secure Software: Classroom Bootcamp

An eight-hour classroom bootcamp on securing web applications and AI systems, using Smarter as the case study. Students attack an AI application with prompt injection, defend it with deterministic controls, map its defenses to the OWASP Top 10 lists, and study how it handles secrets and outbound requests. Smarter assumes that a model will eventually try to do something it should not, and defends with deterministic controls rather than instructions: guardrails on both the prompt and the reply, a single hardened function for every outbound request to a user-supplied URL, credentials kept in encrypted Secrets that the model never sees, and parameterized queries and tool allow lists for least privilege. Every defense is in code that students can read and test. See Course Formats and Grading for the standard bootcamp day, the grading scale and the course policies, and Writing Secure Software for the Internet for an overview of the subject.

At a Glance

Students

Developers and security students who want to understand both classic web security and the new attacks on AI systems.

Deployment

A shared Smarter instance run by the instructor for the attack and defense labs, with an account per team. Students who want to read and run the code clone the repository.

Cost

Expect less than $3 USD of model charges per student, capped by a per-student Budget.

Warning

Students attack only the class’s own Smarter instance, within the rules set by the instructor. Any real vulnerability found in Smarter is reported privately, as described in the project’s SECURITY.md, never in a public issue or an assignment.

Learning Objectives

By the end of the bootcamp, students can:

  1. Explain the OWASP Top 10 and the OWASP Top 10 for LLM Applications, with an example of each category.

  2. Carry out and explain direct and indirect prompt injection against an AI application.

  3. Explain why deterministic controls (guardrails, allow lists, least privilege) are stronger than instructions to a model.

  4. Explain how an application keeps credentials out of prompts, code and logs.

  5. Explain server-side request forgery, and how an application that fetches URLs defends against it.

Technical Prerequisites

  • Required: structured programming in Python or JavaScript, enough to read a function and follow its logic.

  • Required: HTTP: methods, headers, cookies, status codes, and the browser’s developer tools.

  • Required: the command line, curl, Git and a GitHub account.

  • Required: basic generative AI concepts: system and user prompts, and tool calls.

  • Helpful: Linux, Docker, DNS and IP addressing, and YAML.

Schedule

The files that the lectures refer to are listed in full under Smarter in the Source Code.

Block

Min

Content

1

45

Lecture 1: The threat landscape. Attackers, assets and attack surfaces. The OWASP Top 10 and the OWASP Top 10 for LLM Applications. Why AI applications add new attack surfaces: the prompt, retrieved content and tools.

2

45

Lab 1: Red team. Each team builds an LLMClient that guards a secret phrase with system prompt instructions only, then attacks another team’s. The journal records every attempt. Produces Assignment 1.

3

45

Lecture 2: Prompt injection. Direct and indirect injection, jailbreaks, data exfiltration through links and images, and why no instruction can make a model reliably ignore instructions. Guardrails as deterministic controls, and their limits.

4

45

Lab 2: Blue team. Teams add the built-in guardrails prompt_injection_llm_judge_input, secrets_leak_output and script_injection_output to their LLMClient, and the attackers try again. Produces Assignment 2.

5

45

Lecture 3: Secrets. How Smarter stores provider and plugin credentials as encrypted Secrets, why an API plugin’s credentials never appear in a prompt, and the .env file, detect-private-key and detect-aws-credentials hooks.

6

45

Lecture 4: Outbound requests and SSRF. Server-side request forgery, cloud metadata addresses, DNS rebinding, and a reading of smarter/smarter/apps/plugin/plugin/safe_http.py: public https only, fresh DNS resolution, and why that check must never be cached.

7

45

Lab 3: Map the defenses. In groups, map each Smarter defense to a category of the two OWASP lists, with the file or document that implements it, and find the gaps. Produces Assignment 3.

8

45

Lecture 5: Secure development. Static analysis with bandit, dependency scanning, pre-commit hooks, code review, and Smarter’s ADR 016 on security.

9

45

Lab 4: Read the code like an attacker. Run bandit on one Smarter app, triage its findings, and decide which are real. Ungraded.

10

45

Lecture 6: Disclosure and response. Responsible disclosure, security policies, CVEs, how open source projects handle reports, and incident response.

11

30

Capstone kickoff. Students choose a capstone and scope it with the instructor.

Smarter in the Source Code

Every lecture, lab and assignment in this course rests on the files below. Every reference is to the main branch of github.com/smarter-sh/smarter, or of smarter-infrastructure where named, so that students can open each one during class. Python paths are under smarter/smarter/.

Defending an AI Application

Defense

What Smarter does

Where to look

Guardrails on input

Built-in guardrails for prompt injection by keyword and by LLM judge, semantic jailbreak detection, code injection, and redaction of personal data and secrets before the model sees them.

prompt-injection-llm-judge-input.yaml, guardrail/data/guardrails/

Guardrails on output

The model’s reply is checked too, because the model is untrusted: leaked secrets and personal data, script injection, invented citations and regulated advice.

secrets-leak-output.yaml, script-injection-output.yaml

Detectors

Deterministic detectors for credit cards (with the Luhn check), US social security numbers, email addresses, phone numbers, IP addresses, IBANs, AWS access keys, private keys, API keys, JWTs and password assignments.

guardrail/services/strategies/detectors.py

The pipeline

Guardrails run in order, at each stage, and every intervention is recorded with its rationale. Smarter Guardrail.

guardrail/services/pipeline.py, guardrail/models/guardrail.py

Least privilege for tools

A SQL Plugin runs only a fixed, parameterized query, and an MCP client offers only the tools on its allow list, such as the read-only tools of GitHub’s server.

stackademy-plugin-sql.yaml, mcpclients/github.yaml

Secrets and Outbound Requests

Defense

What Smarter does

Where to look

Secrets at rest

Encrypted with Fernet, with a key that is itself a setting held as a Pydantic SecretStr, so it is never printed.

secret/models.py, common/conf/settings.py

Credentials out of prompts

An API Plugin reaches its API through a Connection whose key is a Secret, and Smarter adds it to the request, so the model never sees it. A Proxy keeps provider keys on the server, and callers use revocable Smarter keys instead.

connection/models/api_connection.py, apps/proxy/

Server-side request forgery

One hardened function for every request to a user-supplied URL: https only, port 443, every resolved address public, no credentials in URLs, every redirect re-validated, bounded responses and timeouts. Its docstring states the remaining DNS rebinding window honestly.

plugin/plugin/safe_http.py

MCP servers

The same checks on every request to an MCP server, and no stdio transport, which would run a command on the Smarter server.

mcpclient/connection.py

Never cache a security check

The rule that validate_public_url() must resolve DNS fresh before every request, and why caching it would widen the rebinding window to the cache lifetime.

smarter-development/SKILL.md

Classic Web Security

Defense

What Smarter does

Where to look

Authentication

API keys of which only a hash is stored, compared in constant time, and refused once deactivated.

lib/drf/models.py, lib/drf/token_authentication.py

Object-level authorization

Every resource belongs to an account and a user, and querysets are filtered by owner.

account/models/metadata_with_ownership.py

Probing and scanning

Middleware that blocks requests for sensitive files and common attack paths, and throttles unauthenticated clients that generate many 404s.

lib/django/middleware/sensitive_files.py, lib/django/middleware/excessive_404.py

CSRF and CORS

Django’s CSRF protection for the console, and CORS for an embedded chat limited to the origins listed in its LLMClient’s manifest.

lib/django/middleware/csrf.py, llmclient/cors.py

Infrastructure

A non-root container with read-only application files, Calico network policies with default-deny ingress, and a private network for the cluster.

Dockerfile, smarter-infrastructure: network_policy.tf

Secure Development

Practice

What Smarter does

Where to look

Static analysis and secret scanning

bandit at medium severity and above, detect-private-key and detect-aws-credentials on every commit. ADR 016.

.pre-commit-config.yaml

Dependencies

Dependabot updates, and image scanning in Amazon ECR.

.github/dependabot.yml, build.yml

Tests that cannot touch production

The development containers hold live cloud credentials, so every cloud call in tests is faked, and tests that need real infrastructure are tagged and skipped.

smarter-infrastructure-safety/SKILL.md

Disclosure

A private address for vulnerability reports, and a policy that only the latest version is supported. Security FAQ.

SECURITY.md

Graded Assignments

Assignment 1: Red Team Log

Teams record their attacks on another team’s prompt-only LLMClient: at least ten attempts with different techniques, and which succeeded.

Turn in: the attack log, taken from the journal, with each technique named, and a half-page summary.

Criterion

Pts

Exemplary

Developing

Variety

6

Ten or more attempts across several techniques (role play, encoding, translation, instruction override).

Repeats of one technique.

Evidence

5

From the journal.

Paraphrased.

Classification

5

Each attempt mapped to an OWASP LLM category.

Unclassified.

Summary

4

Explains why the successful attacks worked.

Lists results only.

Assignment 2: Blue Team Report

Teams add guardrails to their LLMClient, are attacked again, and report the before-and-after results, including legitimate prompts that were wrongly blocked.

Turn in: the guardrail manifests, the second attack log, and a one-page report.

Criterion

Pts

Exemplary

Developing

Controls

6

Guardrails chosen for the observed attacks, and configured correctly.

Guardrails added at random.

Effectiveness

5

Most attacks now fail, shown by the log.

No measured improvement.

False positives

5

Legitimate prompts tested; wrongful blocks reported.

Not tested.

Honesty

4

What still gets through is reported.

Claims complete protection.

Assignment 3: Defense Map

Groups map Smarter’s defenses to the OWASP Top 10 and the OWASP Top 10 for LLM Applications, citing the file or document that implements each, and identify three categories where defense is weak or depends on the operator.

Turn in: a table and a one-page gap analysis.

Criterion

Pts

Exemplary

Developing

Coverage

5

Every category of both lists considered.

Several skipped.

Accuracy

7

Each defense correctly described and cited.

Defenses assumed, not found.

Gap analysis

6

Three well-reasoned gaps with suggested mitigations.

Vague gaps.

Clarity

2

Well organized.

Hard to follow.

Capstone Projects

Students choose one. Capstones are due one week after the bootcamp.

Capstone A: A Threat Model of an AI Application

Learning objective. Produce a structured threat model of a real AI application, and verify its most important mitigations by testing.

Students choose a built-in LLMClient with tools (for example research_analyst or data_analyst) and threat model it with STRIDE and the OWASP LLM list: a data flow diagram with trust boundaries, threats, existing mitigations, residual risk, and tests of the three most important mitigations on the class instance.

What students turn in:

  • A threat model document (four to six pages) with a data flow diagram.

  • Evidence of the three mitigation tests, from the journal.

  • A prioritized list of recommendations.

How it is graded (100 points):

Criterion

Pts

Exemplary

Developing

Data flow and trust boundaries

20

Accurate, with every boundary marked.

Boundaries missing.

Threat identification

25

Systematic, covering classic and AI-specific threats.

A few obvious threats.

Mitigations

20

Existing defenses correctly identified in code or docs.

Assumed.

Verification

20

Three mitigations tested, with evidence.

Untested.

Recommendations

15

Prioritized by risk, realistic.

Unprioritized.

Capstone B: A Hardened Assistant

Learning objective. Build an AI application with defense in depth, and prove its security properties with an adversarial test suite.

Students build an LLMClient with at least one tool plugin and access to sensitive information (a fictional employee directory, for example), then harden it with least-privilege configuration, guardrails on input and output, a budget, and a tight system prompt. They write and run an adversarial test suite of at least twenty-five attacks and ten legitimate requests.

What students turn in:

  • The manifests, with comments explaining each security decision.

  • The test suite and its results from the journal, with attack success rate and false positive rate.

  • A two-page security design note.

How it is graded (100 points):

Criterion

Pts

Exemplary

Developing

Defense in depth

25

Several independent layers, each justified.

Relies on the system prompt.

Least privilege

15

Tools can do only what the job needs.

Broad access.

Test suite

25

Twenty-five varied attacks, including indirect injection, plus legitimate requests.

Few or repetitive attacks.

Results

20

Rates measured and honestly reported.

Cherry-picked results.

Design note

15

Explains residual risk.

Claims the system is secure.

Further Study