Software Quality: Two-Week Online Course

A two-week online course on software quality and quality assurance, taught on Smarter’s complete quality process, with six 30-minute recorded lectures and a 90-minute live session each week. Week 1 covers testing and coverage, and week 2 covers automated checks, CI and releases. See Course Formats and Grading for the format, the grading scale and the course policies, and Software Quality and Quality Assurance for an overview of the subject.

At a Glance

Students

Software developers and QA engineers who write Python or JavaScript.

Deployment

Each student runs Smarter from source, installed before Live session 1, with pre-commit hooks set up and a fork on GitHub.

Workload

3 hours of instruction and 4 to 6 hours of assignments each week.

Cost

Free on GitHub Actions for public forks, plus a few cents of model calls when running tests.

Learning Objectives

By the end of the course, students can:

  1. Explain the layers of a quality process, and what each catches.

  2. Run tests and coverage, and identify the riskiest untested code.

  3. Write meaningful unit tests with fixtures and test doubles.

  4. Configure and explain pre-commit hooks and CI workflows.

  5. Explain conventional commits, semantic versioning and automated releases.

Technical Prerequisites

  • Required: Python or JavaScript, enough to write a unit test.

  • Required: Git and GitHub, including pull requests.

  • Required: the command line and Docker basics.

  • Required before Live session 1: Smarter installed from source, and pre-commit run --all-files completed once.

  • Helpful: a unit testing framework, and YAML.

Schedule

Week

Item

Content

1

Lecture 1 (30 min)

What quality means. Defects and their cost, the test pyramid, and the layers of Smarter’s process. ADR 028.

1

Lecture 2 (30 min)

Good unit tests. Arrange, act, assert; fixtures; cleanup; SmarterTestBase; ADR 013.

1

Lecture 3 (30 min)

Test doubles. Mocks, fakes and stubs, patching external services, and the in-memory infrastructure provider.

1

Live session 1 (90 min)

Live coverage run on one app, then pairs write a first test together. Start Assignment 1.

2

Lecture 4 (30 min)

Automated checks. Every hook in .pre-commit-config.yaml: what it catches and what it cannot.

2

Lecture 5 (30 min)

CI. The workflows in .github/workflows/, and CI/CD.

2

Lecture 6 (30 min)

Commits and releases. Conventional commits, commitlint, semantic versioning and semantic-release (Semantic Versioning Guide).

2

Live session 2 (90 min)

Capstone presentations (about 5 minutes each, with 2 minutes of questions).

Assignment 1 is due at the end of week 1, Assignment 2 in the middle of week 2, and Assignment 3 and the capstone at the start of Live session 2.

Graded Assignments

Assignment 1: Coverage Analysis and First Tests

Students run one app’s coverage, identify the three riskiest gaps, and write tests for one of them.

Turn in: a one-page analysis, and a pull request against their fork with the tests.

Criterion

Pts

Exemplary

Developing

Gap analysis

6

Riskiest code, with reasons.

Percentages only.

Tests

8

Behavior and error paths asserted.

Execute code without asserting outcomes.

Conventions

4

Fixtures, cleanup, mocks.

Leaves data, or real calls.

Coverage gain

2

Shown.

Not shown.

Assignment 2: Mocks and Fakes

Students write tests for code that calls an external service (an LLM provider, DNS, or HTTP), once with unittest.mock and once with a fake, and compare the two approaches.

Turn in: a pull request against their fork and a half-page comparison.

Criterion

Pts

Exemplary

Developing

No real calls

6

Provably isolated from the network.

A real call slips through.

Mock test

4

Patches at the right place; asserts the call and the outcome.

Patches the wrong name.

Fake test

4

Uses or extends a fake correctly.

Not attempted.

Comparison

6

Weighs brittleness and realism.

Superficial.

Assignment 3: Bug Hunt Report

Students introduce five deliberate defects of different kinds, one at a time, and record which layer catches each, including CI on their fork.

Turn in: a table with evidence, and a half-page conclusion.

Criterion

Pts

Exemplary

Developing

Variety

5

Five kinds.

Similar.

Evidence

6

Hook output, failing tests, CI runs.

Claims.

Analysis

6

Why each was caught or missed.

Results only.

Improvement

3

A new check proposed.

None.

Capstone Projects

Students choose one, and present it in Live session 2.

Capstone A: Bring a Module to 95%

Learning objective. Raise real production code’s coverage with tests that catch real regressions.

Students bring a module below 90% to at least 95%, and prove the tests are meaningful by showing three deliberate bugs that they catch.

What students turn in:

  • A pull request with tests and coverage before and after.

  • A table of bugs and the tests that catch them.

  • A five-minute presentation.

How it is graded (100 points):

Criterion

Pts

Exemplary

Developing

Coverage

20

95% or more.

Below 90%.

Meaningful assertions

30

Deliberate bugs caught.

Tests pass with bugs.

Conventions

20

Project conventions followed.

Inconsistent.

Edge cases

15

Errors and boundaries.

Happy path.

Presentation

15

Clear and on time.

Overruns.

Capstone B: A Quality Gate for Your Own Project

Learning objective. Build a complete quality process for a project of your own, modeled on a working one.

Students add pre-commit hooks, tests with a coverage threshold, a CI workflow, commitlint and automated semantic releases to a project of their own.

What students turn in:

  • A public repository with the configuration, tests and workflows.

  • A passing run, a failing run and an automated release.

  • A five-minute presentation.

How it is graded (100 points):

Criterion

Pts

Exemplary

Developing

Pre-commit

20

Format, lint, spelling, security, secrets.

Formatting only.

Tests and threshold

20

Enforced in CI.

No threshold.

CI

20

Every push and pull request.

Manual.

Commits and releases

20

Demonstrated.

Not demonstrated.

Presentation

20

Clear, with comparison to Smarter.

Unclear.

Further Study