List API Caching

Every resource list in the web console, e.g. the LLMClient, Secret or Guardrail list, is a React app that asks its Django list api for one page of resources at a time (see List Pagination, Search and Sorting). Each list api is a subclass of SmarterListApiView, which declares the resource’s model and serializer, the columns that sort it, and the relations that its serializer reads. The base class does the rest: it filters the resources by ownership, joins their owners and prefetches their tags and the declared relations, so that a page costs a few queries however many resources it holds, and it caches the page.

Two caches, one request

A list is cached twice: in the browser, by the React list, and on the server, by its list api. When a user opens a list, pages it, sorts it or comes back to it:

The React list paints at once, from the browser’s cache, if it holds that page, with a small Refreshing spinner beside the view toggle. Otherwise it shows its loading skeleton.
It asks the list api for the page anyway (stale-while-revalidate), and replaces what it painted with the response, which it writes back to the browser’s cache.
The list api answers from Redis, if it holds the page for that user, and builds it from the database otherwise, in a few queries, then caches it.

So a list that the user has seen before paints without waiting for the network, and the request that refreshes it is usually a cache hit too.

The browser cache

The React side is shared by every list, in the @smarter/common package (smarter/react/packages/smarter-common):

src/lib/cache.tsx stores pages in the browser’s localStorage, which outlives the browser tab, so a new tab or a return visit paints at once. makeCacheKey() names a page by the list api’s url, the tab (owned or shared), and the page’s number, ordering and filter. Searches are never cached. Each entry holds the page’s objects and its pagination, so the pagination controls paint with it.
sessionScope() adds a hash of the session’s CSRF cookie to every key. Django rotates the CSRF token at every login, so a page cached for one user is never shown to the next user of the same browser, and the token itself is never stored.
readCache() and writeCache() treat a private window, blocked site data or a full quota as a cache miss, never an error. pruneCache() runs on each write: it removes entries older than a week, and keeps only the most recent hundred.
TabbedListView (src/components/TabbedListView), the tabs, toolbar and pagination of every list, reads the cache whenever a tab’s page, ordering or filter changes, and marks the tab isRefreshing while it shows a cached page. Its Refresh button asks the api for a fresh page with ?invalidate_cache=true, which also makes the user’s cached pages on the server stale.

The server cache

The list api caches each page’s whole response in Redis, with cache_results(), in cached_listview_page(). Its key is everything that can change the page: the user, the ownership tab, the page, page size, search, ordering and filter, and the generations described below. A page that is in the cache costs no query beyond the request’s session. On a miss, build_page() builds it with the relations that the list api declared, and its can_delete column asks once per page which rows the user owns, rather than once per row.

Staying correct

A cached page is never stale for long, because its cache key includes three generations, values that change whenever they are invalidated (listview_cache_generation()):

the list’s generation, which changes whenever a resource that the list shows is saved or deleted. Each app connects its list to the models it shows, in its receivers.py, with connect_listview_cache(): the resource, the rows that its serializer nests, and the rows that decide whether it can be deleted, e.g. an LLMClient’s plugins. Because receivers.py is imported by the app’s AppConfig.ready(), the Celery workers invalidate lists too;
the user’s generation of the list, which changes when the user clicks the list’s Refresh button, which sends ?invalidate_cache=true;
the generation of every list, which changes when a user profile, an account or a resource’s tags change, since those can alter what any list shows.

Invalidating a generation is a single cache delete, however many users and pages the list has, and waits for the database transaction to commit, so that a request can’t cache the old rows under the new generation. A change made with QuerySet.update() sends no signal, so code that updates a column that a list shows calls invalidate_listview_cache() afterwards, as the custom domain verification status and the MCPClient connection status do. Pages expire after SMARTER_LISTVIEW_CACHE_EXPIRATION seconds, an hour by default, which bounds how long any other such change can go unnoticed.

Adding a list

# apps/widget/views/listview/api.py
class WidgetListApiView(SmarterListApiView):
    model = Widget
    serializer_class = WidgetSerializer
    listview_cache_kind = "Widget"
    sort_fields = SORT_FIELDS
    select_related = ("provider",)
    prefetch_related = ("widgetpart_set__part",)

# apps/widget/receivers.py
connect_listview_cache("Widget", "widget.Widget", "widget.WidgetPart", "provider.Provider")

A serializer method that reads a reverse relation uses related_rows(), which reads the prefetched rows on a list page, and queries them elsewhere.

Cached pages for the list apis of the React list views in the Smarter web console.

Every resource list, e.g. the LLMClient or Secret list, asks its Django list api for one page of its resources at a time (see smarter.lib.django.pagination). SmarterListApiView is the base class of those apis. It builds the page’s whole response once, and caches it in Redis with cache_results(), per user and per page, so that the next request for the same page costs a few cache reads instead of a query and a serializer per resource.

The cached pages stay correct because their cache key includes three generations, each a value from listview_cache_generation() that changes when it is invalidated:

the list’s generation, e.g. LLMClient’s, which changes whenever a resource that the list shows is saved or deleted (see connect_listview_cache());
the user’s generation of the list, which changes when the user asks for a fresh list, with the list api’s invalidate_cache=true;
the generation of every list, LISTVIEW_CACHE_ALL, which changes whenever a change can alter what any list shows, such as a user’s profile, an account or a resource’s tags.

A new generation leaves the previous pages unread, until they expire after smarter_settings.listview_cache_expiration seconds. So invalidating a list is a single cache delete, however many users and pages it has.

# an app's receivers.py, which its AppConfig.ready() imports, so that the Celery workers
# invalidate the list too
connect_listview_cache("Guardrail", "guardrail.Guardrail", "llmclient.LLMClientGuardrails")

# the app's views/listview/api.py
class GuardrailListApiView(SmarterListApiView):
    model = Guardrail
    serializer_class = GuardrailSerializer
    listview_cache_kind = "Guardrail"

smarter.lib.django.listview.LISTVIEW_CACHE_ALL = '*'

invalidating it makes the cached pages of every list stale.

Type:: The generation of every list

class smarter.lib.django.listview.ListviewPageRequest(user)[source]

Bases: HttpRequest

The request that a cached list page is serialized with, which no http request is.

A cached page is shared by every request of the user, so it is built without one. Its serializer still needs a user, since can_delete depends on it, and a host and scheme, since a file field, e.g. a Provider’s logo, is made an absolute url with request.build_absolute_uri(). Both are the platform’s, smarter_settings.environment_url.

Parameters:: user (User) – The user whose page it is.

__init__(user)[source]

class smarter.lib.django.listview.SmarterListApiView(**kwargs)[source]

Bases: SmarterAuthenticatedNeverCachedWebView

The base class of the list apis of the web console’s React list views.

A list api returns one page of the resources that the authenticated user owns, that are shared with them, or both, as {"user": ..., "admin": ..., "objects": [...], "pagination": {...}}. The page is cached per user (see cached_listview_page()), and ?invalidate_cache=true makes the user’s cached pages of the list stale first, for the list’s Refresh button.

A subclass sets model, serializer_class and listview_cache_kind, and names the relations that its serializer reads in select_related and prefetch_related, so that a page costs a few queries, however many resources it has. The resource’s owner and tags, when it has them, are always fetched with it.

classmethod build_page(user_profile, ownership_filter, page=None, page_size=None, search='', ordering='', list_filter='')[source]

Build one page of the list api’s response, from the database.

The serializer gets a ListviewPageRequest, whose user is the user profile’s. The result is passed through JSON, so that the cache stores plain values, not the serializer’s ReturnDict, which refers to the serializer.

Parameters:

user_profile – The authenticated user’s UserProfile.
ownership_filter (str) – owned, shared or all.
page (str | None) – The page query parameter.
page_size (str | None) – The page_size query parameter.
search (str) – The search query parameter.
ordering (str) – The ordering query parameter.
list_filter (str) – The filter query parameter.

Return type:

dict[str, Any]

Returns:

{"user": ..., "admin": ..., "objects": [...], "pagination": {...}}.

filters: ClassVar[Mapping[str, Q] | None] = None: The list’s named filters, if it has any.

property formatted_class_name: str: Returns a formatted string of the class name for logging purposes.

get(request, *args, **kwargs)[source]

Return the same json as a POST, so that the list api can be browsed.

Return type:: JsonResponse

classmethod get_queryset(user, ownership_filter)[source]

Return the resources of the list that the user may see, with the relations that the serializer reads.

Parameters:

user (User) – The authenticated user.
ownership_filter (str) – owned, shared or all.

Return type:

QuerySet

Returns:

The resources, newest first.

listview_cache_kind: ClassVar[str]: The list’s name in the cache, as connect_listview_cache() invalidates it, e.g. LLMClient.

model: ClassVar[type[Model]]: The resource, whose manager objects is an OwnershipManager, e.g. a MetaDataWithOwnershipModel.

post(request, *args, **kwargs)[source]

Return one page of the list’s resources, which the user owns, are shared with them, or both.

Parameters:

request (HttpRequest) – The request. Its query parameters are page, page_size, search, ordering, filter and invalidate_cache, all optional.
kwargs – ownership_filter, from the url: owned, shared or all (the default).

Return type:

JsonResponse

Returns:

The page, or a 400 response for an unknown ownership filter.

prefetch_related: ClassVar[tuple[str | Prefetch, ...]] = (): The reverse and many-to-many relations that the serializer reads, besides the tags.

search_fields: ClassVar[Sequence[str]] = ('name', 'description'): The fields that a search matches.

select_related: ClassVar[tuple[str, ...]] = (): The forward relations that the serializer reads, besides the owner’s.

serializer_class: ClassVar[type[BaseSerializer]]: The serializer of the list’s resources.

sort_fields: ClassVar[Mapping[str, str]] = mappingproxy({'name': 'name', 'description': 'description', 'createdAt': 'created_at', 'updatedAt': 'updated_at'})

The columns that the list may be sorted by.

See paginate_queryset().

classmethod view_path()[source]

Return the class’s dotted path, by which cached_listview_page() imports it.

Return type:: str

smarter.lib.django.listview.cached_listview_page(view, kind_generation, user_generation, all_generation, user_profile_id, ownership_filter, page, page_size, search, ordering, list_filter)[source]

Return one page of a list api’s response, from the cache when it holds it.

Its parameters are the cache key: everything that can change the page. The generations are not read, but they make the page stale when the list changes (see listview_cache_generation()).

Parameters:

view (str) – The list api’s class, as a dotted path, whose SmarterListApiView.build_page() builds the page.
kind_generation (int) – The list’s generation.
user_generation (int) – The user’s generation of the list.
all_generation (int) – The generation of every list.
user_profile_id (int) – The user profile whose page it is.
ownership_filter (str) – owned, shared or all.
page (str | None) – The page query parameter, as it arrived.
page_size (str | None) – The page_size query parameter, as it arrived.
search (str) – The search query parameter.
ordering (str) – The ordering query parameter.
list_filter (str) – The filter query parameter.

Return type:

dict[str, Any]

Returns:

The list api’s response, as JSON-compatible values.

smarter.lib.django.listview.connect_listview_cache(kind, *model_labels)[source]

Invalidate a list’s cached pages whenever one of these models is saved or deleted.

Call it from an app’s receivers.py, which its AppConfig.ready() imports, so that every process connects it, including the Celery workers, which save resources too. List every model whose rows the list’s serializer shows: the resource, and the related models that it nests, e.g. an LLMClient’s plugins. A many-to-many model is invalidated by m2m_changed too, since add() and remove() save no row through save().

Each model’s proxy models and multi-table children are connected too, since they send their signals as themselves. A change made with QuerySet.update() sends no signal. Call invalidate_listview_cache() after it, if the list shows what it changes.

Parameters:

kind (str) – The list, e.g. LLMClient, or LISTVIEW_CACHE_ALL for every list.
model_labels (str) – The models, as app_label.ModelName, e.g. llmclient.LLMClientPlugin.

Return type:

None

smarter.lib.django.listview.invalidate_listview_cache(kind, user_profile_id=None)[source]

Make the cached pages of a list stale, once the current transaction commits.

Waiting for the commit keeps a request that reads the database before the change commits from caching the old rows under the new generation. Outside a transaction, it happens at once.

Parameters:

kind (str) – The list, e.g. LLMClient, or LISTVIEW_CACHE_ALL for every list.
user_profile_id (int | None) – The user profile whose pages to make stale, or None for every user’s.

Return type:

None

smarter.lib.django.listview.listview_cache_generation(kind, user_profile_id=None)[source]

Return the current generation of a list’s cached pages.

The generation is any value that changes when it is invalidated: the time of the first call after the last invalidation, in nanoseconds. It is part of the cache key of every cached page, so invalidating it makes all of them stale at once.

Parameters:

kind (str) – The list, e.g. LLMClient, or LISTVIEW_CACHE_ALL for every list.
user_profile_id (int | None) – The user profile whose pages of the list it is, or None for every user’s.

Return type:

int

Returns:

The generation.