Search

Search of the indexed files of The Smarter Project’s repositories, and of its blog.

search_documentation(), search_code() and search_blog() search the documentation, the source code and the blog’s posts. The MCPServer’s public tools of the same names call them (see smarter.apps.mcpserver.protocol.tools.search). They search only the indexes that are ready, and queue the others for indexing. All three call search_files(), which ranks the files that smarter.apps.mcpserver.services.indexing.github and smarter.apps.mcpserver.services.indexing.blog stored, for a query, in the database, without embeddings:

  • The query is split into at most MAX_TERMS terms. A quoted phrase is one term, and so is an identifier such as smarter_settings.environment_platform_domain. Common English words are dropped.

  • A file is a candidate if its path or content contains any of the terms.

  • Each term is weighted by how rare it is among the files searched, its inverse document frequency, so that a rare identifier counts for more than a common word.

  • A file’s score adds up, for each term, a bonus for containing it, its number of occurrences, capped at MAX_COUNTED_OCCURRENCES so that long files do not win by length alone, and bonuses for matching the file’s path and title.

Occurrences are counted in SQL, as (LENGTH(LOWER(content)) - LENGTH(REPLACE(LOWER(content), term, ''))) / LENGTH(term), which every database that Django supports can evaluate. Only the best files are then read, to extract their matching lines.

Nothing is cached here: the public tools cache their results. The only writes are those of queuing an index that is not ready.

Note

Experimental. The MCPServer was designed and coded by Claude Code (Anthropic’s Claude Opus 5.5), with Lawrence McDaniel as co-author. It is experimental, and will be documented.

class smarter.apps.mcpserver.services.search.SearchHit(**data)[source]

Bases: BaseModel

A file that matches a search, as search_files() returns it.

The search tools format it as Markdown: a heading, its links, and its lines in a code block.

documentation_url: str | None
kind: RepositoryFileKind
lines: list[SearchLine]
model_config = {'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

path: str
repository: str
score: float
title: str
url: str
class smarter.apps.mcpserver.services.search.SearchLine(**data)[source]

Bases: BaseModel

A line of a file, with its line number, which a SearchHit shows.

model_config = {'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

number: int
text: str
class smarter.apps.mcpserver.services.search.SearchResults(**data)[source]

Bases: BaseModel

The result of a search of the repositories: the hits, and the repositories that were not searched.

A repository that is not indexed yet is queued for indexing, and named in not_indexed, so that the caller can say that it was left out, and when to try again.

hits: list[SearchHit]
model_config = {'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

not_indexed: list[str]
smarter.apps.mcpserver.services.search.query_terms(query)[source]

Split a search query into lowercase terms.

A word shorter than MIN_TERM_LENGTH, or one of STOPWORDS, is dropped, unless it is quoted. Only the first MAX_TERMS terms are kept.

Parameters:

query (str) – The query, e.g. how does "custom domain" verification work.

Return type:

list[str]

Returns:

The terms, without duplicates or common words, e.g. ["custom domain", "verification", "work"].

smarter.apps.mcpserver.services.search.search_blog(query, limit)[source]

Search the posts of The Smarter Project’s blog.

Side effects: database queries, and, if the blog is not indexed yet, a Celery task queued to index it.

Parameters:

  • query (str) – The query, e.g. caching.

  • limit (int) – The maximum number of hits.

Return type:

list[SearchHit]

Returns:

The passages of the posts, best first.

Raises:

SmarterMCPServerServiceError – If the blog is not indexed yet.

smarter.apps.mcpserver.services.search.search_code(repositories, query, limit, path=None)[source]

Search the source code and configuration of the repositories.

Side effects: see search_repositories().

Parameters:

  • repositories (Sequence[ProjectRepository]) – The repositories to search.

  • query (str) – The query, e.g. verify_custom_domain.

  • limit (int) – The maximum number of hits.

  • path (str | None) – Only files under this folder, e.g. smarter/smarter/apps/llmclient, or None.

Return type:

SearchResults

Returns:

The files, best first, with their matching lines, and the repositories that were not searched.

Raises:

SmarterMCPServerServiceError – If none of the repositories is indexed yet.

smarter.apps.mcpserver.services.search.search_documentation(repositories, query, limit)[source]

Search the documentation of the repositories: the Sphinx documentation, READMEs and other prose.

Side effects: see search_repositories().

Parameters:

  • repositories (Sequence[ProjectRepository]) – The repositories to search.

  • query (str) – The query, e.g. custom domain verification.

  • limit (int) – The maximum number of hits.

Return type:

SearchResults

Returns:

The passages, best first, and the repositories that were not searched.

Raises:

SmarterMCPServerServiceError – If none of the repositories is indexed yet.

Example usage:

results = search_documentation([get_repository("smarter")], "custom domain verification", limit=8)
results.hits[0].documentation_url
smarter.apps.mcpserver.services.search.search_files(repositories, query, kind=None, path=None, limit=10)[source]

Search the indexed files of one or more repositories.

Only indexes that are ready are searched. The files are ranked in the database, and only the best limit files are then read, to extract their matching lines.

Side effects: three database queries: the terms’ weights, the ranking, and the best files’ rows.

Parameters:

  • repositories (Sequence[ProjectRepository | ProjectBlog]) – The repositories to search, or the blog.

  • query (str) – The query.

  • kind (str | None) – Only documentation, or only code. Defaults to both.

  • path (str | None) – Only files under this folder, or with this path prefix, e.g. smarter/smarter/apps/llmclient.

  • limit (int) – The maximum number of hits.

Return type:

list[SearchHit]

Returns:

The best hits, best first. Empty if the query has no terms, or nothing matches.

Example usage:

hits = search_files([get_repository("smarter")], "custom domain verification", kind="documentation")
hits[0].documentation_url
smarter.apps.mcpserver.services.search.search_repositories(repositories, query, kind, path, limit)[source]

Search the files of one kind in the repositories that are indexed, and name those that are not.

Side effects: database queries, and Celery tasks queued for the repositories that are not indexed yet (see ready_indexes()).

Parameters:
Return type:

SearchResults

Returns:

The hits, best first, and the repositories that were not searched.

Raises:

SmarterMCPServerServiceError – If none of the repositories is indexed yet.