Repository Scripts

The scripts of a repository that manage.py load_from_github loads.

A repository of Smarter manifests may also hold scripts that set up what its manifests need but can’t declare, e.g. the database that a SqlConnection connects to. The Stackademy project, https://github.com/smarter-sh/stackademy, is the reference: its scripts/create-course-database.sh creates the course database that its SqlConnection and SqlPlugin use.

The scheme

The scripts folder. A repository’s scripts are in its top-level scripts/ folder, SCRIPTS_FOLDER. load_from_github never looks for manifests there.

The configuration file. scripts/scripts.yaml, SCRIPTS_CONFIG, lists the scripts to run, and when. Only the scripts that it lists are run, so the folder may also hold the files that they read, e.g. a SQL file. Its model is RepositoryScriptsConfig:

scripts:
  - path: create-course-database.sh   # relative to scripts/
    stage: pre                        # pre: before the manifests are applied. post: after.
    description: Create the course database that the SqlConnection connects to.
    args: []                          # optional: the script's arguments
    timeout: 300                      # optional: the seconds that it may run, at most 3600

The stages. pre scripts run before any manifest is applied, so that what the manifests name exists when they are applied, e.g. a database. post scripts run after every manifest is applied, and the LLMClients deployed, e.g. to seed data through the platform’s api. Within a stage, the scripts run in the order of the configuration file.
How a script runs. As an executable, with its own shebang line (so it must be committed with its executable bit set), in the repository’s root folder, with the args of its configuration.
What a script receives. Only the environment of script_environment(), never the platform’s own: a repository’s script never sees the platform’s cloud or LLM provider credentials. The database server’s administrator is included, because creating a database and its users is what a pre script typically does.
Failures. A script that fails, i.e. exits non-zero, or runs out of time, doesn’t stop the manifests or the other scripts. load_from_github fails at the end, with the list of what failed. A configuration file that is not valid fails load_from_github before anything runs.
Opt in. Scripts run only with load_from_github --run_scripts, which only manage.py initialize_platform passes, because a pre script runs as the database server’s administrator. manage.py initialize_account doesn’t pass it, so a platform’s shared database is created once, rather than for each account.

class smarter.apps.account.repository_scripts.RepositoryScript(**data)[source]

Bases: BaseModel

One script of scripts/scripts.yaml: which file, when it runs, and how.

args: list[str]

description: str | None

model_config = {'extra': 'forbid', 'frozen': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

path: str

stage: Literal['pre', 'post']

timeout: int

classmethod validate_path(v)[source]

Validate that the path is a file name, or a relative path, inside the scripts folder.

Parameters:: v (str) – The path.
Return type:: str
Returns:: The path, normalized.
Raises:: ValueError – If it is empty, absolute, or leaves the scripts folder.

class smarter.apps.account.repository_scripts.RepositoryScriptsConfig(**data)[source]

Bases: BaseModel

A repository’s scripts/scripts.yaml: the scripts that load_from_github --run_scripts runs.

See the scheme in smarter.apps.account.repository_scripts.

for_stage(stage)[source]

Return the scripts of a stage, in the order of the configuration file.

Parameters:: stage (Literal['pre', 'post']) – SCRIPT_STAGE_PRE or SCRIPT_STAGE_POST.
Returns:: The stage’s scripts.
Return type:: list[RepositoryScript]

model_config = {'extra': 'forbid', 'frozen': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

scripts: list[RepositoryScript]

smarter.apps.account.repository_scripts.SCRIPTS_CONFIG = 'scripts.yaml': The scripts folder’s configuration file, RepositoryScriptsConfig.

smarter.apps.account.repository_scripts.SCRIPTS_FOLDER = 'scripts'

A repository’s scripts folder, at its root.

load_from_github finds no manifests in it.

smarter.apps.account.repository_scripts.SCRIPT_DEFAULT_TIMEOUT = 300: The seconds that a script may run, unless its configuration says otherwise.

smarter.apps.account.repository_scripts.SCRIPT_INHERITED_VARIABLES = ('PATH', 'HOME', 'LANG', 'LC_ALL', 'TZ')

The only variables of the platform’s own environment that a script receives.

('PATH', 'HOME', 'LANG', 'LC_ALL', 'TZ')

smarter.apps.account.repository_scripts.SCRIPT_MAX_TIMEOUT = 3600: The most seconds that a script’s configuration may give it.

smarter.apps.account.repository_scripts.SCRIPT_STAGE_POST = 'post': The stage of the scripts that run after every manifest is applied, and the LLMClients deployed.

smarter.apps.account.repository_scripts.SCRIPT_STAGE_PRE = 'pre': The stage of the scripts that run before any manifest is applied.

smarter.apps.account.repository_scripts.ScriptStage

SCRIPT_STAGE_PRE or SCRIPT_STAGE_POST.

Type:: When a script runs

alias of Literal[‘pre’, ‘post’]

smarter.apps.account.repository_scripts.load_scripts_config(local_path)[source]

Read and validate a repository’s scripts/scripts.yaml.

Parameters:: local_path (str) – The repository’s clone.
Returns:: The configuration, or None if the repository has none.
Return type:: RepositoryScriptsConfig | None
Raises:: SmarterValueError – If the file is not valid YAML, or not a valid configuration, or names a script that doesn’t exist or isn’t executable.

smarter.apps.account.repository_scripts.run_script(local_path, script, env)[source]

Run one of a repository’s scripts, in the repository’s root folder, and log what it writes.

Each line that the script writes is logged on its own, after the script’s name: a line of its standard output at INFO, its report, and a line of its standard error at WARNING, since a script writes problems there. The start, and the time that it took, are logged at INFO.

smarter.apps.account.repository_scripts.run_script() scripts/create-course-database.sh: checked: smarter_test_db.courses has 48 courses

Parameters:

local_path (str) – The repository’s clone.
script (RepositoryScript) – The script’s configuration.
env (dict[str, str]) – The script’s environment, from script_environment().

Returns:

What the script wrote to its standard output.

Return type:

str

Raises:

SmarterValueError – If the script exits non-zero, or runs out of time.

smarter.apps.account.repository_scripts.script_environment(url, local_path, account_number, username)[source]

Return the environment that a repository’s script runs in, and nothing more.

Variable	Value
`SMARTER_REPOSITORY_URL`	The repository’s url.
`SMARTER_REPOSITORY_ROOT`	The clone’s folder, which is also the script’s working folder.
`SMARTER_ACCOUNT_NUMBER`	The account that owns the repository’s resources.
`SMARTER_USERNAME`	The user that owns them.
`SMARTER_DATABASE_HOST`	The host of the platform’s database server.
`SMARTER_DATABASE_PORT`	Its port.
`SMARTER_DATABASE_ADMIN_USERNAME`	Its administrator, `smarter_settings.mysql_root_username`.
`MYSQL_PWD`	The administrator’s password, `smarter_settings.mysql_root_password`, where the MariaDB and MySQL clients read it. Missing if it is not set.

Plus PATH, HOME, LANG, LC_ALL and TZ from the platform’s environment, if they are set, SCRIPT_INHERITED_VARIABLES, and no other: a script never sees the platform’s own credentials.

Parameters:

url (str) – The repository’s url.
local_path (str) – The repository’s clone.
account_number (str) – The account that owns the repository’s resources.
username (str) – The user that owns them.

Returns:

The environment.

Return type:

dict[str, str]

smarter.apps.account.repository_scripts.scripts_config_path(local_path)[source]

Return the path of a repository’s scripts configuration file, whether or not it exists.

Parameters:: local_path (str) – The repository’s clone.
Returns:: <local_path>/scripts/scripts.yaml.
Return type:: str