SAMLoader Class

SAMLoader is a lightweight utility class for loading and parsing Smarter API Manifest (SAM) files into Pydantic models. It provides a simple interface for reading manifest files, validating their contents, and converting them into lightweight structured data models that can be used for initializing SAM Pydantic models.

Usage:

from smarter.apps.secret.manifest.models.secret.model import (
    SAMSecret,
    SAMSecretMetadata,
    SAMSecretSpec,
)
from smarter.lib.manifest.loader import SAMLoader

# initialize the loader from a yaml file.
loader = SAMLoader(manifest_path="path/to/manifest.yaml")

# initialize a SAM Pydantic model from the loader.
self._manifest = SAMSecret(
    apiVersion=self.loader.manifest_api_version,
    kind=self.loader.manifest_kind,
    metadata=SAMSecretMetadata(**self.loader.manifest_metadata),
    spec=SAMSecretSpec(**self.loader.manifest_spec),
)

Smarter API Manifest Loader base class.

class smarter.lib.manifest.loader.SAMLoader(*args, api_version='smarter.sh/v1', kind=None, manifest=None, file_path=None, url=None, **kwargs)[source]

Smarter API Manifest Loader base class.

This class provides the foundational logic for loading, parsing, and validating Smarter API Manifest files, prior to attempting to use the manifest data for initializing a Pydantic model. It is designed to handle manifests provided as JSON or YAML, and supports loading from a string, dictionary, file path, or URL. The loader ensures that the manifest conforms to a specified schema, validating both the structure and the data types of the manifest’s contents. In cases where the syntax or structure of the manifest is invalid, the loader is intended to provide more human readable error messages than those produced by Pydantic alone.

Usage Overview

The SAMLoader is intended to be subclassed for specific manifest types. It performs the following core functions:

Manifest Acquisition: Accepts manifest data in various forms, including a raw string, a Python dictionary, a file path, or a URL. Only one source should be provided at a time. The loader reads and stores the manifest data for further processing.
Specification Enforcement: Maintains a specification dictionary that defines the required structure and data types for the manifest. This includes top-level keys such as API version, kind, metadata, spec, and status, as well as nested metadata requirements.
Format Detection: Automatically detects whether the manifest data is in JSON or YAML format, and parses it accordingly. If the format is invalid or unsupported, an error is raised.
Validation: Recursively validates the manifest data against the specification. This includes checking for required keys, verifying data types, and ensuring that values conform to enumerated options where applicable. Validation errors are raised as exceptions.
Extensibility: Designed for subclassing, allowing child classes to override the specification and validation logic to accommodate custom manifest structures.

Manifest Structure

The expected manifest structure includes the following top-level keys:

apiVersion: The version of the Smarter API. Must be supported by the loader.
kind: The type of manifest. Can be specified directly or inferred from the manifest data.
metadata: A dictionary containing descriptive information such as name, description, version, tags, and annotations. Certain fields are required, while others are optional.
spec: The specification section, which is required and typically defined by subclasses.
status: An optional, read-only section for status information.

Validation Process

Upon initialization, the loader validates the manifest by:

Ensuring the manifest data is present and in a supported format.
Recursively checking each key and value against the specification.
Raising detailed errors if any required keys are missing, data types are incorrect, or values are invalid.

Subclassing

To support custom manifest types, create a subclass of SAMLoader and override the _specification attribute and the validate_manifest method as needed. This allows for the enforcement of additional keys, custom validation logic, and specialized handling of manifest data.

Error Handling

All validation and loading errors are raised as SAMLoaderError exceptions, providing clear feedback on the nature of the issue encountered.

Logging

The loader uses Python’s standard logging library to emit warnings and errors during the validation process, aiding in debugging and traceability.

__init__(*args, api_version='smarter.sh/v1', kind=None, manifest=None, file_path=None, url=None, **kwargs)[source]

Initialize a new instance of the SAMLoader.

This constructor is responsible for acquiring, parsing, and validating a Smarter API Manifest. It supports loading manifest data from several sources, but only one source should be provided at a time.

Parameters:

api_version (str) – The API version of the manifest. Must be one of the supported API versions.
kind (Optional[str]) – The kind of manifest. If not provided, it will be inferred from the manifest data.
manifest (Union[str, dict, None]) – The manifest data, provided as a JSON/YAML string or a Python dictionary.
file_path (Optional[str]) – Path to a file containing the manifest data.
url (Optional[str]) – URL pointing to the manifest data.

Raises:

SAMLoaderError – If the API version is not supported, if no manifest source is provided, if multiple sources are provided, or if the manifest format is invalid.

Acquisition and Validation Process

The constructor checks that the provided API version is supported.
It ensures that exactly one manifest source is specified (manifest, file_path, or url).
The manifest data is loaded from the specified source.
The loader sets the specification keys for API version and kind.
The manifest is validated against the specification, checking for required keys and correct data types.

Child classes may override the specification and validation logic to support custom manifest structures.

property amnesty_urls: list[str]

Returns a list of URLs that are exempt from certain checks.

Returns:: List of URL path strings that are exempt.
Return type:: list[str]

property api_version: str

bool_environment_variable(var_name, default=False)

Retrieves a boolean value from an environment variable.

This method checks the specified environment variable and returns its value as a boolean. It recognizes common truthy values such as “true”, “1”, “yes”, and “on”. If the variable is not set or cannot be interpreted as a boolean, it returns the provided default value.

Parameters:

var_name (str) – The name of the environment variable to check.
default (bool) – The default boolean value to return if the environment variable is not set or invalid.

Returns:

The boolean value of the environment variable or the default.

Return type:

bool

property data_format: SAMDataFormats

data_to_dict(data)

Converts data to a dictionary, handling different types of input.

This method accepts either a dictionary or a string. If a string is provided, it will attempt to parse it as JSON first, and if that fails, as YAML. If parsing fails or the data type is unsupported, a SmarterValueError is raised.

Parameters:: data (Union[dict, str]) – The data to convert, either a dict or a JSON/YAML string.
Returns:: The data as a dictionary.
Return type:: dict
Raises:: SmarterValueError – If the data cannot be converted to a dictionary.

deserves_amnesty(slug)

Determines if a given URL deserves amnesty based on the amnesty URLs list.

This excuses certain endpoints (like health checks) from select middleware checks.

Parameters:: slug (str) – The URL path to check.
Returns:: True if the URL deserves amnesty, False otherwise.
Return type:: bool

dict_is_contained_in(dict1, dict2)

Checks if one dictionary is contained within another.

This method determines if all key-value pairs in dict1 are present in dict2.

Parameters:

dict1 (dict) – The dictionary to check for containment.
dict2 (dict) – The dictionary to check against for containment.

Returns:

True if dict1 is contained in dict2, False otherwise.

Return type:

bool

dict_is_subset(small, big)

Checks if one dictionary is a subset of another.

This method determines if all key-value pairs in the small dictionary are present in the big dictionary. It returns True if the small dictionary is a subset of the big dictionary, and False otherwise.

Parameters:

small (dict) – The dictionary to check as a subset.
big (dict) – The dictionary to check against as a superset.

Returns:

True if the small dictionary is a subset of the big dictionary, False otherwise.

Return type:

bool

property file_path: str | None

property formatted_class_name: str

Returns the class name formatted for logging.

Returns:: The formatted class name as a string.
Return type:: str

property formatted_data: str

formatted_json(json_obj)

Formats a JSON object as a pretty-printed string with ANSI color codes for logging.

Parameters:: json_obj (Union[dict, list]) – The JSON object (dict or list) to format.
Returns:: A string representation of the JSON object with ANSI color codes.
Return type:: str

property formatted_state_not_ready: str

Returns the not-ready state formatted for logging.

Returns:: The formatted not-ready state as a string.
Return type:: str

property formatted_state_ready: str

Returns the readiness state formatted for logging.

Returns:: The formatted readiness state as a string.
Return type:: str

formatted_text(text, color_code='\\x1b[1;31m')

Formats text with ANSI color codes for logging.

Parameters:

text (str) – The text to format.
color_code (str) – The ANSI color code to apply.

Returns:

The formatted text with ANSI color codes.

Return type:

str

formatted_text_blue(text)

Formats text in bold dark blue for logging.

Parameters:: text (str) – The text to format.
Returns:: The formatted text in bold dark blue.
Return type:: str

formatted_text_green(text)

Formats text in bright green for logging.

Parameters:: text (str) – The text to format.
Returns:: The formatted text in bright green.
Return type:: str

formatted_text_red(text)

Formats text in dark red for logging.

Parameters:: text (str) – The text to format.
Returns:: The formatted text in dark red.
Return type:: str

generate_fernet_encryption_key()

Generates a Fernet encryption key.

This method creates a new Fernet encryption key, which can be used for secure encryption and decryption of data. The generated key is returned as a URL-safe base64-encoded string.

Returns:: A new Fernet encryption key.
Return type:: str

get_key(key)[source]

Get a key from the manifest’s JSON data.

Parameters:: key (str) – The key to retrieve.
Returns:: The value of the key, or None if the key does not exist.
Return type:: Any

get_readonly_csv_file(file_path)

Retrieves a read-only file object for a CSV file.

This method opens the specified CSV file in read-only mode and returns a file object that can be used to read its contents. It ensures that the file is not modified during the reading process.

Parameters:: file_path (str) – The path to the CSV file to open.
Returns:: A read-only file object for the specified CSV file.
Return type:: file

get_readonly_yaml_file(file_path)

Retrieves a read-only file object for a YAML file.

This method opens the specified YAML file in read-only mode and returns a file object that can be used to read its contents. It ensures that the file is not modified during the reading process.

Parameters:: file_path (str) – The path to the YAML file to open.
Returns:: A read-only file object for the specified YAML file.
Return type:: file

property health_check_urls: list[str]

Returns a list of URL paths that are considered health check endpoints.

Returns:: List of health check URL path strings.
Return type:: list[str]

property json_data: dict[str, Any] | None

property kind: str

loader_ready_state()[source]

Returns a string representation of the loader’s readiness state.

Returns:: “ready” if the loader is ready, otherwise “not ready”.
Return type:: str

log_loader_state()[source]

Log the current state of the SAMLoader instance for debugging purposes.

Returns:: None

property manifest: str | dict[str, Any] | None

property manifest_api_version: str

Returns the API version of the manifest.

Returns:: The API version string.
Return type:: str

property manifest_kind: str

Returns the kind of the manifest.

Returns:: The kind string.
Return type:: str

property manifest_metadata: dict

Returns the metadata section of the manifest.

Returns:: The metadata dictionary.
Return type:: dict

property manifest_metadata_keys: list[str]

Returns a list of all metadata keys defined in the SAMMetadataKeys enumeration.

Returns:: A list of metadata key strings.
Return type:: list[str]

property manifest_spec: dict

Returns the spec section of the manifest.

Returns:: The spec dictionary.
Return type:: dict

property manifest_spec_keys: list[str]

Returns a list of all spec keys defined in the SAMSpecKeys enumeration.

This should be overridden by child classes to provide the specific spec keys relevant to their manifest type.

Returns:: A list of spec key strings.
Return type:: list[str]

property manifest_status: dict

Returns the status section of the manifest.

Returns:: The status dictionary.
Return type:: dict

property manifest_status_keys: list[str]

Returns a list of all status keys defined in the SAMStatusKeys enumeration.

This should be overridden by child classes to provide the specific status keys relevant to their manifest type.

Returns:: A list of status key strings.
Return type:: list[str]

mask_string(string='', mask_char='*', mask_length=4, string_length=8)

Masks a string for secure logging.

This utility function masks all but the last unmasked_chars characters of the input string, replacing them with asterisks. It is useful for logging sensitive information like API keys or passwords.

Parameters:

string (Optional[str]) – The string to be masked.
mask_char (str) – The character used for masking.
mask_length (int) – The number of characters to mask.
string_length (int) – The length of the string to consider for masking.

Returns:

The masked string.

Return type:

str

property name: str

pydantic_model_dump()[source]

Return a dictionary representation of the manifest data suitable for use with Pydantic models.

This method produces a dictionary that can be directly passed as keyword arguments to any descendant of the AbstractSAMBase class. This enables seamless integration with Pydantic-based data validation and serialization workflows.

The returned dictionary includes the following keys:

apiVersion: The API version of the manifest.
kind: The kind of manifest.
metadata: The manifest metadata section.
spec: The manifest specification section.
status: The manifest status section.

Example usage:

SAMObject(**loader.pydantic_model_dump())

Returns:: Dictionary representation of the manifest data, suitable for Pydantic model instantiation.
Return type:: dict

property raw_data: str | dict | None

property ready: bool

Returns whether the manifest is ready for use.

This property returns True if the manifest has been successfully validated and is in a valid state. A manifest is considered ready if all required sections—API version, kind, metadata, and spec—are present and non-empty. This property is useful for checking the readiness of a manifest before attempting to use it in downstream processes or integrations.

Returns:: True if the manifest is valid and ready to be used, otherwise False.
Return type:: bool

recursive_sort_dict(data)

Recursively sorts a dictionary by its keys.

This method takes a dictionary and returns a new dictionary with all keys sorted in ascending order. If any values are also dictionaries, they will be sorted recursively as well.

Parameters:: data (dict) – The dictionary to sort.
Returns:: A new dictionary with all keys sorted.
Return type:: dict

rfc1034_compliant_str(name)

Converts a string to an RFC 1034 compliant format.

This method takes a string and converts it to a format that complies with RFC 1034, which is commonly used for domain names. It replaces invalid characters with hyphens and ensures the resulting string is lowercase.

Parameters:: name (str) – The string to convert to RFC 1034 compliant format.
Returns:: The converted string in RFC 1034 compliant format.
Return type:: str

rfc1034_compliant_to_snake(name)

Converts an RFC 1034 compliant string to snake_case.

This method takes a string in RFC 1034 compliant format and converts it to snake_case. It replaces hyphens with underscores and ensures the resulting string is lowercase.

Parameters:: name (str) – The RFC 1034 compliant string to convert.
Returns:: The converted string in snake_case.
Return type:: str

smarter_build_absolute_uri(request)

Attempts to get the absolute URI from a request object.

This utility function tries to retrieve the request URL from any valid child class of django.http.HttpRequest. It is especially useful in unit tests or scenarios where the request object may not implement build_absolute_uri().

Parameters:: request (HttpRequest) – The request object.
Returns:: The absolute request URL.
Return type:: Optional[str]
Raises:: SmarterValueError – If the URI cannot be built from the request.

sorted_dict(data)

Returns a new dictionary with keys sorted.

Parameters:: data (dict) – The dictionary to sort.
Returns:: A new dictionary with sorted keys.
Return type:: dict

property source: str

property specification: dict

to_camel_case(data, convert_values=False)

Converts a snake_case string to camelCase.

This method takes a string in snake_case format and converts it to camelCase. It is useful for standardizing naming conventions across different formats.

Parameters:

data (Union[str, dict[str, object], list[object], object]) – The snake_case string to convert.
convert_values (bool) – Whether to convert the values of dictionaries and lists recursively.

Returns:

The converted string in camelCase.

Return type:

Any

to_json()[source]

Return the manifest data as a JSON string.

Returns:: The manifest data in JSON format.
Return type:: dict[str, Any]

to_snake_case(data, convert_values=False)

Converts a camelCase or PascalCase string to snake_case.

This method takes a string in camelCase or PascalCase format and converts it to snake_case. It is useful for standardizing naming conventions across different formats.

Parameters:

data (Union[str, dict[str, object], list[object], object]) – The camelCase or PascalCase string to convert.
convert_values (bool) – Whether to convert the values of dictionaries and lists recursively.

Returns:

The converted string in snake_case.

Return type:

Any

property unformatted_class_name: str

Returns the raw class name without formatting.

Returns:: The unformatted class name as a string.
Return type:: str

This is useful for logging or serialization where the plain class name is needed.

property url: str | None

validate_manifest()[source]

Validate the manifest data.

This method performs a comprehensive validation of the manifest data against the expected specification. Validation is performed recursively, ensuring that all required keys are present, that values are of the correct data types, and that any enumerated or restricted values are respected. The validation process is driven by the specification dictionary defined for the loader, which outlines the required structure and constraints for the manifest.

The validation process includes the following steps:

Checks that the manifest data is present and not empty.
Ensures that the manifest data is in a supported format (JSON or YAML).
Recursively traverses the manifest data, validating each key and value against the corresponding entry in the specification.
For each key: - If the specification entry is a dictionary, validation is performed recursively on the nested structure. - If the specification entry is a tuple, the method checks for required keys, correct data types, and any special options (such as optional or read-only). - If the specification entry is a list, the method checks that the value is one of the allowed options. - Otherwise, the method checks for exact value and type matches.
Raises a SAMLoaderError with a descriptive message if any validation check fails.

This method is intended to be called automatically during loader initialization, but can also be invoked manually to re-validate the manifest after modifications.

Subclasses may override this method to implement additional or custom validation logic as needed.

Raises:: SAMLoaderError – If the manifest data is missing, in an unsupported format, or fails validation.

property yaml_data: str | None

exception smarter.lib.manifest.loader.SAMLoaderError(message='')[source]

Exception class for all errors raised by the Smarter API Manifest Loader.

This is the base error type for manifest validation, parsing, and loading operations in the Smarter API system. All errors encountered during manifest handling, including schema violations, unsupported formats, and missing data, should be raised as or derived from SAMLoaderError.

Parameters

This class does not require any parameters for instantiation, but it may be initialized with a custom error message describing the specific failure.

Parameters:: message (str) – A descriptive error message explaining the cause of the error.

Usage Example

# Example: Raising a loader error for a missing required key
if not manifest.get("apiVersion"):
    raise SAMLoaderError("Missing required key: apiVersion")

# Example: Handling loader errors in client code
try:
    loader = SAMLoader(manifest=my_manifest)
except SAMLoaderError as err:
    print(f"Manifest validation failed: {err}")

Note

All manifest validation and parsing errors should use this class for consistency and traceability.
The get_formatted_err_message property provides a static, human-readable error label for logging and display.

Attention

Catching this exception broadly may mask specific validation issues. Always inspect the error message for details.
This class is intended for use within the manifest loader and related validation logic. For other error types, use the appropriate exception class.

Raises:: This class is raised directly or via subclassing for any manifest loader error.

__init__(message='')

add_note(object, /): Exception.add_note(note) – add a note to the exception

args

property get_formatted_err_message: Return the static formatted error message for SAMLoader errors.

with_traceback(object, /): Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

smarter.lib.manifest.loader.validate_key(key, key_value, spec)[source]

Validate a manifest key and its value against a specification.

This function enforces schema rules for manifest keys and values, supporting multiple validation strategies based on the type of the specification provided. It is a foundational utility for manifest validation and is used throughout the Smarter API Manifest Loader system.

Parameters

Parameters:

key (str) – The manifest key to validate. Must be a string or an Enum with a string value.
key_value (Any) – The value associated with the manifest key. The expected type and constraints depend on the specification.
spec (Any) – The specification against which the key and value are validated. This can be: - A list: The value must be one of the items in the list. - A tuple: The first element is the expected data type; the second is a list of key options (e.g., required, optional, readonly). - Any other type: The value must match the type and value of the spec.

Validation Logic

If the specification is a list, the value must be present in the list.
If the specification is a tuple, the value’s type and presence are validated according to the tuple’s contents:
- The first element is the expected type (e.g., str, dict).
- The second element is a list of options, such as REQUIRED, OPTIONAL, or READONLY.
- If the key is required and the value is missing or empty, an error is raised.
- If the value’s type does not match the expected type, an error is raised.
If the specification is any other type, the value must match both the type and the value of the spec.

Examples

# Example 1: List validation
validate_key("color", "red", ["red", "green", "blue"])
# Passes if "red" is in the list

# Example 2: Tuple validation
validate_key("name", "Widget", (str, [SAMSpecificationKeyOptions.REQUIRED]))
# Passes if "Widget" is a string and the key is required

# Example 3: Exact value validation
validate_key("apiVersion", "v1", "v1")
# Passes if the value matches the spec exactly

Note

All manifest keys must be strings. If an Enum is provided, its value is used.
If the value is missing or of the wrong type, a SAMLoaderError is raised with a descriptive message.
This function is intended for internal use in manifest validation routines and is not typically called directly.

Attention

Ensure that the specification accurately reflects the schema requirements for your manifest type.
Improper use of this function may result in manifest validation failures or runtime errors.

Raises:: SAMLoaderError – If the key or value does not conform to the specification.