SAMLLMHostCompute

smarter.models.sam.llmhostcompute

SAM model of the LLMHostCompute kind, generated from the Smarter platform’s JSON Schema.

Do not edit: run scripts/generate_sam_models.py. See smarter.models.sam.

class smarter.models.sam.llmhostcompute.SAMDependency

Bases: smarter.models.sam.base.SAMModel

A resource that depends on the resource that a manifest describes.

kind: str

Kind[String]: The SAM kind of the resource that depends on this resource, e.g. LLMClient.

Read only.

name: str

Name[String]: The name of the resource that depends on this resource.

Read only.

class smarter.models.sam.llmhostcompute.SAMLLMHostComputeCost

Bases: smarter.models.sam.base.SAMModel

Spec.cost: the price of one node, from which LLMHosts’ costs are reported.

currency: str | None

LLMHostCompute.spec.cost.currency[str]: e.g. USD.

perHour: Union

LLMHostCompute.spec.cost.perHour[decimal]: the cost per hour of one node, e.g. its on-demand price.

class smarter.models.sam.llmhostcompute.SAMLLMHostComputeGpu

Bases: smarter.models.sam.base.SAMModel

Spec.node.gpu: the node’s GPUs.

count: int

LLMHostCompute.spec.node.gpu.count[int]: the node’s GPUs.

memoryGb: int

LLMHostCompute.spec.node.gpu.memoryGb[int]: the memory of one GPU, in GB.

resource: str | None

LLMHostCompute.spec.node.gpu.resource[str]: the Kubernetes extended resource that advertises the GPUs.

type: str

LLMHostCompute.spec.node.gpu.type[str]: the GPU model, e.g. L4, A10G, H100-80GB.

class smarter.models.sam.llmhostcompute.SAMLLMHostComputeMetadata

Bases: smarter.models.sam.base.SAMModel

Smarter API LLMHostCompute Manifest - Metadata class.

annotations: None | list[dict[str, str | int | float | bool | datetime.date | datetime.datetime | Annotated[str, StringConstraints(strip_whitespace=None, to_upper=None, to_lower=None, strict=None, min_length=None, max_length=None, pattern='^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$')] | uuid.UUID | bytes | list | dict[str, Any]]]

The manifest annotations.

Used for storing arbitrary metadata as

key-value pairs. Example: [{‘smarter.sh/test-manifest/project-name’: ‘Scooby dooby do’}]. The key should be a valid url-friendly string. The value accepts multi-line string values (YAML block scalars) and various scalar types including str, int, float, bool, datetime.date, datetime.datetime, decimal.Decimal, uuid.UUID, bytes, list, dict.

description: str | None

The description for this resource.

Be brief. Keep it under 255 characters.

name: str

The camelCase name of the manifest resource.

tags: list[str] | None

The tags of the manifest.

Used for generic resource categorization and search. Example: [‘tag1’, ‘tag2’]

version: str | None

The semantic version of the manifest.

Example: 0.1.0

class smarter.models.sam.llmhostcompute.SAMLLMHostComputeNode

Bases: smarter.models.sam.base.SAMModel

Spec.node: the kind of node.

amiType: str | None

LLMHostCompute.spec.node.amiType[str]: e.g. AL2023_x86_64_NVIDIA.

Defaults to the NVIDIA AMI for a GPU node, else the standard AMI.

capacityType: Optional

LLMHostCompute.spec.node.capacityType[str]: ON_DEMAND or SPOT.

cpu: int

LLMHostCompute.spec.node.cpu[int]: the node’s vCPUs.

diskSizeGb: Optional

LLMHostCompute.spec.node.diskSizeGb[int]: the node’s root volume, for container images.

Model weights are on the LLMHosts’ own volumes.

gpu: smarter.models.sam.llmhostcompute.SAMLLMHostComputeGpu | None

LLMHostCompute.spec.node.gpu[object]: the node’s GPUs.

None for a CPU node.

instanceType: str

LLMHostCompute.spec.node.instanceType[str]: e.g. g6.2xlarge.

memoryGb: int

LLMHostCompute.spec.node.memoryGb[int]: the node’s memory, in GiB.

class smarter.models.sam.llmhostcompute.SAMLLMHostComputeNodeGroup

Bases: smarter.models.sam.base.SAMModel

Spec.nodeGroup: the node group of the nodes.

maxNodes: Optional

LLMHostCompute.spec.nodeGroup.maxNodes[int]: the most nodes that Smarter adds to the node group.

subnetIds: list[str] | None

LLMHostCompute.spec.nodeGroup.subnetIds[list]: the node group’s subnets.

Defaults to the platform’s. One subnet keeps the nodes in one availability zone, with the LLMHosts’ model volumes.

class smarter.models.sam.llmhostcompute.SAMLLMHostComputeSpec

Bases: smarter.models.sam.base.SAMModel

Smarter API LLMHostCompute Manifest LLMHostCompute.spec.

cost: smarter.models.sam.llmhostcompute.SAMLLMHostComputeCost | None

LLMHostCompute.spec.cost[object]: for reporting.

node: SAMLLMHostComputeNode

LLMHostCompute.spec.node[object]: the kind of node.

nodeGroup: smarter.models.sam.llmhostcompute.SAMLLMHostComputeNodeGroup | None

LLMHostCompute.spec.nodeGroup[object]: the node group of the nodes.

class smarter.models.sam.llmhostcompute.SAMLLMHostComputeStatus

Bases: smarter.models.sam.base.SAMModel

Smarter API LLMHostCompute Manifest - Status class.

Read only. Besides ownership, it reports the node group, as of its most recent reconcile.

accountNumber: str

LLMHostCompute.status.account_number: The account owner of this LLMHostCompute.

Read only.

created: datetime

The date in which this resource was created.

Read only.

dependencies: list[smarter.models.sam.llmhostcompute.SAMDependency] | None

Dependencies[List]: The resources that depend on this resource, and that must be deleted, or stop referring to it, before it can be deleted.

Lists only the resources that you can read. Read only.

desiredNodes: int | None

LLMHostCompute.status.desiredNodes: the node group’s desired size. Read only.

lastReconciledAt: datetime.datetime | None

LLMHostCompute.status.lastReconciledAt: when it was last reconciled.

Read only.

llmhosts: int | None

LLMHostCompute.status.llmhosts: the LLMHosts that run on it.

Read only.

message: str | None

LLMHostCompute.status.message: the result of the last reconcile. Read only.

modified: datetime

The date in which this resource was most recently changed.

Read only.

nodeGroupName: str | None

LLMHostCompute.status.nodeGroupName: the cloud’s name of the node group.

Read only.

nodeGroupStatus: str | None

LLMHostCompute.status.nodeGroupStatus: e.g. absent, CREATING or ACTIVE.

Read only.

readyNodes: int | None

LLMHostCompute.status.readyNodes: the nodes that have joined the cluster.

Read only.

recordLocator: str

RecordLocator[String]: An optional identifier used to locate the resource record associated with this manifest.

Read only.

username: str

LLMHostCompute.status.username: The Smarter user who created this LLMHostCompute.

Read only.

class smarter.models.sam.llmhostcompute.SAMLLMHostCompute

Bases: smarter.models.sam.base.SAMModel

Smarter API Manifest - LLMHostCompute.

apiVersion: str

ApiVersion[String]: Required.

The API version of the AbstractSAMBase.

kind: str

Kind[String]: Required.

The kind of resource described by the manifest.

metadata: SAMLLMHostComputeMetadata

LLMHostCompute.metadata[obj]: Required, the LLMHostCompute metadata.

spec: SAMLLMHostComputeSpec

LLMHostCompute.spec[obj]: Required, the LLMHostCompute specification.

status: smarter.models.sam.llmhostcompute.SAMLLMHostComputeStatus | None

LLMHostCompute.status[obj]: Optional, Read-only.

Stateful status information about the LLMHostCompute.