ModelConfig¶

The ModelConfig class wraps model and tokenizer loading.

ModelConfig ¶

ModelConfig(model_id: str = None, path: str = None, dtype: str = 'float16', device: str = 'auto')

Model and Tokenizer

Parameters:

Name	Type	Description	Default
`model_id`	`str`	Model ID (Hugging Face Hub ID).	`None`
`path`	`str`	Path to the saved model and tokenizer.	`None`
`dtype`	`str`	Data type. Defaults to "float16".	`'float16'`
`device`	`str`	Device to use ("cpu", "cuda", "auto"). Defaults to "auto".	`'auto'`

Example

model_config = ModelConfig(model_id="TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T") model = model_config.load_model() tokenizer = model_config.load_tokenizer()

get_model_id_or_path ¶

get_model_id_or_path()

Get the model ID or path

load_model ¶

load_model(device_map=None)

Load the model

Automatically selects the appropriate AutoModel class based on the model's architecture (CausalLM, Vision2Seq, etc.).

load_tokenizer ¶

load_tokenizer()

Load the tokenizer

has_additional_data ¶

has_additional_data()

Check if the model has additional data

Returns True if there are settings other than model_id, path, dtype, device. Currently always returns False. Should return True when additional settings are added.