objects

MLModels

MLModels (short for Machine Learning Models) is a designated place to store your models within a project. They can be easily shared within a project, used in apps and tracked.

Upload

MLModels can take the form of a directory or a single file. Suppose you have a model saved on your local machine with ./model/config.pt and ./model/weights.h5 files. When the model is a directory, use push_dir=True (this argument is here to prevent from unintentionally uploading directories).

from pyhectiqlab import Run
run = Run(name="Training", project="lab/demo")

# Do your training here and save your model

model_dir = "./model/"
run.add_mlmodel(model_dir, name="linear-regression", push_dir=True)

Your mlmodel is now uploaded to the lab. Dataset upload is made sync on the main thread.

You can modify the README.md of an existing model using

run.set_mlmodel_readme("./README.md", name="linear-regression, version="1.0.0")

Download

The download code snippets are always available prefilled from the web application, in the tab Usage.

You can download any mlmodels from a project you have access to. You can download mlmodels using the python client or the command line client.

Python client

If you choose to download the mlmodel from the python client, you can choose to use a run context such as

run.download_mlmodel("cat-images")

When downloaded from within a run context, the run is automatically attached to the mlmodel. In other words, in the run page, a link to the mlmodel will be made available. Eventually, we plan to integrate the full mlmodel-model-run flow to help users understand their experiments. For this reason, we recommend using run.download_mlmodel to prepare the flow diagram.

The second option is to download it outside a run:

from pyhectiqlab.mlmodels import download_mlmodel
download_mlmodel(mlmodel_name='cat-images', project_path='lab/demo')

Command line

The third option is to use the command line

hectiqlab download-mlmodel -p lab/demo -n cat-images

For all download methods, other parameters are available (mlmodel version, save path, overwrite). See methods details.

When you download a mlmodel, a directory is created with the format {name}-{version}. Also with the mlmodel files, a HLab_README.md is created with meta information and a hidden file .hlab_meta.json.

Record usage

It is often useful to track the existing mlmodel usage. You can use run.add_mlmodel_usage and run.add_mlmodel_usage_from_dirpath to keep track of this usage. The former takes the name and version as arguments while the latter infers the name and version from the path directory where it's been saved. Using these methods help build the flow diagram of your practice.

from pyhectiqlab import Run
run = Run(name="Linear regression prediction", project="lab/demo")

mlmodel_path = "./cat-images-1.0.0"
run.add_mlmodel_usage_from_dirpath(mlmodel_path)

# Equivalent
run.add_mlmodel_usage("cat-images", version="1.0.0")

Methods

pyhectiqlab.Run

Run.add_mlmodel(source_path: str, name: str, version: str = None, description: str = None, push_dir: bool = False, resume_upload: bool = False)

Upload a mlmodel to the lab.

Parameters

Property	Type	Default	Description
`source_path`	`str`	`-`	Path to the mlmodel file or directory.
`name`	`str`	`-`	The mlmodel name. The lab will use a slugified version of the name.
`version`	`str`	`None`	Version in format '{major}.{minor}.{micro}' (e.g., 1.2.0). If `None`, the version 1.0.0 is assigned or an increment of minor of the latest version of the mlmodel with this name (e.g. 1.3.3 -> 1.4.0)
`description`	`str`	`None`	An optional short description of the mlmodel. For larger description, push a README.md file in the root path.
`push_dir`	`bool`	`False`	If the `source_path` is a directory, use `push_dir=True` to confirm that you understand that a directory will be uploaded.
`resume_upload`	`bool`	`False`	If `True`, you'll be able to push files on an existing mlmodel.

Run.download_mlmodel(mlmodel_name: str, version: str = None, save_path: str = ./, overwrite: bool = False)

Download an existing mlmodel from a run. Returns the path to the saved mlmodel.

Parameters

Property	Type	Default	Description
`mlmodel_name`	`str`	`-`	Name of the mlmodel.
`version`	`str`	`None`	Specific version of the mlmodel. If `None`, the latest version is fetched.
`save_path`	`str`	`./`	Path to where the mlmodel will be saved.
`overwrite`	`bool`	`False`	Set to `True` if you want to download the mlmodel again even if it is already saved on your machine.

Run.add_mlmodel_usage_from_dirpath(dirpath: str)

Attach a run to an existing mlmodel saved on your machine. The name and version of the mlmodel is inferred from the directory.

Parameters

Property	Type	Default	Description
`dirpath`	`str`	`-`	Path to where the mlmodel is saved.

Run.add_mlmodel_usage(name: str, version: str)

Attach the run to an existing mlmodel.

Parameters

Property	Type	Default	Description
`name`	`str`	`-`	MLModel name
`version`	`str`	`-`	MLModel version

pyhectiqlab

pyhectiqlab.mlmodels.download_mlmodel(mlmodel_name: str, project: str, version: str, save_path: str = ./, overwrite: str = False)

Download an existing mlmodel without a run context.

Parameters

Property	Type	Default	Description
`mlmodel_name`	`str`	`-`	MLModel name
`project`	`str`	`-`	Project name
`version`	`str`	`-`	Specific version of the mlmodel. If `None`, the latest version is fetched.
`save_path`	`str`	`./`	Save path.
`overwrite`	`str`	`False`	Set to `True` if you want to download the mlmodel again even if it is already saved on your machine.