Models
A model is a versioned set of files a run produced for use elsewhere, such as weights. Models work exactly like datasets: hl.models has every function hl.datasets has, and hlab model every command of hlab dataset.
Create a model
import hectiqlab as hl
with hl.run("Train two-tower"):
...
hl.models.create("checkpoints/", name="two-tower")Created inside a run, the model is linked to it. The run waits for the upload before it ends.
location and files say where its folder lives and which files vouch for it, as for a dataset:
hl.models.create("checkpoints/", name="two-tower", location="gs://acme-ml/two-tower/v4")Download a model
folder = hl.models.download("two-tower", "1.3").result()Without a version, you get the latest. Without into, the files land in the cache, in <cache>/<project>/models/<name>/<version>/.
It fetches only what is missing or damaged, checks each file that vouches against the checksum Hectiq Lab keeps, and refuses a folder that holds another version. A model in a gs:// folder comes from that folder, with your Google login. A model kept on a machine, or data that is not files, cannot be downloaded: download fails and says where its data lives. Called while a run is open, it records that the run read this version.
Find a model on this machine
folder = hl.models.resolve("two-tower", "1.3").unwrap()resolve gives the folder of an intact copy on this machine and never downloads. Without a version, it picks the highest major.minor with an intact copy on this machine and logs the pick at INFO; a higher version skipped because its copies are damaged is logged as a warning. When there is no intact copy, the error gives the hlab model download command to run, or, for a model kept on a machine or data that is not files, where its data lives. The CLI prints the path alone, so $(hlab model resolve two-tower) works in a shell.
A script that loads a model resolves it inside its run, after hlab model download was done once beforehand:
with hl.run("Evaluate two-tower"):
folder = hl.models.resolve("two-tower", "1.3").unwrap()
...Check a copy
checks = hl.models.verify("two-tower", "1.3").unwrap()verify gives each copy on this machine and its state, and exits with 1 on the CLI when no copy is intact.
The marker and the cache
A model's folder carries the same .hectiqlab.json marker as a dataset's, and downloads land in the same cache. hlab cache list and hlab cache clear cover models too; see datasets.
Everything else
| To | Python | CLI |
|---|---|---|
| Read | hl.models.get(name, version) | hlab model get |
| List | hl.models.list() | hlab model list |
| List the files | hl.models.files(id) | hlab model files |
| Change | hl.models.update(id, ...) | hlab model update |
| Tag | hl.models.tag(id, *tags) | hlab model tag |
| Attach to a run | hl.models.attach(id) | |
| Delete | hl.models.delete(id) | hlab model delete |
Each is described on the datasets page.