Runs ​
Runs are the central concept in Hectiq Lab. They are used to track experiments, tests, and evaluations. Runs are created within a project and can be organized with tags. They can be categorized and tagged to make them easier to find. Runs can also be created with a configuration, which can be used to store the parameters and hyperparameters of the run.
Create a run ​
You can create a new run with various options. The only required option is the project. The project can be specified globally using set_project (see Projects) or locally using the project parameter.
import pyhectiqlab.functional as hl
run = hl.create_run(title="My first run")from pyhectiqlab import Run
run = Run(title="My first run")from pyhectiqlab import Run
with Run(title="My first run") as run:
pass# The project must be specified as there is no current project
hectiq-lab Run.create --title "My first run" --project "hectiq-ai/demo"| Name | Type | Default | Description |
|---|---|---|---|
title | str | None | The title of the run. * |
project | str | None | The project of the run. If None, the current project is used. |
category | Literal[None, 'develop', 'evaluation', 'test', 'training'] | None | The category of the run. |
config | Optional[dict] | None | The config of the run. |
tags | Optional[List] | None | A list of tags to attach to the run at creation. |
reuse | bool | False | Set to True to reuse a run with the same title already exists. |
*Either the rank or the title must be given. If both are given, the title is used.
TIP
👋 If you don't want to create a new run, you can use the reuse parameter. If reuse=True, the run will be reused if a run with the same title exists in the project. It is useful when you want to keep the same run for the same experiment. By default, reuse=False.
WARNING
If the project is not found, it turns the client to offline mode, where no API call are allowed to go through. Then, it logs the error and returns None. Also, if the run cannot be created on the API side, it logs the error and returns None.
Retrieve an existing run ​
You can retrieve an existing run by its rank. The rank is a unique identifier for the run within the project. The rank is available in the web application.
import pyhectiqlab.functional as hl
run = hl.retrieve_run(rank=1)from pyhectiqlab import Run
run = Run.retrieve(rank=1)from pyhectiqlab import Run
with Run(rank=1) as run:
pass# The project must be specified as there is no current project
hectiq-lab Run.retrieve --rank 1 --project "hectiq-ai/demo"INFO
Runs cannot be retrieved by the title. This is because the title is not guaranteed to be unique.
| Name | Type | Default | Description |
|---|---|---|---|
rank | int | The rank of the run. | |
project | Optional[str] | None | The project of the run. If None, the project is taken from the context. |
fields | Optional[list[str]] | None | List of fields to retrieve along the run, that will be outputed. |
setup_with | Optional[str] | True | Set to True to setup the run with the result. |
WARNING
Like at run creation, if the project is not found, it turns the client to offline mode, logs the error and returns None. If the run cannot be found within the project, it logs the error and returns None.
Check if a run exists ​
You can check if a run exists by its rank.
import pyhectiqlab.functional as hl
exists = hl.run_exists(rank=1)from pyhectiqlab import Run
exists = Run.exists(rank=1)hectiq-lab Run.exists --rank 1 --project "hectiq-ai/demo"| Name | Type | Default | Description |
|---|---|---|---|
rank | int | The rank of the run. | |
project | Optional[str] | None | The project of the run. If None, the project is taken from the context. |
Ready ​
To use most of the methods, you have to initialize a run in the current context. To make sure that the run has been initialized, you can use the ready method. It will return True if the run has been initialized and False otherwise. If the return is False, you will not be able to use most of the push methods.
import pyhectiqlab.functional as hl
ready = hl.run_ready()from pyhectiqlab import Run
Run.ready()For the methods below, you'll need to have initialized a run object. You can do this by creating a new run or retrieving an existing one. For simplicity, we'll ignore the creation of the run object and assume that it is available. You can check if the run has been initialized by running Run.ready() or hl.run_ready() for the functional approach.
Most of the methods accept the optional run_id parameter. If you don't provide the run_id parameter, the method will use the current run. Certain methods require the project parameter. If you don't provide the project parameter, the method will use the current project.
Rename a run ​
You can rename a run using the rename method.
import pyhectiqlab.functional as hl
hl.rename_run(title="My new title", run_id="9c29ckw0")run.rename(title="My new title")hectiq-lab Run.rename --run_id 9c29ckw0 --title "My new title"| Name | Type | Default | Description |
|---|---|---|---|
title | str | The new title of the run. | |
run_id | str | ID of the run. |
Change the status ​
The status of the run indicates the current state of the run. It can be changed manually or automatically by the system. You have multiple predefined statuses that you can use. You can also create your own statuses.
import pyhectiqlab.functional as hl
hl.set_run_status(status="custom_status")
hl.failed()
hl.stopped()
hl.completed()
hl.pending()
hl.running()
hl.training()run.set_status(status="custom_status")
run.failed()
run.stopped()
run.completed()
run.pending()
run.running()
run.training()hectiq-lab Run.set_status --run_id 9c29ckw0 --status "custom_status"
hectiq-lab Run.failed --run_id 9c29ckw0
hectiq-lab Run.stopped --run_id 9c29ckw0
hectiq-lab Run.completed --run_id 9c29ckw0
hectiq-lab Run.pending --run_id 9c29ckw0
hectiq-lab Run.running --run_id 9c29ckw0
hectiq-lab Run.training --run_id 9c29ckw0Add metrics ​
You can add metrics to the run using the add_metric method. You can only add one metric per call. However, the metrics are buffered and sent to the server in batches, so you don't have to worry about the performance.
import pyhectiqlab.functional as hl
hl.add_metric(key="accuracy", step=5, value=0.9)run.add_metric(key="accuracy", step=5, value=0.9)| Name | Type | Default | Description |
|---|---|---|---|
key | str | The name of the metric. | |
step | int | The step at which the metric was recorded. | |
value | float | The value of the metric. |
Add a config ​
You can add a configuration file to the run using the add_config method. You can only add one configuration file per run. If the run has already a configuration, it will be overwritten. You configuration file must be serializable to JSON using orjson.
import pyhectiqlab.functional as hl
from pyhectiqlab import Config
config = Config(a=1, b=2, c=3)
hl.add_config(config=config)from pyhectiqlab import Config
config = Config(a=1, b=2, c=3)
run.add_config(config=config)| Name | Type | Default | Description |
|---|---|---|---|
config | Union[pyhectiqlab.Config, dict] | The configuration object. |
Add and detach tags ​
Tags can be attached / detached to models by using the add_tags and detach_tag method.
import pyhectiqlab.functional as hl
hl.add_tags_to_run(run_id="9c29ckw0", tags=["some", "tag"])
hl.detach_tag_from_run(run_id="9c29ckw0", tag="some")from pyhectiqlab import Run
Run.add_tags(run_id="9c29ckw0", tags=["some", "tag"])
Run.detach_tag(run_id="9c29ckw0", tag="some")hectiq-lab Run.attach --run_id "9c29ckw0" --tags "some" --tags "tag"
hectiq-lab Run.detach_tag --run_id "9c29ckw0" --tag "some"WARNING
If the run ID does not exist, it logs an error and returns None.
Add an artifact ​
Any file ​
You can add an artifact to the run using the add_artifact method. Artifact is a file that you want to store in the run.
import pyhectiqlab.functional as hl
hl.add_artifact(path="path/to/file")run.add_artifact(path="path/to/file")hectiq-lab Run.add_artifact --run_id 9c29ckw0 --path "path/to/file"
hectiq-lab Artifact.create --run_id 9c29ckw0 --path "path/to/file"| Name | Type | Default | Description |
|---|---|---|---|
path | str | - | Path to the file to upload. |
name | str, optional | None | Name of the artifact or the group of artifacts. If None, the name is the basename of the file. |
step | int, optional | None | The optional step stamp of the artifacts. If None, the artifacts is not considered as a step artifact. |
run_id | str, optional | None | ID of the run. If None, the current run ID is used. |
project | str, optional | None | The project of the artifact. If None, the current project is used. |
wait_response | bool | False | Set to true to upload sync. If False, the upload is made in background. |
To group artifacts in the web application, use a share name for all the artifacts you want to group.
WARNING
If the project or the run ID are not found, the artifact creation is interrupted and logged as an error.
Matplotlib figure ​
For matplotlib, you can use the add_figure method to add a figure to the run. The figure will be saved to png and stored as an artifact.
import pyhectiqlab.functional as hl
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
hl.add_figure(fig=fig)import matplotlib.pyplot as plt
fig, ax = plt.subplots()
run.add_figure(fig=fig)| Name | Type | Default | Description |
|---|---|---|---|
figure (matplotlib.pyplot.Figure) | The figure to save. | ||
name | str | - | The name of the artifact. |
step | int, optional | None | The optional step stamp of the artifacts. If None, the artifacts is not considered as a step artifact. |
dpi | int | 200 | The dpi of the figure. |
extension | str | png | The extension of the figure. |
wait_response | bool | False | Set to true to upload sync. If False, the upload is made in background. |
run_id | str, optional | None | ID of the run. If None, the current run ID is used. |
project | str, optional | None | The project of the artifact. If None, the current project is used. |
Properties and local methods ​
Below are the properties that can be accessed from the run object as Run.property or run.property.
| Name | Type | Description |
|---|---|---|
id | Optional[str] | The ID of the run. |
rank | Optional[int] | The rank of the run. |
slug | Optional[str] | The slug of the run. |
category | str | The category of the run. |
config | dict | The configuration of the run. |
tmp_artifacts | str | The temporary directory where the artifacts are stored. |
Local methods are methods that are only available on the run object. They are not available globally.
get_id ​
Get the ID of the run. The functional alias is hl.get_run_id. If the id parameter is provided, it will return the provided ID. Otherwise, it will return the ID of the current run.
| Name | Type | Default | Description |
|---|---|---|---|
id | Optional[str] | The ID of the run. |
get_rank ​
Get the rank of the run. The functional alias is hl.get_run_rank. If the rank parameter is provided, it will return the provided rank. Otherwise, it will return the rank of the current run.
| Name | Type | Default | Description |
|---|---|---|---|
rank | Optional[int] | The rank of the run. |
get_slug ​
Get the slug of the run title. The functional alias is hl.get_run_slug. If the slug parameter is provided, it will return the provided slug. Otherwise, it will return the slug of the current run.
| Name | Type | Default | Description |
|---|---|---|---|
slug | Optional[str] | The slug of the run. |
ready ​
Check if the run is ready. The functional alias is hl.run_is_ready. It will return True if the run is ready and False otherwise.