Lab documentation

Appearance

Changelog of the Python Client ​

This changelog lists the changes in the python client of pyhectiqlab.

[v3.1.14.dev0] - October 25, 2024 ​

  • Add method Run.add_current_notebook() (and hl.add_current_notebook()) to add the current notebook to the run. Currently, it's only working if the notebook is executed from vscode.
  • Support on_main_process of the library 🤗 Accelerate. If the library is used, most methods will work on the main process so you don't need to worry about it nor to change your code.
  • A class HectiqLabTracker from pyhectiqlab.tracker is available to track the progress of the training for 🤗 accelerate. It is compatible with the library Accelerate.

[v3.1.13] - October 25, 2024 ​

  • Fix issue when creating a model of a dataset using the client. It was possible that another model was attached to the run, which was not the desired behavior.
  • Set to debug the logging of the loading of the configuration file.
  • Remove the error logging on the first authentication.

[v3.1.12] - October 17, 2024 ​

  • Most errors are now swallowed by default. This mechanism is useful to prevent the Lab from breaking a script. To raise an error when it occurs, set the environment variable HECTIQLAB_RAISE_ERROR to 1.
  • When downloading a dataset or model to a path that includes a name and version, the specified directory is used directly, rather than creating a new subdirectory. For example, if the download path is path/to/dataset-1.1, the dataset will be downloaded to path/to/dataset-1.1 and not path/to/dataset-1.1/dataset-1.1.

[v3.1.11] - September 12, 2024 ​

  • Fix issue when downloading large datasets.

[v3.1.10] - August 1, 2024 ​

  • Add a 'Did you mean' feature to your CLI. If you make a typo in the command, the CLI will suggest the closest command to what you typed. For example:
>>> hectiq-lab dataset.crea
‼️  No such command 'dataset.crea'.

🔍 Did you mean one of these?
    Dataset.create
    Dataset.retrieve
    Dataset.update
    Dataset.delete
  • Automatically store your git diff of your tracked repository in the Lab. When you run a command, the Lab will store the diff of your repository in the Lab. This feature is enable by default. To disable it, set the environment variable HECTIQLAB_DO_NOT_STORE_DIFF to 1.

[v3.1.9] - July 24, 2024 ​

  • When a step completes, the step outputs in the logger the time it took to complete the step.
  • Fix a critical issue of status code.

[v3.1.8] - July 22, 2024 ​

DANGER

🚫 This version has been removed from distribution. This version contains a critical error that renders the code unusable. The bug doesn't impact the security of the lab. Upgrade to v3.1.9 for the fix.

  • Hide the error message when the request is 204 (on upload).

[v3.1.7] - July 18, 2024 ​

  • Fix the issue when uploading a single file in a dataset or in a model. The issue occurred when the path was a file and not a directory. s

[v3.1.6] - July 1, 2024 ​

  • Fix issue where allowing dirty state was not working properly. The default state lets the user use the lab even if the repository is dirty.
  • Fix issues with progress not found in the client.

[v3.1.5] - July 10, 2024 ​

  • Hot fix in dataset new CLI method.
  • Hot fix in authenticate.

[v3.1.3] - July 9, 2024 ​

  • Updated documentation.
  • Improved error messages and doc strings in Run, Dataset, Model and Step. Now, doc strings specify when a function logs errors and exits; also available in the documentation.
  • Environment variables have been moved from pyhectiqlab/settings/__init__.py to pyhectiqlab/settings/.env.public.
  • Dataset and Model can now be updated with new files via the function upload using the functional API and the CLI:
python
from pyhectiqlab import functional as hl, Model

# All lines below do the same thing
Model.upload(id="asda2a68197ad", path="path/to/model/files", project="hectiq-ai/demo") # object-oriented
hl.upload_model(id="asda2a68197ad", path="path/to/model/files", project="hectiq-ai/demo") # functional
!hectiq-lab Model.upload --id "asda2a68197ad" --path "path/to/model/files" --project hectiq-ai/demo # CLI

[v3.1.2] - July 4, 2024 ​

  • Message.publish now returns a dict if successful, and None otherwise.
  • Update of the progress tracking during uploads and downloads, allowing for multiple tracked tasks. Use the register_progress function to keep track of a rich.progress.Progress object defined at the root of a script.
python
import os
from rich.progress import Progress
from pyhectiqlab import functional as hl

os.environ["HECTIQLAB_HIDE_PROGRESS"] = False # ensures that progress is shown

with Progress() as prog:
    task = prog.add_task(total=3)
    hl.register_progress(prog)
    for _ in range(3):
        hl.download_dataset(...) # triggers the internal progress bar for the download

See documentation for more information.

  • Improved error messages at Dataset creation when project is not found, and at Tag attachment when no run is available.

[v3.1.1] - June 19, 2024 ​

  • Fix: When calling Project.allow_dirty, the environment variable HECTIQLAB_ALLOW_DIRTY is always called.
  • Adding functional method get_run_slug, which gets a slugified version of the current run title.
  • Update the Run documentation.

[v3.1.0] - June 18, 2024 ​

  • News feature: you can send messages to your runs. To publish a message, you can use the following command:
bash
hectiq-lab Message.publish --run "9sk7x6" --key "learning-rate" --value 0.001

and then receive the message in a run like this

python
import pyhectiqlab.functional as hl
message: Message.view = hl.get_message(key="learning-rate")
if message:
    message.ack() # Prevents the message from being retrieved again.
    print(message.key, message.value)

See the documentation for more information.

  • A bug concerning default parameters has been fixed in clisync==1.3.2 so that the requirements are now clisync>=1.3.2.
  • The documentation now contains a new section about the settings of the package.

[v3.0.18] - June 14, 2024 ​

  • Fix issue when the credentials are missing. When the credentials are missing, an error message is displayed and the offline mode is activated.

[v3.0.17] - June 13, 2024 ​

  • Renaming the environment variable HECTIQLAB_DEBUG_MODE to HECTIQLAB_OFFLINE_MODE. When set to True, no data is sent to the Lab.
  • Adding a section in the documentation about the properties of the Runs.

[v3.0.16] - June 12, 2024 ​

  • Adding HECTIQLAB_DEBUG_MODE to environment variables.
  • Adding a property Run.slug that returns the slug of the run. This slug may not be unique.

[v3.0.15] - June 11, 2024 ​

  • Fix minor issue with keep_latest_version in Dataset.list and Model.list.

[v3.0.14] - June 11, 2024 ​

  • Fix issues with some environment variables and better handling of those.
  • Fix issue with dirty package handling.
  • Adding rich progression bars. The TQDM package is no longer used.
  • The concept of blocks has been removed from the SDK. Hence, Block no longer exists.
  • Handling of synchronous / asynchronous processes when in Jupyter notebooks.
  • Increase unit test coverage.

[v3.0.13] - June 7, 2024 ​

  • Fix issue with Model.list and Dataset.list.
  • Adding the flag --generate to hectiq-lab authenticate to be able to generate an API key without storing it in the credentials file.
  • Fix issue with initialisation of runs.
  • Adding a parameter reuse to the creation of run (Run.create(), Run(), create_run). If reuse=True, the run will be reused if a run with the same title exists in the project. If reuse=False, a new run will be created. By default, reuse=False.

[v3.0.12] - June 7, 2024 ​

  • Fix download/upload in notebook
  • Centralization of env vars in settings.

[v3.0.11] - June 6, 2024 ​

  • Add error messages when the lab is failing silently (missing projects, etc.)
  • Fix include Step and Artifact.

[v3.0.9] - May 24, 2024 ​

  • Fix an issue with download_artifact.

[v3.0.8] - May 24, 2024 ​

  • Automatically storing in the Lab the packages versions at initialization of the run.
  • Setup the logging level to INFO by default for httpx.
  • Bug fix occuring with Run.retrieve_config_by_id.

[v3.0.7] - May 16, 2024 ​

  • Adding time out in download / upload
  • Warnings are raised with logging
  • Overwrite mechanics in dataset / model download
  • Hotfix in add_metric, functional mode, run categories, run context enter, run repr

[v3.0.5] - May 15, 2024 ​

  • Fix issues with --wait_response so that it is by default to True on the CLI.
  • Fix issues with model and dataset creation and downloads. The documentation has been updated to reflect the changes.

[v3.0.4] - May 8, 2024 ​

  • 🎉 Official release of the new v3 of the Lab.