runs

Package versions and git

Package version and git tracking is an important aspect of reproductibility in machine learning. Hectiqlab helps you track the versions using a frictionless method (a hard method would be to create a Docker images on every run).

System and packages

Create a run.

from pyhectiqlab import Run
run = Run(name="Hello world", project="lab/demo")

Import your packages and store the versions.

import numpy as np 
import torch
import my_customer_package

run.add_package_versions(globals())

By default, run.add_package_versions(globals()) will store the versions of the packages in memory, your python version, some system features and the git state of the run (if any). Note that this method is different from a pip freeze as it does not log all the packages installed in your environment, but only the imported packages.

Custom git repo

If you are using a package from a git repertoire, you can log the git state of this specific package.

# It's not required to import the package before using add_package_repo_state
import my_custom_package 
run.add_package_repo_state("my_custom_package")

The run will log the git state (commit, branch, remote url) of my_custom_package.__path__.

Methods

pyhectiqlab.Run

Run.add_package_versions(packages: dict, with_sys: bool = True, with_python: bool = True, with_git: bool = True)

Log the current states of your python packages, your system specifications, and git repo.

Parameters

Property	Type	Default	Description
`packages`	`dict`	`-`	A dict with keys are names of package and values are modules. We recommand using `globals()` that stores all the packages in memory.
`with_sys`	`bool`	`True`	If `True`, the system state is logged (Compiler, OS, Machine, Hostname, etc.).
`with_python`	`bool`	`True`	If `True`, the python version is logged.
`with_git`	`bool`	`True`	If `True` and the execution script is locatted in a git repo, then the git state will be stored (hash, origin, branch).

Run.add_package_repo_state(package: str)

Log the current git state of the package at its __path__.

Parameters

Property	Type	Default	Description
`package`	`str`	`-`	Name of the target package.