Config

A Config object is an abstract object designed to store your configurations with a simple API.

Initialization

You can initialize a configuration with any type parameters

from pyhectiqlab import Config
config = Config(a=5, b="hello world", c=custom_object)

The initialization supports nested configurations.

dataset_config = Config(batch_size=128, shuffle=True)
model_config   = Config(layers=50)
config         = Config(dataset=dataset_config,
                        model=model_config)

You can convert an existing dict into a Config. If the argument is nested, the resulting Config with converts the nested dict into Config.

data = {"dataset": {"batch_size": 128, "sub": {"shuffle": True}}}
config = Config.from_dict(data)
# nested field `config.dataset.sub` is a `Config`

Modify the configuration

You can modify the state with __setattr__ and __setitem__ methods.

config.a = 2
config["a"] = 2 # Equivalent

Note that _lock and _state are protected attribute names used internally.

Lock and unlock

If you prefer to play on the safe side, you can activate a locked state where the attributes cannot be changed.

config.lock()
config.a = 5 # raise an Exception
config.unlock()
config.a = 5 # Succeed

Access the state

The state is accessible with a dot and dict-like notation, which are both supported for hierarchical parameters.

dataset_config = Config(batch_size=128, shuffle=True)
config = Config(dataset=dataset_config)

assert config.dataset.batch_size==128
assert config["dataset"]["batch_size"] == 128

The kwargs iterator **config is defined for a simple use in your methods.

def f(a, b):
    print(a, b)
config = Config(a="Hello", b="World")
f(**config)

If an attribute is queried but the state is absent, a None value is returned.

config = Config()
assert config.a == None

Saving and restoring

The config object can be pushed to the lab within a run context. A json dump will be made. Make sure the state is fully json encodable.

# Push the config to the lab
run.add_config(config)

You can fetch an existing configuration from a run with Config.download. The result is a Config object and a dict with the other elements had been pushed with run.add_meta.

from pyhectiqlab import Config
config, meta = Config.download(run="demo-run", project="lab/demo")

If you'd rather save it locally, config.save let's you do a json dump on your machine and load it back.

config.save("./config.json")
config = Config.load("./config.json")

Config template

A ConfigTemplate is a convenient object to support pydantic on your configuration classes. It is a direct children of pydantic.BaseModel and supports few useful methods such as help() and to_config().

We use the ConfigTemplate to support parameter validation with pydantic. Below is an example of how to use it

from pyhectiqlab import ConfigTemplate
from pydantic import Field, validator
from typing import List

class Params(ConfigTemplate):
    path: str = Field(default="./", description="The dataset directory")
    strides: List[int] = Field(description="List of model's strides")

    @validator("strides")
    def validate_strides(cls, value):
        assert len(value)==4, "Please provide 4 strides"
        return value

print(Params.help())
params = Params(strides=[2, 4, 8, 16])
hectiqlab_config = params.to_config()

Related methods