Skip to main content
Version: Next

Config Search Path

The Config Search Path is a list of paths that Hydra searches in order to find non-primary configs. It is similar to the Python PYTHONPATH.

  • When a config is requested, The first matching config in the search path is used.
  • Each search path element has a schema prefix such as file:// or pkg:// that corresponds to a ConfigSourcePlugin.
    • file:// points to a file system path. It can either be an absolute path or a relative path. Relative path will be resolved to absolute based on the current working dir. Path separator is / on all Operating Systems.
    • pkg:// points to an importable Python module, with . being the separator. __init__.py files are needed in directories for Python to treat them as packages.

You can inspect the search path and the configurations loaded by Hydra via the --info flag:

$ python my_app.py --info searchpath

There are a few ways to modify the config search path, enabling Hydra to access configuration in different locations. Use a combination of the methods described below:

Using @hydra.main()​

Using the config_path parameter @hydra.main(). The config_path is relative to location of the Python script.

Overriding hydra.searchpath config​

Β Example (Click Here)

In some cases you may want to add multiple locations to the search path. For example, an app may want to read the configs from an additional Python module or an additional directory on the file system. Another example is in unit testing, where the defaults list in a config loaded from the tests/configs folder may make reference to another config from the app/configs folder. If the config_path or config_dir argument passed to @hydra.main or to one of the initialization methods points to tests/configs, the configs located in app/configs will not be discoverable unless Hydra's search path is modified.

You can configure hydra.searchpath in your primary config or from the command line.

info

hydra.searchpath can only be configured in the primary config. Attempting to configure it in other configs will result in an error.

In this example, we add a second config directory - additional_conf, next to the first config directory:

β”œβ”€β”€ __init__.py
β”œβ”€β”€ conf
β”‚Β Β  β”œβ”€β”€ config.yaml
β”‚Β Β  └── dataset
β”‚Β Β  └── cifar10.yaml
β”œβ”€β”€ additional_conf
β”‚Β Β  β”œβ”€β”€ __init__.py
β”‚Β Β  └── dataset
β”‚Β Β  └── imagenet.yaml
└── my_app.py
my_app.py

@hydra.main(version_base=None, config_path="conf", config_name="config")
def my_app(cfg: DictConfig) -> None:
print(OmegaConf.to_yaml(cfg))


if __name__ == "__main__":
my_app()

conf/config.yaml is the primary config for my_app.py, config groups cifar10 and imagenet are under different folders. We can add additional_conf to hydra.searchpath for Hydra to discover dataset/imagenet.

config.yaml
defaults:
- dataset: cifar10

hydra:
searchpath:
- pkg://additional_conf
# You can also use file based schema:
# - file:///etc/my_app
# - file://${oc.env:HOME}/.my_app
my_app.py output
dataset:
name: cifar10
path: /datasets/cifar10






Overriding dataset=imagenet from the commandline:

command line override
python my_app.py dataset=imagenet


my_app.py output
dataset:
name: imagenet
path: /datasets/imagenet

hydra.searchpath can be defined or overridden via the command line as well:

command line override
python my_app.py 'hydra.searchpath=[pkg://additional_conf]'

Overriding --config-dir from the command line​

This is a less flexible alternative to hydra.searchpath. See this page for more info.

Creating a SearchPathPlugin​

Β Example (Click Here)

Framework authors may want to add their configs to the search path automatically once their package is installed, eliminating the need for any actions from the users. This can be achieved using a SearchPathPlugin. Check the example plugin linked above for more details.