Configuration¶

This section describes the config submodule of the API (esmvalcore.experimental).

Config¶

Configuration of ESMValCore/Tool is done via the Config object. The global configuration can be imported from the esmvalcore.experimental module as CFG:

>>> from esmvalcore.experimental import CFG
>>> CFG
Config({'auxiliary_data_dir': PosixPath('/home/user/auxiliary_data'),
        'compress_netcdf': False,
        'config_developer_file': None,
        'config_file': PosixPath('/home/user/.esmvaltool/config-user.yml'),
        'drs': {'CMIP5': 'default', 'CMIP6': 'default'},
        'exit_on_warning': False,
        'log_level': 'info',
        'max_parallel_tasks': None,
        'output_dir': PosixPath('/home/user/esmvaltool_output'),
        'output_file_type': 'png',
        'profile_diagnostic': False,
        'remove_preproc_dir': True,
        'rootpath': {'CMIP5': '~/default_inputpath',
                     'CMIP6': '~/default_inputpath',
                     'default': '~/default_inputpath'},
        'save_intermediary_cubes': False)

The parameters for the user configuration file are listed here.

CFG is essentially a python dictionary with a few extra functions, similar to matplotlib.rcParams. This means that values can be updated like this:

>>> CFG['output_dir'] = '~/esmvaltool_output'
>>> CFG['output_dir']
PosixPath('/home/user/esmvaltool_output')

Notice that CFG automatically converts the path to an instance of pathlib.Path and expands the home directory. All values entered into the config are validated to prevent mistakes, for example, it will warn you if you make a typo in the key:

>>> CFG['output_directory'] = '~/esmvaltool_output'
InvalidConfigParameter: `output_directory` is not a valid config parameter.

Or, if the value entered cannot be converted to the expected type:

>>> CFG['max_parallel_tasks'] = '🐜'
InvalidConfigParameter: Key `max_parallel_tasks`: Could not convert '🐜' to int

Config is also flexible, so it tries to correct the type of your input if possible:

>>> CFG['max_parallel_tasks'] = '8'  # str
>>> type(CFG['max_parallel_tasks'])
int

By default, the config is loaded from the default location (/home/user/.esmvaltool/config-user.yml). If it does not exist, it falls back to the default values. to load a different file:

>>> CFG.load_from_file('~/my-config.yml')

Or to reload the current config:

>>> CFG.reload()

Session¶

Recipes and diagnostics will be run in their own directories. This behaviour can be controlled via the Session object. A Session can be initiated from the global Config.

>>> session = CFG.start_session(name='my_session')

A Session is very similar to the config. It is also a dictionary, and copies all the keys from the Config. At this moment, session is essentially a copy of CFG:

>>> print(session == CFG)
True
>>> session['output_dir'] = '~/my_output_dir'
>>> print(session == CFG)  # False
False

A Session also knows about the directories where the data will stored. The session name is used to prefix the directories.

>>> session.session_dir
/home/user/my_output_dir/my_session_20201203_155821
>>> session.run_dir
/home/user/my_output_dir/my_session_20201203_155821/run
>>> session.work_dir
/home/user/my_output_dir/my_session_20201203_155821/work
>>> session.preproc_dir
/home/user/my_output_dir/my_session_20201203_155821/preproc
>>> session.plot_dir
/home/user/my_output_dir/my_session_20201203_155821/plots

Unlike the global configuration, of which only one can exist, multiple sessions can be initiated from Config.

API reference¶

ESMValTool config module.

esmvalcore.experimental.config.CFG¶: ESMValCore configuration. By default this will loaded from the file ~/.esmvaltool/config-user.yml.

Classes:

`Config`(args, *kwargs)	ESMValTool configuration object.
`Session`(config[, name])	Container class for session configuration and directory information.

class esmvalcore.experimental.config.Config(*args, **kwargs)[source]¶

ESMValTool configuration object.

Do not instantiate this class directly, but use esmvalcore.experimental.CFG instead.

Methods:

`load_from_file`(filename)	Load user configuration from the given file.
`reload`()	Reload the config file.
`start_session`(name)	Start a new session from this configuration object.

load_from_file(filename: Union[os.PathLike, str])[source]¶: Load user configuration from the given file.

reload()[source]¶: Reload the config file.

start_session(name: str)[source]¶

Start a new session from this configuration object.

Parameters: name (str) – Name of the session.
Return type: Session

class esmvalcore.experimental.config.Session(config: dict, name: str = 'session')[source]¶

Container class for session configuration and directory information.

Do not instantiate this class directly, but use CFG.start_session instead.

Parameters

config (dict) – Dictionary with configuration settings.
name (str) – Name of the session to initialize, for example, the name of the recipe (default=’session’).

Attributes:

`config_dir`	Return user config directory.
`main_log`	Return main log file.
`main_log_debug`	Return main log debug file.
`plot_dir`	Return plot directory.
`preproc_dir`	Return preproc directory.
`relative_main_log`
`relative_main_log_debug`
`relative_plot_dir`
`relative_preproc_dir`
`relative_run_dir`
`relative_work_dir`
`run_dir`	Return run directory.
`session_dir`	Return session directory.
`work_dir`	Return work directory.

Methods:

`from_config_user`(config_user)	Convert config-user dict to API-compatible Session object.
`set_session_name`([name])	Set the name for the session.
`to_config_user`()	Turn the Session object into a recipe-compatible dict.

property config_dir¶: Return user config directory.

classmethod from_config_user(config_user: dict) → esmvalcore.experimental.config._config_object.Session[source]¶

Convert config-user dict to API-compatible Session object.

For example, _recipe.Recipe._cfg.

property main_log¶: Return main log file.

property main_log_debug¶: Return main log debug file.

property plot_dir¶: Return plot directory.

property preproc_dir¶: Return preproc directory.

relative_main_log = PosixPath('run/main_log.txt')¶

relative_main_log_debug = PosixPath('run/main_log_debug.txt')¶

relative_plot_dir = PosixPath('plots')¶

relative_preproc_dir = PosixPath('preproc')¶

relative_run_dir = PosixPath('run')¶

relative_work_dir = PosixPath('work')¶

property run_dir¶: Return run directory.

property session_dir¶: Return session directory.

session_name: Optional[str]¶

set_session_name(name: str = 'session')[source]¶

Set the name for the session.

The name is used to name the session directory, e.g. session_20201208_132800/. The date is suffixed automatically.

to_config_user() → dict[source]¶

Turn the Session object into a recipe-compatible dict.

This dict is compatible with the config-user argument in esmvalcore._recipe.Recipe.

property work_dir¶: Return work directory.