Configuration

This section describes the config submodule of the API (esmvalcore.experimental).

Config

Configuration of ESMValCore/Tool is done via the Config object. The global configuration can be imported from the esmvalcore.experimental module as CFG:

>>> from esmvalcore.experimental import CFG
>>> CFG
Config({'auxiliary_data_dir': PosixPath('/home/user/auxiliary_data'),
        'compress_netcdf': False,
        'config_developer_file': None,
        'config_file': PosixPath('/home/user/.esmvaltool/config-user.yml'),
        'drs': {'CMIP5': 'default', 'CMIP6': 'default'},
        'exit_on_warning': False,
        'log_level': 'info',
        'max_parallel_tasks': None,
        'output_dir': PosixPath('/home/user/esmvaltool_output'),
        'output_file_type': 'png',
        'profile_diagnostic': False,
        'remove_preproc_dir': True,
        'rootpath': {'CMIP5': '~/default_inputpath',
                     'CMIP6': '~/default_inputpath',
                     'default': '~/default_inputpath'},
        'save_intermediary_cubes': False,
        'write_netcdf': True,
        'write_plots': True})

The parameters for the user configuration file are listed here.

CFG is essentially a python dictionary with a few extra functions, similar to matplotlib.rcParams. This means that values can be updated like this:

>>> CFG['output_dir'] = '~/esmvaltool_output'
>>> CFG['output_dir']
PosixPath('/home/user/esmvaltool_output')

Notice that CFG automatically converts the path to an instance of pathlib.Path and expands the home directory. All values entered into the config are validated to prevent mistakes, for example, it will warn you if you make a typo in the key:

>>> CFG['output_directory'] = '~/esmvaltool_output'
InvalidConfigParameter: `output_directory` is not a valid config parameter.

Or, if the value entered cannot be converted to the expected type:

>>> CFG['max_parallel_tasks'] = '🐜'
InvalidConfigParameter: Key `max_parallel_tasks`: Could not convert '🐜' to int

Config is also flexible, so it tries to correct the type of your input if possible:

>>> CFG['max_parallel_tasks'] = '8'  # str
>>> type(CFG['max_parallel_tasks'])
int

By default, the config is loaded from the default location (/home/user/.esmvaltool/config-user.yml). If it does not exist, it falls back to the default values. to load a different file:

>>> CFG.load_from_file('~/my-config.yml')

Or to reload the current config:

>>> CFG.reload()

Session

Recipes and diagnostics will be run in their own directories. This behaviour can be controlled via the Session object. A Session can be initiated from the global Config.

>>> session = CFG.start_session(name='my_session')

A Session is very similar to the config. It is also a dictionary, and copies all the keys from the Config. At this moment, session is essentially a copy of CFG:

>>> print(session == CFG)
True
>>> session['output_dir'] = '~/my_output_dir'
>>> print(session == CFG)  # False
False

A Session also knows about the directories where the data will stored. The session name is used to prefix the directories.

>>> session.session_dir
/home/user/my_output_dir/my_session_20201203_155821
>>> session.run_dir
/home/user/my_output_dir/my_session_20201203_155821/run
>>> session.work_dir
/home/user/my_output_dir/my_session_20201203_155821/work
>>> session.preproc_dir
/home/user/my_output_dir/my_session_20201203_155821/preproc
>>> session.plot_dir
/home/user/my_output_dir/my_session_20201203_155821/plots

Unlike the global configuration, of which only one can exist, multiple sessions can be initiated from Config.

API reference

ESMValTool config module.

esmvalcore.experimental.config.CFG

ESMValCore configuration. By default this will loaded from the file ~/.esmvaltool/config-user.yml.

Classes:

Config(*args, **kwargs)

ESMValTool configuration object.

Session(config, name)

Container class for session configuration and directory information.

class esmvalcore.experimental.config.Config(*args, **kwargs)[source]

ESMValTool configuration object.

Do not instantiate this class directly, but use esmvalcore.experimental.CFG instead.

Methods:

load_from_file(filename)

Load user configuration from the given file.

reload()

Reload the config file.

start_session(name)

Start a new session from this configuration object.

load_from_file(filename)[source]

Load user configuration from the given file.

reload()[source]

Reload the config file.

start_session(name: str)[source]

Start a new session from this configuration object.

Parameters

name (str) – Name of the session.

Returns

Return type

Session

class esmvalcore.experimental.config.Session(config: dict, name: str = 'session')[source]

Container class for session configuration and directory information.

Do not instantiate this class directly, but use CFG.start_session instead.

Parameters
  • config (dict) – Dictionary with configuration settings.

  • name (str) – Name of the session to initialize, for example, the name of the recipe (default=’session’).

Attributes:

config_dir

Return user config directory.

main_log

Return main log file.

main_log_debug

Return main log debug file.

plot_dir

Return plot directory.

preproc_dir

Return preproc directory.

relative_main_log

relative_main_log_debug

relative_plot_dir

relative_preproc_dir

relative_run_dir

relative_work_dir

run_dir

Return run directory.

session_dir

Return session directory.

work_dir

Return work directory.

Methods:

from_config_user(config_user)

Convert config-user dict to API-compatible Session object.

set_session_name([name])

Set the name for the session.

to_config_user()

Turn the Session object into a recipe-compatible dict.

property config_dir

Return user config directory.

classmethod from_config_user(config_user: dict)esmvalcore.experimental.config._config_object.Session[source]

Convert config-user dict to API-compatible Session object.

For example, _recipe.Recipe._cfg.

property main_log

Return main log file.

property main_log_debug

Return main log debug file.

property plot_dir

Return plot directory.

property preproc_dir

Return preproc directory.

relative_main_log = PosixPath('run/main_log.txt')
relative_main_log_debug = PosixPath('run/main_log_debug.txt')
relative_plot_dir = PosixPath('plots')
relative_preproc_dir = PosixPath('preproc')
relative_run_dir = PosixPath('run')
relative_work_dir = PosixPath('work')
property run_dir

Return run directory.

property session_dir

Return session directory.

set_session_name(name: str = 'session')[source]

Set the name for the session.

The name is used to name the session directory, e.g. session_20201208_132800/. The date is suffixed automatically.

to_config_user()dict[source]

Turn the Session object into a recipe-compatible dict.

This dict is compatible with the config-user argument in esmvalcore._recipe.Recipe.

property work_dir

Return work directory.