Configuration#
This section describes the config
module.
Config#
Configuration of ESMValCore/Tool is done via the Config
object.
The global configuration can be imported from the esmvalcore.config
module as CFG
:
>>> from esmvalcore.config import CFG
>>> CFG
Config({'auxiliary_data_dir': PosixPath('/home/user/auxiliary_data'),
'compress_netcdf': False,
'config_developer_file': None,
'config_file': PosixPath('/home/user/.esmvaltool/config-user.yml'),
'drs': {'CMIP5': 'default', 'CMIP6': 'default'},
'exit_on_warning': False,
'log_level': 'info',
'max_parallel_tasks': None,
'output_dir': PosixPath('/home/user/esmvaltool_output'),
'output_file_type': 'png',
'profile_diagnostic': False,
'remove_preproc_dir': True,
'rootpath': {'CMIP5': '~/default_inputpath',
'CMIP6': '~/default_inputpath',
'default': '~/default_inputpath'},
'save_intermediary_cubes': False)
The parameters for the user configuration file are listed here.
CFG
is essentially a python dictionary with a few extra functions, similar to matplotlib.rcParams
.
This means that values can be updated like this:
>>> CFG['output_dir'] = '~/esmvaltool_output'
>>> CFG['output_dir']
PosixPath('/home/user/esmvaltool_output')
Notice that CFG
automatically converts the path to an instance of pathlib.Path
and expands the home directory.
All values entered into the config are validated to prevent mistakes, for example, it will warn you if you make a typo in the key:
>>> CFG['output_directory'] = '~/esmvaltool_output'
InvalidConfigParameter: `output_directory` is not a valid config parameter.
Or, if the value entered cannot be converted to the expected type:
>>> CFG['max_parallel_tasks'] = '🐜'
InvalidConfigParameter: Key `max_parallel_tasks`: Could not convert '🐜' to int
Config
is also flexible, so it tries to correct the type of your input if possible:
>>> CFG['max_parallel_tasks'] = '8' # str
>>> type(CFG['max_parallel_tasks'])
int
By default, the config is loaded from the default location (/home/user/.esmvaltool/config-user.yml
).
If it does not exist, it falls back to the default values.
to load a different file:
>>> CFG.load_from_file('~/my-config.yml')
Or to reload the current config:
>>> CFG.reload()
Session#
Recipes and diagnostics will be run in their own directories.
This behaviour can be controlled via the Session
object.
A Session
can be initiated from the global Config
.
>>> session = CFG.start_session(name='my_session')
A Session
is very similar to the config.
It is also a dictionary, and copies all the keys from the Config
.
At this moment, session
is essentially a copy of CFG
:
>>> print(session == CFG)
True
>>> session['output_dir'] = '~/my_output_dir'
>>> print(session == CFG) # False
False
A Session
also knows about the directories where the data will stored.
The session name is used to prefix the directories.
>>> session.session_dir
/home/user/my_output_dir/my_session_20201203_155821
>>> session.run_dir
/home/user/my_output_dir/my_session_20201203_155821/run
>>> session.work_dir
/home/user/my_output_dir/my_session_20201203_155821/work
>>> session.preproc_dir
/home/user/my_output_dir/my_session_20201203_155821/preproc
>>> session.plot_dir
/home/user/my_output_dir/my_session_20201203_155821/plots
Unlike the global configuration, of which only one can exist, multiple sessions can be initiated from Config
.
API reference#
Configuration module.
- esmvalcore.config.CFG#
ESMValCore configuration.
By default this will be loaded from the file
~/.esmvaltool/config-user.yml
.
Classes:
|
ESMValTool configuration object. |
|
Container class for session configuration and directory information. |
- class esmvalcore.config.Config(*args, **kwargs)[source]#
ESMValTool configuration object.
Do not instantiate this class directly, but use
esmvalcore.config.CFG
instead.Methods:
load_from_file
([filename])Load user configuration from the given file.
reload
()Reload the config file.
start_session
(name)Start a new session from this configuration object.
- class esmvalcore.config.Session(config: dict, name: str = 'session')[source]#
Container class for session configuration and directory information.
Do not instantiate this class directly, but use
CFG.start_session
instead.- Parameters:
Attributes:
Return user config directory.
Return main log file.
Return main log debug file.
Return plot directory.
Return preproc directory.
Return run directory.
Return session directory.
Return work directory.
Methods:
from_config_user
(config_user)Convert config-user dict to API-compatible Session object.
set_session_name
([name])Set the name for the session.
Turn the Session object into a recipe-compatible dict.
- property config_dir#
Return user config directory.
- classmethod from_config_user(config_user: dict) Session [source]#
Convert config-user dict to API-compatible Session object.
Deprecated since v2.8.0, scheduled for removal in v2.9.0.
For example, _recipe.Recipe._cfg.
- property main_log#
Return main log file.
- property main_log_debug#
Return main log debug file.
- property plot_dir#
Return plot directory.
- property preproc_dir#
Return preproc directory.
- relative_main_log = PosixPath('run/main_log.txt')#
- relative_main_log_debug = PosixPath('run/main_log_debug.txt')#
- relative_plot_dir = PosixPath('plots')#
- relative_preproc_dir = PosixPath('preproc')#
- relative_run_dir = PosixPath('run')#
- relative_work_dir = PosixPath('work')#
- property run_dir#
Return run directory.
- property session_dir#
Return session directory.
- set_session_name(name: str = 'session')[source]#
Set the name for the session.
The name is used to name the session directory, e.g. session_20201208_132800/. The date is suffixed automatically.
- to_config_user() dict [source]#
Turn the Session object into a recipe-compatible dict.
Deprecated since v2.8.0, scheduled for removal in v2.9.0.
This dict is compatible with the config-user argument in
esmvalcore._recipe.Recipe
.
- property work_dir#
Return work directory.