Configurationο
This section describes the config
submodule of the API (esmvalcore.experimental
).
Configο
Configuration of ESMValCore/Tool is done via the Config
object.
The global configuration can be imported from the esmvalcore.experimental
module as CFG
:
>>> from esmvalcore.experimental import CFG
>>> CFG
Config({'auxiliary_data_dir': PosixPath('/home/user/auxiliary_data'),
'compress_netcdf': False,
'config_developer_file': None,
'config_file': PosixPath('/home/user/.esmvaltool/config-user.yml'),
'drs': {'CMIP5': 'default', 'CMIP6': 'default'},
'exit_on_warning': False,
'log_level': 'info',
'max_parallel_tasks': None,
'output_dir': PosixPath('/home/user/esmvaltool_output'),
'output_file_type': 'png',
'profile_diagnostic': False,
'remove_preproc_dir': True,
'rootpath': {'CMIP5': '~/default_inputpath',
'CMIP6': '~/default_inputpath',
'default': '~/default_inputpath'},
'save_intermediary_cubes': False)
The parameters for the user configuration file are listed here.
CFG
is essentially a python dictionary with a few extra functions, similar to matplotlib.rcParams
.
This means that values can be updated like this:
>>> CFG['output_dir'] = '~/esmvaltool_output'
>>> CFG['output_dir']
PosixPath('/home/user/esmvaltool_output')
Notice that CFG
automatically converts the path to an instance of pathlib.Path
and expands the home directory.
All values entered into the config are validated to prevent mistakes, for example, it will warn you if you make a typo in the key:
>>> CFG['output_directory'] = '~/esmvaltool_output'
InvalidConfigParameter: `output_directory` is not a valid config parameter.
Or, if the value entered cannot be converted to the expected type:
>>> CFG['max_parallel_tasks'] = 'π'
InvalidConfigParameter: Key `max_parallel_tasks`: Could not convert 'π' to int
Config
is also flexible, so it tries to correct the type of your input if possible:
>>> CFG['max_parallel_tasks'] = '8' # str
>>> type(CFG['max_parallel_tasks'])
int
By default, the config is loaded from the default location (/home/user/.esmvaltool/config-user.yml
).
If it does not exist, it falls back to the default values.
to load a different file:
>>> CFG.load_from_file('~/my-config.yml')
Or to reload the current config:
>>> CFG.reload()
Sessionο
Recipes and diagnostics will be run in their own directories.
This behaviour can be controlled via the Session
object.
A Session
can be initiated from the global Config
.
>>> session = CFG.start_session(name='my_session')
A Session
is very similar to the config.
It is also a dictionary, and copies all the keys from the Config
.
At this moment, session
is essentially a copy of CFG
:
>>> print(session == CFG)
True
>>> session['output_dir'] = '~/my_output_dir'
>>> print(session == CFG) # False
False
A Session
also knows about the directories where the data will stored.
The session name is used to prefix the directories.
>>> session.session_dir
/home/user/my_output_dir/my_session_20201203_155821
>>> session.run_dir
/home/user/my_output_dir/my_session_20201203_155821/run
>>> session.work_dir
/home/user/my_output_dir/my_session_20201203_155821/work
>>> session.preproc_dir
/home/user/my_output_dir/my_session_20201203_155821/preproc
>>> session.plot_dir
/home/user/my_output_dir/my_session_20201203_155821/plots
Unlike the global configuration, of which only one can exist, multiple sessions can be initiated from Config
.
API referenceο
ESMValTool config module.
- esmvalcore.experimental.config.CFGο
ESMValCore configuration. By default this will loaded from the file ~/.esmvaltool/config-user.yml.
Classes:
|
ESMValTool configuration object. |
|
Container class for session configuration and directory information. |
- class esmvalcore.experimental.config.Config(*args, **kwargs)[source]ο
ESMValTool configuration object.
Do not instantiate this class directly, but use
esmvalcore.experimental.CFG
instead.Methods:
load_from_file
(filename)Load user configuration from the given file.
reload
()Reload the config file.
start_session
(name)Start a new session from this configuration object.
- load_from_file(filename: Union[os.PathLike, str])[source]ο
Load user configuration from the given file.
- class esmvalcore.experimental.config.Session(config: dict, name: str = 'session')[source]ο
Container class for session configuration and directory information.
Do not instantiate this class directly, but use
CFG.start_session
instead.- Parameters
Attributes:
Return user config directory.
Return main log file.
Return main log debug file.
Return plot directory.
Return preproc directory.
Return run directory.
Return session directory.
Return work directory.
Methods:
from_config_user
(config_user)Convert config-user dict to API-compatible Session object.
set_session_name
([name])Set the name for the session.
Turn the Session object into a recipe-compatible dict.
- property config_dirο
Return user config directory.
- classmethod from_config_user(config_user: dict) esmvalcore.experimental.config._config_object.Session [source]ο
Convert config-user dict to API-compatible Session object.
For example, _recipe.Recipe._cfg.
- property main_logο
Return main log file.
- property main_log_debugο
Return main log debug file.
- property plot_dirο
Return plot directory.
- property preproc_dirο
Return preproc directory.
- relative_main_log = PosixPath('run/main_log.txt')ο
- relative_main_log_debug = PosixPath('run/main_log_debug.txt')ο
- relative_plot_dir = PosixPath('plots')ο
- relative_preproc_dir = PosixPath('preproc')ο
- relative_run_dir = PosixPath('run')ο
- relative_work_dir = PosixPath('work')ο
- property run_dirο
Return run directory.
- property session_dirο
Return session directory.
- set_session_name(name: str = 'session')[source]ο
Set the name for the session.
The name is used to name the session directory, e.g. session_20201208_132800/. The date is suffixed automatically.
- to_config_user() dict [source]ο
Turn the Session object into a recipe-compatible dict.
This dict is compatible with the config-user argument in
esmvalcore._recipe.Recipe
.
- property work_dirο
Return work directory.