Dataset

Classes and functions for defining, finding, and loading data.

Classes:

Dataset(**facets)

Define datasets, find the related files, and load them.

Functions:

datasets_to_recipe(datasets[, recipe])

Create or update a recipe from datasets.

class esmvalcore.dataset.Dataset(**facets: Union[str, Sequence[str], Number])[source]

Define datasets, find the related files, and load them.

Parameters: **facets – Facets describing the dataset. See esmvalcore.esgf.facets.FACETS for the mapping between the facet names used by ESMValCore and those used on ESGF.

ancillaries

List of ancillary datasets.

Type: list[Dataset]

facets

Facets describing the dataset.

Type: esmvalcore.typing.Facets

Methods:

`add_ancillary`(**facets)	Add an ancillary dataset.
`augment_facets`()	Add extra facets.
`copy`(**facets)	Create a copy.
`find_files`()	Find files.
`from_files`()	Create datasets based on the available files.
`from_ranges`()	Create a list of datasets from short notations.
`load`()	Load dataset.
`set_facet`(key, value[, persist])	Set facet.
`set_version`()	Set the `'version'` facet based on the available data.
`summary`([shorten])	Summarize the content of dataset.

Attributes:

`files`	The files associated with this dataset.
`minimal_facets`	Return a dictionary with the persistent facets.
`session`	A `esmvalcore.config.Session` associated with the dataset.

add_ancillary(**facets: Union[str, Sequence[str], Number]) → None[source]

Add an ancillary dataset.

This is a convenience function that will create a copy of the current dataset, update its facets with the values specified in **facets, and append it to Dataset.ancillaries. For more control over the creation of the ancillary dataset, first create a new Dataset describing the ancillary dataset and then append it to Dataset.ancillaries.

Parameters: **facets – Facets describing the ancillary variable.

augment_facets() → None[source]

Add extra facets.

This function will update the dataset with additional facets from various sources.

copy(**facets: Union[str, Sequence[str], Number]) → Dataset[source]

Create a copy.

Parameters: **facets – Update these facets in the copy. Note that for ancillary datasets attached to the dataset, the 'short_name' and 'mip' facets will not be updated with these values.
Returns: A copy of the dataset.
Return type: Dataset

property files: Sequence[Union[ESGFFile, LocalFile]]: The files associated with this dataset.

find_files() → None[source]

Find files.

Look for files and populate the Dataset.files property of the dataset and its ancillary datasets.

from_files() → Iterator[Dataset][source]

Create datasets based on the available files.

Yields: Dataset – Datasets representing the available files.

from_ranges() → list['Dataset'][source]

Create a list of datasets from short notations.

This expands the 'ensemble' and 'sub_experiment' facets in the dataset definition if they are ranges.

For example 'ensemble'='r(1:3)i1p1f1' will be expanded to three datasets, with 'ensemble' values 'r1i1p1f1', 'r2i1p1f1', 'r3i1p1f1'.

Returns: The datasets.
Return type: list[Dataset]

load() → Cube[source]

Load dataset.

Raises: InputFilesNotFound – When no files were found.
Returns: An iris cube with the data corresponding the the dataset.
Return type: iris.cube.Cube

property minimal_facets: Dict[str, Union[str, Sequence[str], Number]]: Return a dictionary with the persistent facets.

property session: Session: A esmvalcore.config.Session associated with the dataset.

set_facet(key: str, value: Union[str, Sequence[str], Number], persist: bool = True)[source]

Set facet.

Parameters

key – The name of the facet.
value – The value of the facet.
persist – When writing a dataset to a recipe, only persistent facets will get written.

set_version() → None[source]: Set the 'version' facet based on the available data.

summary(shorten: bool = False) → str[source]

Summarize the content of dataset.

Parameters: shorten – Shorten the summary.
Returns: A summary describing the dataset.
Return type: str

esmvalcore.dataset.datasets_to_recipe(datasets: Iterable[Dataset], recipe: dict | None = None) → dict[source]

Create or update a recipe from datasets.

Parameters

datasets – Datasets to use in the recipe.
recipe – If provided, the datasets in the recipe will be replaced. The value provided here should be a recipe file that is loaded using e.g. yaml.safe_load().

Examples

See Composing recipes for example use cases.

Returns: The recipe with the datasets. To convert the dict to a recipe, use e.g. yaml.safe_dump().
Return type: dict
Raises: RecipeError – Raised when a dataset is missing the diagnostic facet.