# Dataset

Classes and functions for defining, finding, and loading data.

**Classes:**

| [`Dataset`](#esmvalcore.dataset.Dataset)(\*\*facets)   | Define datasets, find the related files, and load them.   |
|------------------------------------------------------------------------|-----------------------------------------------------------|

**Data:**

| [`INHERITED_FACETS`](#esmvalcore.dataset.INHERITED_FACETS)   | Inherited facets.   |
|---------------------------------------------------------------------|---------------------|

**Functions:**

| [`datasets_to_recipe`](#esmvalcore.dataset.datasets_to_recipe)(datasets[, recipe])   | Create or update a recipe from datasets.   |
|-------------------------------------------------------------------------------------------|--------------------------------------------|

### *class* esmvalcore.dataset.Dataset(\*\*facets: FacetValue)

Define datasets, find the related files, and load them.

* **Parameters:**
  **\*\*facets** (*FacetValue*) – Facets describing the dataset. See [Facets](../reference/facets.html.md#facets) for the mapping between
  the facet names used by ESMValCore and those used on ESGF and
  [Variables and CMOR Tables](../reference/cmor_tables.html.md#cmor-tables) to find out which variables are available.

#### supplementaries

List of supplementary datasets.

* **Type:**
  [list](https://docs.python.org/3/builtins/stdtypes.html#list)[[Dataset](#esmvalcore.dataset.Dataset)]

#### facets

Facets describing the dataset.

* **Type:**
  `esmvalcore.typing.Facets`

**Methods:**

| [`add_supplementary`](#esmvalcore.dataset.Dataset.add_supplementary)(\*\*facets)    | Add an supplementary dataset.                          |
|-----------------------------------------------------------------------------------|--------------------------------------------------------|
| [`augment_facets`](#esmvalcore.dataset.Dataset.augment_facets)()                 | Add additional facets.                                 |
| [`copy`](#esmvalcore.dataset.Dataset.copy)(\*\*facets)                 | Create a copy.                                         |
| [`find_files`](#esmvalcore.dataset.Dataset.find_files)()                     | Find files.                                            |
| [`from_files`](#esmvalcore.dataset.Dataset.from_files)()                     | Create datasets based on the available files.          |
| [`from_ranges`](#esmvalcore.dataset.Dataset.from_ranges)()                    | Create a list of datasets from short notations.        |
| [`from_recipe`](#esmvalcore.dataset.Dataset.from_recipe)(recipe, session)     | Read datasets from a recipe.                           |
| [`load`](#esmvalcore.dataset.Dataset.load)()                           | Load dataset.                                          |
| [`set_facet`](#esmvalcore.dataset.Dataset.set_facet)(key, value[, persist]) | Set facet.                                             |
| [`set_version`](#esmvalcore.dataset.Dataset.set_version)()                    | Set the `'version'` facet based on the available data. |
| [`summary`](#esmvalcore.dataset.Dataset.summary)([shorten])               | Summarize the content of dataset.                      |

**Attributes:**

| [`files`](#esmvalcore.dataset.Dataset.files)          | The files associated with this dataset.                                                                                             |
|-----------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|
| [`minimal_facets`](#esmvalcore.dataset.Dataset.minimal_facets) | Return a dictionary with the persistent facets.                                                                                     |
| [`session`](#esmvalcore.dataset.Dataset.session)        | A [`esmvalcore.config.Session`](esmvalcore.config.html.md#esmvalcore.config.Session) associated with the dataset. |

#### add_supplementary(\*\*facets: FacetValue) → [None](https://docs.python.org/3/builtins/constants.html#None)

Add an supplementary dataset.

This is a convenience function that will create a copy of the current
dataset, update its facets with the values specified in `**facets`,
and append it to [`Dataset.supplementaries`](#esmvalcore.dataset.Dataset.supplementaries). For more control
over the creation of the supplementary dataset, first create a new
[`Dataset`](#esmvalcore.dataset.Dataset) describing the supplementary dataset and then append
it to [`Dataset.supplementaries`](#esmvalcore.dataset.Dataset.supplementaries).

* **Parameters:**
  **\*\*facets** (*FacetValue*) – Facets describing the supplementary variable.
* **Return type:**
  None

#### augment_facets() → [None](https://docs.python.org/3/builtins/constants.html#None)

Add additional facets.

This function will update the dataset with additional facets from
various sources. These include [Extra Facets](../quickstart/configure.html.md#config-extra-facets) as well as
facets read from the controlled voculary included in the CMOR tables
if applicable.

* **Return type:**
  None

#### copy(\*\*facets: FacetValue) → [Dataset](#esmvalcore.dataset.Dataset)

Create a copy.

* **Parameters:**
  **\*\*facets** (*FacetValue*) – Update these facets in the copy. Note that for supplementary
  datasets attached to the dataset, the `'short_name'` and
  `'mip'` facets will not be updated with these values.
* **Returns:**
  A copy of the dataset.
* **Return type:**
  [Dataset](#esmvalcore.dataset.Dataset)

#### *property* files *: [list](https://docs.python.org/3/builtins/stdtypes.html#list)[[DataElement](esmvalcore.io.protocol.html.md#esmvalcore.io.protocol.DataElement)]*

The files associated with this dataset.

#### find_files() → [None](https://docs.python.org/3/builtins/constants.html#None)

Find files.

Look for files and populate the [`Dataset.files`](#esmvalcore.dataset.Dataset.files) property of
the dataset and its supplementary datasets.

* **Return type:**
  None

#### from_files() → Iterator[[Dataset](#esmvalcore.dataset.Dataset)]

Create datasets based on the available files.

The facet values for local files are retrieved from the directory tree
where the directories represent the facets values.
See [CMIP data](../quickstart/find_data.html.md#cmor-drs) for more information on this kind of file
organization.

[`glob.glob()`](https://docs.python.org/3/library/glob.html#glob.glob) patterns can be used as facet values to select
multiple datasets.
If for some of the datasets not all glob patterns can be expanded
(e.g. because the required facet values cannot be inferred from the
directory names), these datasets will be ignored, unless this happens
to be all datasets.

If [`glob.glob()`](https://docs.python.org/3/library/glob.html#glob.glob) patterns are used in supplementary variables and
multiple matching datasets are found, only the supplementary dataset
that has most facets in common with the main dataset will be attached.

Supplementary datasets will in inherit the facet values from the main
dataset for those facets listed in [`INHERITED_FACETS`](#esmvalcore.dataset.INHERITED_FACETS).

### Examples

See [Discovering data](../notebooks/discovering-data.html.md) notebook for example use cases.

* **Yields:**
  *Dataset* – Datasets representing the available files.
* **Return type:**
  Iterator[[Dataset](#esmvalcore.dataset.Dataset)]

#### from_ranges() → [list](https://docs.python.org/3/builtins/stdtypes.html#list)[[Dataset](#esmvalcore.dataset.Dataset)]

Create a list of datasets from short notations.

This expands the `'ensemble'` and `'sub_experiment'` facets in the
dataset definition if they are ranges.

For example `'ensemble'='r(1:3)i1p1f1'` will be expanded to
three datasets, with `'ensemble'` values `'r1i1p1f1'`,
`'r2i1p1f1'`, `'r3i1p1f1'`.

* **Returns:**
  The datasets.
* **Return type:**
  [list](https://docs.python.org/3/builtins/stdtypes.html#list)[[Dataset](#esmvalcore.dataset.Dataset)]

#### *static* from_recipe(recipe: Path | [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [dict](https://docs.python.org/3/builtins/stdtypes.html#dict), session: [Session](esmvalcore.config.html.md#esmvalcore.config.Session)) → [list](https://docs.python.org/3/builtins/stdtypes.html#list)[[Dataset](#esmvalcore.dataset.Dataset)]

Read datasets from a recipe.

* **Parameters:**
  * **recipe** (*Path* *|* [*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *|* [*dict*](https://docs.python.org/3/builtins/stdtypes.html#dict)) – [Recipe](../recipe/index.html.md#recipe) to load the datasets from. The value
    provided here should be either a path to a file, a recipe file
    that has been loaded using e.g. `yaml.safe_load()`, or an
    [`str`](https://docs.python.org/3/builtins/stdtypes.html#str) that can be loaded using `yaml.safe_load()`.
  * **session** ([*Session*](esmvalcore.config.html.md#esmvalcore.config.Session)) – Datasets to use in the recipe.
* **Returns:**
  A list of datasets.
* **Return type:**
  [list](https://docs.python.org/3/builtins/stdtypes.html#list)[[Dataset](#esmvalcore.dataset.Dataset)]

#### load() → Cube

Load dataset.

* **Raises:**
  [**InputFilesNotFound**](esmvalcore.exceptions.html.md#esmvalcore.exceptions.InputFilesNotFound) – When no files were found.
* **Returns:**
  An [`iris`](https://scitools-iris.readthedocs.io/en/stable/generated/api/iris.html#module-iris) cube with the data corresponding the the dataset.
* **Return type:**
  [iris.cube.Cube](https://scitools-iris.readthedocs.io/en/stable/generated/api/iris.cube.html#iris.cube.Cube)

#### *property* minimal_facets *: Facets*

Return a dictionary with the persistent facets.

#### *property* session *: [Session](esmvalcore.config.html.md#esmvalcore.config.Session)*

A [`esmvalcore.config.Session`](esmvalcore.config.html.md#esmvalcore.config.Session) associated with the dataset.

#### set_facet(key: [str](https://docs.python.org/3/builtins/stdtypes.html#str), value: FacetValue, persist: [bool](https://docs.python.org/3/builtins/functions.html#bool) = True) → [None](https://docs.python.org/3/builtins/constants.html#None)

Set facet.

* **Parameters:**
  * **key** ([*str*](https://docs.python.org/3/builtins/stdtypes.html#str)) – The name of the facet.
  * **value** (*FacetValue*) – The value of the facet.
  * **persist** ([*bool*](https://docs.python.org/3/builtins/functions.html#bool)) – When writing a dataset to a recipe, only persistent facets
    will get written.
* **Return type:**
  None

#### set_version() → [None](https://docs.python.org/3/builtins/constants.html#None)

Set the `'version'` facet based on the available data.

* **Return type:**
  None

#### summary(shorten: [bool](https://docs.python.org/3/builtins/functions.html#bool) = False) → [str](https://docs.python.org/3/builtins/stdtypes.html#str)

Summarize the content of dataset.

* **Parameters:**
  **shorten** ([*bool*](https://docs.python.org/3/builtins/functions.html#bool)) – Shorten the summary.
* **Returns:**
  A summary describing the dataset.
* **Return type:**
  [str](https://docs.python.org/3/builtins/stdtypes.html#str)

### esmvalcore.dataset.INHERITED_FACETS *: [list](https://docs.python.org/3/builtins/stdtypes.html#list)[[str](https://docs.python.org/3/builtins/stdtypes.html#str)]* *= ['dataset', 'region', 'domain', 'driver', 'grid', 'project', 'timerange']*

Inherited facets.

Supplementary datasets created based on the available files using the
[`Dataset.from_files()`](#esmvalcore.dataset.Dataset.from_files) method will inherit the values of these facets from
the main dataset.

### esmvalcore.dataset.datasets_to_recipe(datasets: Iterable[[Dataset](#esmvalcore.dataset.Dataset)], recipe: Path | [str](https://docs.python.org/3/builtins/stdtypes.html#str) | [dict](https://docs.python.org/3/builtins/stdtypes.html#dict)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), Any] | [None](https://docs.python.org/3/builtins/constants.html#None) = None) → [dict](https://docs.python.org/3/builtins/stdtypes.html#dict)

Create or update a recipe from datasets.

* **Parameters:**
  * **datasets** (*Iterable* *[*[*Dataset*](#esmvalcore.dataset.Dataset) *]*) – Datasets to use in the recipe.
  * **recipe** (*Path* *|* [*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *|* [*dict*](https://docs.python.org/3/builtins/stdtypes.html#dict) *[*[*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *,* *Any* *]*  *|* *None*) – [Recipe](../recipe/index.html.md#recipe) to load the datasets from. The value
    provided here should be either a path to a file, a recipe file
    that has been loaded using e.g. `yaml.safe_load()`, or an
    [`str`](https://docs.python.org/3/builtins/stdtypes.html#str) that can be loaded using `yaml.safe_load()`.
* **Return type:**
  [dict](https://docs.python.org/3/builtins/stdtypes.html#dict)

### Examples

See [Composing recipes](../notebooks/composing-recipes.html.md) notebook for example use cases.

* **Returns:**
  The recipe with the datasets. To convert the [`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict) to a
  [recipe](../recipe/index.html.md#recipe), use e.g. `yaml.safe_dump()`.
* **Return type:**
  [dict](https://docs.python.org/3/builtins/stdtypes.html#dict)
* **Raises:**
  [**RecipeError**](esmvalcore.exceptions.html.md#esmvalcore.exceptions.RecipeError) – Raised when a dataset is missing the `diagnostic` facet.
* **Parameters:**
  * **datasets** (*Iterable* *[*[*Dataset*](#esmvalcore.dataset.Dataset) *]*)
  * **recipe** (*Path* *|* [*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *|* [*dict*](https://docs.python.org/3/builtins/stdtypes.html#dict) *[*[*str*](https://docs.python.org/3/builtins/stdtypes.html#str) *,* *Any* *]*  *|* *None*)