Recipe output

This section describes the recipe_output submodule of the API (esmvalcore.experimental).

After running a recipe, output is returned by the run() method. Alternatively, it can be retrieved using the get_output() method.

>>> recipe_output = recipe.get_output()

recipe_output is a mapping of the individual tasks and their output filenames (data and image files) with a set of attributes describing the data.

>>> recipe_output
timeseries/script1:
  DataFile('tas_amsterdam_CMIP5_CanESM2_Amon_historical_r1i1p1_tas_1850-2000.nc')
  DataFile('tas_amsterdam_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000.nc')
  DataFile('tas_amsterdam_MultiModelMean_Amon_tas_1850-2000.nc')
  DataFile('tas_global_CMIP5_CanESM2_Amon_historical_r1i1p1_tas_1850-2000.nc')
  DataFile('tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000.nc')
  ImageFile('tas_amsterdam_CMIP5_CanESM2_Amon_historical_r1i1p1_tas_1850-2000.png')
  ImageFile('tas_amsterdam_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000.png')
  ImageFile('tas_amsterdam_MultiModelMean_Amon_tas_1850-2000.png')
  ImageFile('tas_global_CMIP5_CanESM2_Amon_historical_r1i1p1_tas_1850-2000.png')
  ImageFile('tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000.png')

map/script1:
  DataFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.nc')
  DataFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.nc')
  ImageFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.png')
  ImageFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.png')

Output is grouped by the task that produced them. They can be accessed like a dictionary.

>>> task_output = recipe_output['map/script1']
>>> task_output
map/script1:
  DataFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.nc')
  DataFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.nc')
  ImageFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.png')
  ImageFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.png')

The task output has a list of files associated with them, usually image (.png) or data files (.nc). To get a list of all files, use files().

>>> print(task_output.files)
(DataFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.nc'),
..., ImageFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.png'))

It is also possible to select the image (image_files()) files or data files (data_files()) only.

>>> for image_file in task_output.image_files:
>>>     print(image_file)
ImageFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.png')
ImageFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.png')

>>> for data_file in task_output.data_files:
>>>     print(data_file)
DataFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.nc')
DataFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.nc')

Working with output files

Output comes in two kinds, DataFile corresponds to data files in .nc format and ImageFile corresponds to plots in .png format (see below). Both object are derived from the same base class (OutputFile) and therefore share most of the functionality.

For example, author information can be accessed as instances of Contributor via

>>> output_file = task_output[0]
>>> output_file.authors
(Contributor('Andela, Bouwe', institute='NLeSC, Netherlands', orcid='https://orcid.org/0000-0001-9005-8940'),
 Contributor('Righi, Mattia', institute='DLR, Germany', orcid='https://orcid.org/0000-0003-3827-5950'))

And associated references as instances of Reference via

>>> output_file.references
(Reference('acknow_project'),)

OutputFile also knows about associated files

>>> data_file.citation_file
Path('.../tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000_citation.bibtex')
>>> data_file.data_citation_file
Path('.../tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000_data_citation_info.txt')
>>> data_file.provenance_svg_file
Path('.../tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000_provenance.svg')
>>> data_file.provenance_xml_file
Path('.../tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000_provenance.xml')

Working with image files

Image output uses IPython magic to plot themselves in a notebook environment.

>>> image_file = recipe_output['map/script1'].image_files[0]
>>> image_file

For example:

Using IPython.display, it is possible to show all image files.

>>> from IPython.display import display
>>>
>>> task = recipe_output['map/script1']
>>> for image_file in task.image_files:
>>>      display(image_file)

Working with data files

Data files can be easily loaded using xarray:

>>> data_file = recipe_output['timeseries/script1'].data_files[0]
>>> data = data_file.load_xarray()
>>> type(data)
xarray.core.dataset.Dataset

Or iris:

>>> cube = data_file.load_iris()
>>> type(cube)
iris.cube.CubeList

API reference

API for handing recipe output.

Classes:

`DataFile`(path[, attributes])	Container for data output.
`DiagnosticOutput`(name, task_output[, title, ...])	Container for diagnostic output.
`ImageFile`(path[, attributes])	Container for image output.
`OutputFile`(path[, attributes])	Base container for recipe output files.
`RecipeOutput`(task_output[, session, info])	Container for recipe output.
`TaskOutput`(name, files)	Container for task output.

class esmvalcore.experimental.recipe_output.DataFile(path: str, attributes: Optional[dict] = None)[source]

Bases: esmvalcore.experimental.recipe_output.OutputFile

Container for data output.

Attributes:

`authors`	List of recipe authors.
`caption`	Return the caption of the file (fallback to path).
`citation_file`	Return path of citation file (bibtex format).
`data_citation_file`	Return path of data citation info (txt format).
`kind`
`provenance_xml_file`	Return path of provenance file (xml format).
`references`	List of project references.

Methods:

`create`(path[, attributes])	Construct new instances of OutputFile.
`load_iris`()	Load data using iris.
`load_xarray`()	Load data using xarray.

property authors: tuple: List of recipe authors.

property caption: str: Return the caption of the file (fallback to path).

property citation_file: Return path of citation file (bibtex format).

classmethod create(path: str, attributes: Optional[dict] = None) → esmvalcore.experimental.recipe_output.OutputFile

Construct new instances of OutputFile.

Chooses a derived class if suitable.

property data_citation_file: Return path of data citation info (txt format).

kind: Optional[str] = 'data'

load_iris()[source]: Load data using iris.

load_xarray()[source]: Load data using xarray.

property provenance_xml_file: Return path of provenance file (xml format).

property references: tuple: List of project references.

class esmvalcore.experimental.recipe_output.DiagnosticOutput(name, task_output, title=None, description=None)[source]

Bases: object

Container for diagnostic output.

Parameters

name (str) – Name of the diagnostic
title (str) – Title of the diagnostic
description (str) – Description of the diagnostic
task_output (list of TaskOutput) – List of task output.

class esmvalcore.experimental.recipe_output.ImageFile(path: str, attributes: Optional[dict] = None)[source]

Bases: esmvalcore.experimental.recipe_output.OutputFile

Container for image output.

Attributes:

`authors`	List of recipe authors.
`caption`	Return the caption of the file (fallback to path).
`citation_file`	Return path of citation file (bibtex format).
`data_citation_file`	Return path of data citation info (txt format).
`kind`
`provenance_xml_file`	Return path of provenance file (xml format).
`references`	List of project references.

Methods:

`create`(path[, attributes])	Construct new instances of OutputFile.
`to_base64`()	Encode image as base64 to embed in a Jupyter notebook.

property authors: tuple: List of recipe authors.

property caption: str: Return the caption of the file (fallback to path).

property citation_file: Return path of citation file (bibtex format).

classmethod create(path: str, attributes: Optional[dict] = None) → esmvalcore.experimental.recipe_output.OutputFile

Construct new instances of OutputFile.

Chooses a derived class if suitable.

property data_citation_file: Return path of data citation info (txt format).

kind: Optional[str] = 'image'

property provenance_xml_file: Return path of provenance file (xml format).

property references: tuple: List of project references.

to_base64() → str[source]: Encode image as base64 to embed in a Jupyter notebook.

class esmvalcore.experimental.recipe_output.OutputFile(path: str, attributes: Optional[dict] = None)[source]

Bases: object

Base container for recipe output files.

Use OutputFile.create(path=’<path>’, attributes=attributes) to initialize a suitable subclass.

Parameters

path (str) – Name of output file
attributes (dict) – Attributes corresponding to the recipe output

Attributes:

`authors`	List of recipe authors.
`caption`	Return the caption of the file (fallback to path).
`citation_file`	Return path of citation file (bibtex format).
`data_citation_file`	Return path of data citation info (txt format).
`kind`
`provenance_xml_file`	Return path of provenance file (xml format).
`references`	List of project references.

Methods:

create(path[, attributes])

Construct new instances of OutputFile.

property authors: tuple: List of recipe authors.

property caption: str: Return the caption of the file (fallback to path).

property citation_file: Return path of citation file (bibtex format).

classmethod create(path: str, attributes: Optional[dict] = None) → esmvalcore.experimental.recipe_output.OutputFile[source]

Construct new instances of OutputFile.

Chooses a derived class if suitable.

property data_citation_file: Return path of data citation info (txt format).

kind: Optional[str] = None

property provenance_xml_file: Return path of provenance file (xml format).

property references: tuple: List of project references.

class esmvalcore.experimental.recipe_output.RecipeOutput(task_output: dict, session=None, info=None)[source]

Bases: collections.abc.Mapping

Container for recipe output.

Parameters: task_output (dict) – Dictionary with recipe output grouped by task name. Each task value is a mapping of the filenames with the product attributes.

diagnostics

Dictionary with recipe output grouped by diagnostic.

Type: dict

info

The recipe used to create the output.

Type: RecipeInfo

session

The session used to run the recipe.

Type: Session

Methods:

`from_core_recipe_output`(recipe_output)	Construct instance from _recipe.Recipe output.
`get`(k[,d])
`items`()
`keys`()
`read_main_log`()	Read log file.
`read_main_log_debug`()	Read debug log file.
`render`([template])	Render output as html.
`values`()
`write_html`()	Write output summary to html document.

classmethod from_core_recipe_output(recipe_output: dict)[source]

Construct instance from _recipe.Recipe output.

The core recipe format is not directly compatible with the API. This constructor does the following:

Convert config-user dict to an instance of Session
Converts the raw recipe dict to RecipeInfo

Parameters: recipe_output (dict) – Output from _recipe.Recipe.get_product_output

get(k[, d]) → D[k] if k in D, else d. d defaults to None.

items() → a set-like object providing a view on D's items

keys() → a set-like object providing a view on D's keys

read_main_log() → str[source]: Read log file.

read_main_log_debug() → str[source]: Read debug log file.

render(template=None)[source]

Render output as html.

templateTemplate: An instance of jinja2.Template can be passed to customize the output.

values() → an object providing a view on D's values

write_html()[source]

Write output summary to html document.

A html file index.html gets written to the session directory.

class esmvalcore.experimental.recipe_output.TaskOutput(name: str, files: dict)[source]

Bases: object

Container for task output.

Parameters

name (str) – Name of the task
files (dict) – Mapping of the filenames with the associated attributes.

Attributes:

`data_files`	Return a tuple of data objects.
`image_files`	Return a tuple of image objects.

Methods:

from_task(task)

Create an instance of TaskOutput from a Task.

property data_files: tuple: Return a tuple of data objects.

classmethod from_task(task) → esmvalcore.experimental.recipe_output.TaskOutput[source]

Create an instance of TaskOutput from a Task.

Where task is an instance of esmvalcore._task.BaseTask.

property image_files: tuple: Return a tuple of image objects.