Recipe output

This section describes the recipe_output submodule of the API (esmvalcore.experimental).

After running a recipe, output is returned by the run() method. Alternatively, it can be retrieved using the get_output() method.

>>> recipe_output = recipe.get_output()

recipe_output is a mapping of the individual tasks and their output filenames (data and image files) with a set of attributes describing the data.

>>> recipe_output
timeseries/script1:
  DataFile('tas_amsterdam_CMIP5_CanESM2_Amon_historical_r1i1p1_tas_1850-2000.nc')
  DataFile('tas_amsterdam_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000.nc')
  DataFile('tas_amsterdam_MultiModelMean_Amon_tas_1850-2000.nc')
  DataFile('tas_global_CMIP5_CanESM2_Amon_historical_r1i1p1_tas_1850-2000.nc')
  DataFile('tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000.nc')
  ImageFile('tas_amsterdam_CMIP5_CanESM2_Amon_historical_r1i1p1_tas_1850-2000.png')
  ImageFile('tas_amsterdam_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000.png')
  ImageFile('tas_amsterdam_MultiModelMean_Amon_tas_1850-2000.png')
  ImageFile('tas_global_CMIP5_CanESM2_Amon_historical_r1i1p1_tas_1850-2000.png')
  ImageFile('tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000.png')

map/script1:
  DataFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.nc')
  DataFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.nc')
  ImageFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.png')
  ImageFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.png')

Output is grouped by the task that produced them. They can be accessed like a dictionary.

>>> task_output = recipe_output['map/script1']
>>> task_output
map/script1:
  DataFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.nc')
  DataFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.nc')
  ImageFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.png')
  ImageFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.png')

The task output has a list of files associated with them, usually image (.png) or data files (.nc). To get a list of all files, use files().

>>> print(task_output.files)
(DataFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.nc'),
..., ImageFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.png'))

It is also possible to select the image (image_files()) files or data files (data_files()) only.

>>> for image_file in task_output.image_files:
>>>     print(image_file)
ImageFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.png')
ImageFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.png')

>>> for data_file in task_output.data_files:
>>>     print(data_file)
DataFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.nc')
DataFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.nc')

Working with output files

Output comes in two kinds, DataFile corresponds to data files in .nc format and ImageFile corresponds to plots in .png format (see below). Both object are derived from the same base class (OutputFile) and therefore share most of the functionality.

For example, author information can be accessed as instances of Contributor via

>>> output_file = task_output[0]
>>> output_file.authors
(Contributor('Andela, Bouwe', institute='NLeSC, Netherlands', orcid='https://orcid.org/0000-0001-9005-8940'),
 Contributor('Righi, Mattia', institute='DLR, Germany', orcid='https://orcid.org/0000-0003-3827-5950'))

And associated references as instances of Reference via

>>> output_file.references
(Reference('acknow_project'),)

OutputFile also knows about associated files

>>> data_file.citation_file
Path('.../tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000_citation.bibtex')
>>> data_file.data_citation_file
Path('.../tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000_data_citation_info.txt')
>>> data_file.provenance_svg_file
Path('.../tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000_provenance.svg')
>>> data_file.provenance_xml_file
Path('.../tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000_provenance.xml')

Working with image files

Image output uses IPython magic to plot themselves in a notebook environment.

>>> image_file = recipe_output['map/script1'].image_files[0]
>>> image_file

For example:

../_images/api_recipe_output.png

Using IPython.display, it is possible to show all image files.

>>> from IPython.display import display
>>>
>>> task = recipe_output['map/script1']
>>> for image_file in task.image_files:
>>>      display(image_file)

Working with data files

Data files can be easily loaded using xarray:

>>> data_file = recipe_output['timeseries/script1'].data_files[0]
>>> data = data_file.load_xarray()
>>> type(data)
xarray.core.dataset.Dataset

Or iris:

>>> cube = data_file.load_iris()
>>> type(cube)
iris.cube.CubeList

API reference

API for handing recipe output.

Classes:

DataFile(path, attributes)

Container for data output.

ImageFile(path, attributes)

Container for image output.

OutputFile(path, attributes)

Base container for recipe output files.

RecipeOutput(task_output[, session, info])

Container for recipe output.

TaskOutput(name, files)

Container for task output.

class esmvalcore.experimental.recipe_output.DataFile(path: str, attributes: Optional[dict] = None)[source]

Bases: esmvalcore.experimental.recipe_output.OutputFile

Container for data output.

Attributes:

authors

List of recipe authors.

caption

Return the caption of the file (fallback to path).

citation_file

Return path of citation file (bibtex format).

data_citation_file

Return path of data citation info (txt format).

kind

provenance_svg_file

Return path of provenance file (svg format).

provenance_xml_file

Return path of provenance file (xml format).

references

List of project references.

Methods:

create(path[, attributes])

Construct new instances of OutputFile.

load_iris()

Load data using iris.

load_xarray()

Load data using xarray.

property authors

List of recipe authors.

property caption

Return the caption of the file (fallback to path).

property citation_file

Return path of citation file (bibtex format).

classmethod create(path: str, attributes: Optional[dict] = None)esmvalcore.experimental.recipe_output.OutputFile

Construct new instances of OutputFile.

Chooses a derived class if suitable.

property data_citation_file

Return path of data citation info (txt format).

kind: Optional[str] = 'data'
load_iris()[source]

Load data using iris.

load_xarray()[source]

Load data using xarray.

property provenance_svg_file

Return path of provenance file (svg format).

property provenance_xml_file

Return path of provenance file (xml format).

property references

List of project references.

class esmvalcore.experimental.recipe_output.ImageFile(path: str, attributes: Optional[dict] = None)[source]

Bases: esmvalcore.experimental.recipe_output.OutputFile

Container for image output.

Attributes:

authors

List of recipe authors.

caption

Return the caption of the file (fallback to path).

citation_file

Return path of citation file (bibtex format).

data_citation_file

Return path of data citation info (txt format).

kind

provenance_svg_file

Return path of provenance file (svg format).

provenance_xml_file

Return path of provenance file (xml format).

references

List of project references.

Methods:

create(path[, attributes])

Construct new instances of OutputFile.

to_base64()

Encode image as base64 to embed in a Jupyter notebook.

property authors

List of recipe authors.

property caption

Return the caption of the file (fallback to path).

property citation_file

Return path of citation file (bibtex format).

classmethod create(path: str, attributes: Optional[dict] = None)esmvalcore.experimental.recipe_output.OutputFile

Construct new instances of OutputFile.

Chooses a derived class if suitable.

property data_citation_file

Return path of data citation info (txt format).

kind: Optional[str] = 'image'
property provenance_svg_file

Return path of provenance file (svg format).

property provenance_xml_file

Return path of provenance file (xml format).

property references

List of project references.

to_base64()str[source]

Encode image as base64 to embed in a Jupyter notebook.

class esmvalcore.experimental.recipe_output.OutputFile(path: str, attributes: Optional[dict] = None)[source]

Bases: object

Base container for recipe output files.

Use OutputFile.create(path=’<path>’, attributes=attributes) to initialize a suitable subclass.

Parameters
  • path (str) – Name of output file

  • attributes (dict) – Attributes corresponding to the recipe output

Attributes:

authors

List of recipe authors.

caption

Return the caption of the file (fallback to path).

citation_file

Return path of citation file (bibtex format).

data_citation_file

Return path of data citation info (txt format).

kind

provenance_svg_file

Return path of provenance file (svg format).

provenance_xml_file

Return path of provenance file (xml format).

references

List of project references.

Methods:

create(path[, attributes])

Construct new instances of OutputFile.

property authors

List of recipe authors.

property caption

Return the caption of the file (fallback to path).

property citation_file

Return path of citation file (bibtex format).

classmethod create(path: str, attributes: Optional[dict] = None)esmvalcore.experimental.recipe_output.OutputFile[source]

Construct new instances of OutputFile.

Chooses a derived class if suitable.

property data_citation_file

Return path of data citation info (txt format).

kind: Optional[str] = None
property provenance_svg_file

Return path of provenance file (svg format).

property provenance_xml_file

Return path of provenance file (xml format).

property references

List of project references.

class esmvalcore.experimental.recipe_output.RecipeOutput(task_output: dict, session=None, info=None)[source]

Bases: collections.abc.Mapping

Container for recipe output.

Parameters

task_output (dict) – Dictionary with recipe output grouped by task name. Each task value is a mapping of the filenames with the product attributes.

Methods:

from_core_recipe_output(recipe_output)

Construct instance from _recipe.Recipe output.

get(k[,d])

items()

keys()

read_main_log()

Read log file.

read_main_log_debug()

Read debug log file.

render([template])

Render output as html.

values()

write_html()

Write output summary to html document.

classmethod from_core_recipe_output(recipe_output: dict)[source]

Construct instance from _recipe.Recipe output.

The core recipe format is not directly compatible with the API. This constructor does the following:

  1. Convert config-user dict to an instance of Session

  2. Converts the raw recipe dict to RecipeInfo

Parameters

recipe_output (dict) – Output from _recipe.Recipe.get_product_output

get(k[, d])D[k] if k in D, else d.  d defaults to None.
items()a set-like object providing a view on D’s items
keys()a set-like object providing a view on D’s keys
read_main_log()str[source]

Read log file.

read_main_log_debug()str[source]

Read debug log file.

render(template=None)[source]

Render output as html.

templateTemplate

An instance of jinja2.Template can be passed to customize the output.

values()an object providing a view on D’s values
write_html()[source]

Write output summary to html document.

A html file index.html gets written to the session directory.

class esmvalcore.experimental.recipe_output.TaskOutput(name: str, files: dict)[source]

Bases: object

Container for task output.

Parameters
  • name (str) – Name of the task

  • files (dict) – Mapping of the filenames with the associated attributes.

Attributes:

data_files

Return a tuple of data objects.

image_files

Return a tuple of image objects.

Methods:

from_task(task)

Create an instance of TaskOutput from a Task.

property data_files

Return a tuple of data objects.

classmethod from_task(task)esmvalcore.experimental.recipe_output.TaskOutput[source]

Create an instance of TaskOutput from a Task.

Where task is an instance of esmvalcore._task.BaseTask.

property image_files

Return a tuple of image objects.