Recipe output¶
This section describes the recipe_output
submodule of the API (esmvalcore.experimental
).
After running a recipe, output is returned by the run()
method. Alternatively, it can be retrieved using the get_output()
method.
>>> recipe_output = recipe.get_output()
recipe_output
is a mapping of the individual tasks and their output
filenames (data and image files) with a set of attributes describing the
data.
>>> recipe_output
timeseries/script1:
DataFile('tas_amsterdam_CMIP5_CanESM2_Amon_historical_r1i1p1_tas_1850-2000.nc')
DataFile('tas_amsterdam_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000.nc')
DataFile('tas_amsterdam_MultiModelMean_Amon_tas_1850-2000.nc')
DataFile('tas_global_CMIP5_CanESM2_Amon_historical_r1i1p1_tas_1850-2000.nc')
DataFile('tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000.nc')
ImageFile('tas_amsterdam_CMIP5_CanESM2_Amon_historical_r1i1p1_tas_1850-2000.png')
ImageFile('tas_amsterdam_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000.png')
ImageFile('tas_amsterdam_MultiModelMean_Amon_tas_1850-2000.png')
ImageFile('tas_global_CMIP5_CanESM2_Amon_historical_r1i1p1_tas_1850-2000.png')
ImageFile('tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000.png')
map/script1:
DataFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.nc')
DataFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.nc')
ImageFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.png')
ImageFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.png')
Output is grouped by the task that produced them. They can be accessed like a dictionary.
>>> task_output = recipe_output['map/script1']
>>> task_output
map/script1:
DataFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.nc')
DataFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.nc')
ImageFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.png')
ImageFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.png')
The task output has a list of files associated with them, usually image
(.png
) or data files (.nc
). To get a list of all files, use
files()
.
>>> print(task_output.files)
(DataFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.nc'),
..., ImageFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.png'))
It is also possible to select the image (image_files()
) files or data files (data_files()
) only.
>>> for image_file in task_output.image_files:
>>> print(image_file)
ImageFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.png')
ImageFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.png')
>>> for data_file in task_output.data_files:
>>> print(data_file)
DataFile('CMIP5_CanESM2_Amon_historical_r1i1p1_tas_2000-2000.nc')
DataFile('CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_2000-2000.nc')
Working with output files¶
Output comes in two kinds, DataFile
corresponds to data
files in .nc
format and ImageFile
corresponds to plots
in .png
format (see below). Both object are derived from the same base class
(OutputFile
) and therefore share most of the functionality.
For example, author information can be accessed as instances of Contributor
via
>>> output_file = task_output[0]
>>> output_file.authors
(Contributor('Andela, Bouwe', institute='NLeSC, Netherlands', orcid='https://orcid.org/0000-0001-9005-8940'),
Contributor('Righi, Mattia', institute='DLR, Germany', orcid='https://orcid.org/0000-0003-3827-5950'))
And associated references as instances of Reference
via
>>> output_file.references
(Reference('acknow_project'),)
OutputFile
also knows about associated files
>>> data_file.citation_file
Path('.../tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000_citation.bibtex')
>>> data_file.data_citation_file
Path('.../tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000_data_citation_info.txt')
>>> data_file.provenance_svg_file
Path('.../tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000_provenance.svg')
>>> data_file.provenance_xml_file
Path('.../tas_global_CMIP6_BCC-ESM1_Amon_historical_r1i1p1f1_tas_1850-2000_provenance.xml')
Working with image files¶
Image output uses IPython magic to plot themselves in a notebook environment.
>>> image_file = recipe_output['map/script1'].image_files[0]
>>> image_file
For example:
Using IPython.display
, it is possible to show all image files.
>>> from IPython.display import display
>>>
>>> task = recipe_output['map/script1']
>>> for image_file in task.image_files:
>>> display(image_file)
Working with data files¶
Data files can be easily loaded using xarray
:
>>> data_file = recipe_output['timeseries/script1'].data_files[0]
>>> data = data_file.load_xarray()
>>> type(data)
xarray.core.dataset.Dataset
Or iris
:
>>> cube = data_file.load_iris()
>>> type(cube)
iris.cube.CubeList
API reference¶
API for handing recipe output.
Classes:
|
Container for data output. |
|
Container for image output. |
|
Base container for recipe output files. |
|
Container for recipe output. |
|
Container for task output. |
- class esmvalcore.experimental.recipe_output.DataFile(path: str, attributes: Optional[dict] = None)[source]¶
Bases:
esmvalcore.experimental.recipe_output.OutputFile
Container for data output.
Attributes:
List of recipe authors.
Return the caption of the file (fallback to path).
Return path of citation file (bibtex format).
Return path of data citation info (txt format).
Return path of provenance file (svg format).
Return path of provenance file (xml format).
List of project references.
Methods:
create
(path[, attributes])Construct new instances of OutputFile.
Load data using iris.
Load data using xarray.
- property authors: tuple¶
List of recipe authors.
Return the caption of the file (fallback to path).
- property citation_file¶
Return path of citation file (bibtex format).
- classmethod create(path: str, attributes: Optional[dict] = None) → esmvalcore.experimental.recipe_output.OutputFile¶
Construct new instances of OutputFile.
Chooses a derived class if suitable.
- property data_citation_file¶
Return path of data citation info (txt format).
- property provenance_svg_file¶
Return path of provenance file (svg format).
- property provenance_xml_file¶
Return path of provenance file (xml format).
- property references: tuple¶
List of project references.
- class esmvalcore.experimental.recipe_output.ImageFile(path: str, attributes: Optional[dict] = None)[source]¶
Bases:
esmvalcore.experimental.recipe_output.OutputFile
Container for image output.
Attributes:
List of recipe authors.
Return the caption of the file (fallback to path).
Return path of citation file (bibtex format).
Return path of data citation info (txt format).
Return path of provenance file (svg format).
Return path of provenance file (xml format).
List of project references.
Methods:
create
(path[, attributes])Construct new instances of OutputFile.
Encode image as base64 to embed in a Jupyter notebook.
- property authors: tuple¶
List of recipe authors.
Return the caption of the file (fallback to path).
- property citation_file¶
Return path of citation file (bibtex format).
- classmethod create(path: str, attributes: Optional[dict] = None) → esmvalcore.experimental.recipe_output.OutputFile¶
Construct new instances of OutputFile.
Chooses a derived class if suitable.
- property data_citation_file¶
Return path of data citation info (txt format).
- property provenance_svg_file¶
Return path of provenance file (svg format).
- property provenance_xml_file¶
Return path of provenance file (xml format).
- property references: tuple¶
List of project references.
- class esmvalcore.experimental.recipe_output.OutputFile(path: str, attributes: Optional[dict] = None)[source]¶
Bases:
object
Base container for recipe output files.
Use OutputFile.create(path=’<path>’, attributes=attributes) to initialize a suitable subclass.
- Parameters
Attributes:
List of recipe authors.
Return the caption of the file (fallback to path).
Return path of citation file (bibtex format).
Return path of data citation info (txt format).
Return path of provenance file (svg format).
Return path of provenance file (xml format).
List of project references.
Methods:
create
(path[, attributes])Construct new instances of OutputFile.
- property authors: tuple¶
List of recipe authors.
Return the caption of the file (fallback to path).
- property citation_file¶
Return path of citation file (bibtex format).
- classmethod create(path: str, attributes: Optional[dict] = None) → esmvalcore.experimental.recipe_output.OutputFile[source]¶
Construct new instances of OutputFile.
Chooses a derived class if suitable.
- property data_citation_file¶
Return path of data citation info (txt format).
- property provenance_svg_file¶
Return path of provenance file (svg format).
- property provenance_xml_file¶
Return path of provenance file (xml format).
- property references: tuple¶
List of project references.
- class esmvalcore.experimental.recipe_output.RecipeOutput(task_output: dict, session=None, info=None)[source]¶
Bases:
collections.abc.Mapping
Container for recipe output.
- Parameters
task_output (dict) – Dictionary with recipe output grouped by task name. Each task value is a mapping of the filenames with the product attributes.
Methods:
from_core_recipe_output
(recipe_output)Construct instance from _recipe.Recipe output.
get
(k[,d])items
()keys
()Read log file.
Read debug log file.
render
([template])Render output as html.
values
()Write output summary to html document.
- classmethod from_core_recipe_output(recipe_output: dict)[source]¶
Construct instance from _recipe.Recipe output.
The core recipe format is not directly compatible with the API. This constructor does the following:
Convert config-user dict to an instance of
Session
Converts the raw recipe dict to
RecipeInfo
- Parameters
recipe_output (dict) – Output from _recipe.Recipe.get_product_output
- get(k[, d]) → D[k] if k in D, else d. d defaults to None.¶
- items() → a set-like object providing a view on D’s items¶
- keys() → a set-like object providing a view on D’s keys¶
- render(template=None)[source]¶
Render output as html.
- template
Template
An instance of
jinja2.Template
can be passed to customize the output.
- template
- values() → an object providing a view on D’s values¶
- class esmvalcore.experimental.recipe_output.TaskOutput(name: str, files: dict)[source]¶
Bases:
object
Container for task output.
- Parameters
Attributes:
Return a tuple of data objects.
Return a tuple of image objects.
Methods:
from_task
(task)Create an instance of TaskOutput from a Task.
- property data_files: tuple¶
Return a tuple of data objects.
- classmethod from_task(task) → esmvalcore.experimental.recipe_output.TaskOutput[source]¶
Create an instance of TaskOutput from a Task.
Where task is an instance of esmvalcore._task.BaseTask.
- property image_files: tuple¶
Return a tuple of image objects.